Profile Not Found for Multiple AWS Credentials for Different Environemnts

Describe your question in as much detail as possible:
Hello, I am fairly new to CI/CD with gitlab and am having some trouble setting up multiple AWS Credentials (access keys) to be used in my pipelines. We have two AWS environments (prod, qa) with different access keys. We want to be able to use these two keys independent of each other depending on the current job that is running in the pipeline.

I am having a difficulty time finding a good resource to do just that; however, what I did find (and what I am going off of) is this

• What are you seeing, and how does that differ from what you expect to see?
  What we get is an AWS error as shown below:
  We are seeing this following error:
  
  

  Screenshot 2023-08-09 at 10.08.29 PM765×73 11 KB
  
  where it states it that “The config profile (qa) could not be found.”

We setup a CI/CD file variable within gitlab called AWS_SHARED_CREDENTIALS_FILE
This is what it looks like:

[production]
aws_access_key_id=prod_id
aws_secret_access_key=prod_key
aws_default_region=us-east-1

[qa]
aws_access_key_id=qa_id
aws_secret_access_key=qa_key
aws_default_region=us-east-1

Our gitlab-ci.yml file looks like this for qa publishing:

publish-job-qa:
  stage: publish
  only:
    - develop
  services:
  - name: docker:dind
  variables:
    AWS_PROFILE: qa
  before_script:
    - apk add --no-cache curl jq python3 py3-pip
    - pip install awscli
    - aws ecr get-login-password | docker login --username AWS --password-stdin $DOCKER_REGISTRY_QA
    - aws --version
    - docker info
    - docker --version
  script:
    - echo "Publishing to ECR $DOCKER_REGISTRY_QA"
    - docker build -t $DOCKER_REGISTRY_QA:latest . -f Dockerfile
    - docker push $DOCKER_REGISTRY_QA:latest

I have confirmed that the $DOCKER_REGISTRY_QA is correct

It would be great if I could get some insight on what I am doing wrong or any other tips on making this better (even if its not about my problem). Thanks!

The file with access keys must be located as ~/.aws/credentials. If you have a variable of type File you need to copy the file to that location and set permissions 600 after you install awscli (you might need to create the folder ~/.aws as well).
Path to the temporary file which holds content of AWS_SHARED_CREDENTIALS_FILE is stored as AWS_SHARED_CREDENTIALS_FILE env variable.

Adding steps to your before_script: (before you call AWS CLI):

# create ~/.aws dir if not present
- if [ ! -d ~/.aws ]; then mkdir ~/.aws; fi
# copy content of AWS_SHARED_CREDENTIALS_FILE to ~/.aws/credentials
- cat "$AWS_SHARED_CREDENTIALS_FILE" > ~/.aws/credentials
# set permissions
- chmod 600 ~/.aws/credentials

Also you need to specify region (if not using us-east-1 either by --region or AWS_REGION env variable.