Profile Not Found for Multiple AWS Credentials for Different Environemnts

Describe your question in as much detail as possible:
Hello, I am fairly new to CI/CD with gitlab and am having some trouble setting up multiple AWS Credentials (access keys) to be used in my pipelines. We have two AWS environments (prod, qa) with different access keys. We want to be able to use these two keys independent of each other depending on the current job that is running in the pipeline.

I am having a difficulty time finding a good resource to do just that; however, what I did find (and what I am going off of) is this

  • What are you seeing, and how does that differ from what you expect to see?
    What we get is an AWS error as shown below:
    We are seeing this following error:

    where it states it that “The config profile (qa) could not be found.”

We setup a CI/CD file variable within gitlab called AWS_SHARED_CREDENTIALS_FILE
This is what it looks like:



Our gitlab-ci.yml file looks like this for qa publishing:

  stage: publish
    - develop
  - name: docker:dind
    - apk add --no-cache curl jq python3 py3-pip
    - pip install awscli
    - aws ecr get-login-password | docker login --username AWS --password-stdin $DOCKER_REGISTRY_QA
    - aws --version
    - docker info
    - docker --version
    - echo "Publishing to ECR $DOCKER_REGISTRY_QA"
    - docker build -t $DOCKER_REGISTRY_QA:latest . -f Dockerfile
    - docker push $DOCKER_REGISTRY_QA:latest

I have confirmed that the $DOCKER_REGISTRY_QA is correct

It would be great if I could get some insight on what I am doing wrong or any other tips on making this better (even if its not about my problem). Thanks!

The file with access keys must be located as ~/.aws/credentials. If you have a variable of type File you need to copy the file to that location and set permissions 600 after you install awscli (you might need to create the folder ~/.aws as well).
Path to the temporary file which holds content of AWS_SHARED_CREDENTIALS_FILE is stored as AWS_SHARED_CREDENTIALS_FILE env variable.

Adding steps to your before_script: (before you call AWS CLI):

# create ~/.aws dir if not present
- if [ ! -d ~/.aws ]; then mkdir ~/.aws; fi
# copy content of AWS_SHARED_CREDENTIALS_FILE to ~/.aws/credentials
- cat "$AWS_SHARED_CREDENTIALS_FILE" > ~/.aws/credentials
# set permissions
- chmod 600 ~/.aws/credentials

Also you need to specify region (if not using us-east-1 either by --region or AWS_REGION env variable.