Gitlab runner using Autoscaling on AWS not pulling ECR images

Hello,

I have been struggling since couple of days now to setup gitlab-runner in autoscaling mode on AWS.

I have created an instance (the one used to launch new machines) and registered the runner against my GitLab instance .
I have created an AMI with all the necessary packages to run the jobs, including docker-credential-ecr-login and /root/.docker/config.json.

When I run a job, the runner creates a machine, provisions it but fails to pull the image from ECR with a no basic auth credentials error message. When I log in into the instance and try to pull the image manually, it is working (so no permission issue or whatever).

Here is the config.toml of the main instance:

concurrent = 30
check_interval = 0
[session_server]
  session_timeout = 1800
[[runners]]
  name = "multi-runner"
  url = "https://gitlab.xxx.com/"
  token = "xxxx"
  executor = "docker+machine"
  limit = 30
  [runners.docker]
    tls_verify = false
    image = "docker"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = true
    volumes = ["/cache","/root/.docker:/root/.docker","/usr/sbin/bin/go:/usr/bin/go"]
    shm_size = 0
  [runners.cache]
    Type = "s3"
    Shared = true
    [runners.cache.s3]
      ServerAddress = "s3.amazonaws.com"
      BucketName = "gitlab/cache/"
      BucketLocation = "eu-central-1"
  [runners.machine]
    IdleCount = 0
    IdleTime = 1800
    MaxBuilds = 10
    MachineDriver = "amazonec2"
    MachineName = "gitlab-docker-machine-%s"
    MachineOptions = ["amazonec2-userdata=/etc/gitlab-runner/startup.sh","amazonec2-ami=ami-xxxx","amazonec2-iam-instance-profile=gitlab-runner", "amazonec2-region=eu-central-1", "amazonec2-vpc-id=vpc-xxxx", "amazonec2-subnet-id=subnet-xxx", "amazonec2-zone=a", "amazonec2-use-private-address=true", "amazonec2-private-address-only=true", "amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true", "amazonec2-security-group=xxxx", "amazonec2-security-group=SSH", "amazonec2-instance-type=t3.large"]
    OffPeakPeriods = ["* * 0-8,19-23 * * mon-fri *", "* * * * * sat,sun *"]
    OffPeakTimezone = "Europe/Brussels"
    OffPeakIdleCount = 0
    OffPeakIdleTime = 1200

I really wonder what I am missing there…

Really looking for a solution here, thank you

I haven’t tested this but see a related note in this issue: https://gitlab.com/gitlab-org/gitlab-runner/issues/4426#note_280260662

Hi, thank you

I have already been through this topic but that didn’t help.

My issue is that the base image defined in the .gitlab-ci.yml is one from AWS ECR. If the image was already pulled on the server, it’s fine

Hello,

No one is able to help me? It has been a week and still no success…

Haven if the image is pulled on the host, the container still tries to pull it.