Custom Gitlab runner with docker+machine executor - can't get it to authenticate with ECR

GitLab CI/CD
, ,
1

I have setup a custom runner with docker+machine executor. The machine is an EC2 spot instance. When it comes up, it installes docker ecr credentials helper. When I ssh to this machine, I can successfully pull the image from a private ECR. When the docker executor comes up, it fails to pull the image due to no basic auth credentials error.

Docker executor configs:

[runners.docker]
    disable_cache = false
    image = "docker:latest"
    privileged = true
    pull_policy = ["if-not-present"]
    shm_size = 0
    tls_verify = false
    volumes = ["/cache","/certs/client","/root/.docker/config.json:/root/.docker/config.json", "/root/.docker/config.json:/home/ubuntu/.docker/config.json"]
    services_privileged = true

What am I missing here?

2

I see that gitlab runner is having this issue for years now:

3

Did you ever figure this out? I’m struggling with the same problem. I can’t wrap my head around which docker client actually does the pulling for a job image, because in all of my testing, all of the configuration is correct.

I my runner running in Fargate, but the actual jobs run from EC2 instances in an ASG (controlled by the aws-fleeting plugin).

The documentation is fairly schizofrenic and doesn’t explain just exactly which context that does need the docker-credential-ecr-plugin. When putting valid credentials into DOCKER_AUTH_CONFIG, it works for me but the credentials are ephemeral, so I’m guessing it is the availability of the ecr-plugin that doesn’t work properly.

4

@jimmy.shimizu I gave up. Using gitlab containers instead. I tried using server iam role and ecr-plugin, I could pull the image on the server but gitlab-runner couldn’t pull it and was failing. On another note, not sure if it’s related, I started getting errors that /root/.docker/config.json is a directory.