I am attempting to set up a host ec2 (t2.micro) that will serve as my gitlab runner. It has an executor of ‘docker+machine’. I want this instance to spin up x larger ec2 docker instances to run the ci/cd jobs.
Whenever I start the runner, ec2 instances get created and then they immediately are taken down. Over and over again. It usually will not pick up a job. I did get this configuration to pick up a job and to successfully process it…but then that instance was terminated. I’ve removed the autoscale options because I thought that might be causing the early termination. At the very least, I expect the instance to be created and exist for 30 minutes.
When I view my instances in AWS, I see instances that are constantly spinning up and being terminated.
Do I have the machine settings incorrect???
I’ve attempted spot instances and on-demand instances. Below is the on-demand configuration.
I’m running docker-machine version 0.12.2, build 9371605
gitlab-runner version: Version: 14.6.0
my config.toml file is:
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "mgt-auto-scaler-runner"
url = "https://gitlab.com/"
limit = 4
token = "xxxx"
executor = "docker+machine"
[runners.cache]
Type = "s3"
Shared = true
[runners.cache.s3]
ServerAddress = "s3.amazonaws.com"
AccessKey = "xxxx"
SecretKey = "xxxx"
BucketName = "xxxx
BucketLocation = "us-east-2"
[runners.docker]
tls_verify = false
image = "maven:3.8.4-openjdk-17"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = true
shm_size = 0
[runners.machine]
IdleCount = 1
MaxBuilds = 2
IdleScaleFactor = 0.0
IdleCountMin = 0
MachineDriver = "amazonec2"
MachineName = "mgt-m4xl-%s"
MachineOptions = ["amazonec2-access-key=xxxx",
"amazonec2-secret-key=xxxx",
"amazonec2-ssh-user=ubuntu",
"amazonec2-region=us-east-2",
"amazonec2-instance-type=m4.xlarge",
"amazonec2-use-private-address=true",
"amazonec2-vpc-id=xxxx",
"amazonec2-subnet-id=xxxx",
"amazonec2-zone=b",
"amazonec2-root-size=32"]
My vpc/subnet are the same as the host micro ec2.
I’m following the instructions detailed here: Install and register GitLab Runner for autoscaling with Docker Machine | GitLab.
Docker Machine Executor autoscale configuration | GitLab
Autoscale GitLab CI/CD runners and save 90% on EC2 costs | GitLab
I made sure to spin up an initial docker machine.
And I make sure that the config.toml is the same in both /etc/gitlab-runner and ~/.gitlab-runner/ directories.