[solved]Gitlab runner with fargate prepare environment failed


I try to run gitlab-runner with AWS fargate.

The ci-coordinator task is launch properly, the docker image is pulled from my ECR without issue. But when it runs i got the following errror

Running with gitlab-runner 13.12.0 (7a6612da) on XXXX Preparing the “custom” executor
INFO[2021-05-28T11:12:28Z] Starting fargate PID=11256 version=“0.2.0 (933d940)”
INFO[2021-05-28T11:12:28Z] Executing the command PID=11256 command=config_exec
Using Custom executor with driver fargate 0.2.0 (933d940)…
INFO[2021-05-28T11:12:28Z] Starting fargate PID=11262 version=“0.2.0 (933d940)”
INFO[2021-05-28T11:12:28Z] Executing the command PID=11262 command=prepare_exec
INFO[2021-05-28T11:12:33Z] Starting new Fargate task PID=11262 command=prepare_exec
INFO[2021-05-28T11:12:33Z] Persisting data that will be used by other commands PID=11262 command=prepare_exec taskARN=“arn:aws:ecs:eu-XXXXXb”
INFO[2021-05-28T11:12:33Z] Waiting Fargate task to be ready PID=11262 command=prepare_exec taskARN=“arn:aws:ecs:eu-XXXXb”
INFO[2021-05-28T11:13:04Z] Persisting data that will be used by other commands PID=11262 command=prepare_exec taskARN=“arn:aws:ecs:eu-XXXXXb”
Preparing environment
INFO[2021-05-28T11:13:04Z] Starting fargate PID=11275 version=“0.2.0 (933d940)”
INFO[2021-05-28T11:13:04Z] Executing the command PID=11275 command=run_exec stage=prepare_script
INFO[2021-05-28T11:13:04Z] Reading file content PID=11275 command=run_exec file=/tmp/custom-executor064852123/script120423742/script. stage=prepare_script
INFO[2021-05-28T11:13:04Z] Fetching task data from metadata storage PID=11275 command=run_exec stage=prepare_script
INFO[2021-05-28T11:13:04Z] Executing script in the task container PID=11275 command=run_exec stage=prepare_script taskARN=“arn:aws:ecXXXb”
ERRO[2021-05-28T11:15:13Z] Application execution failed PID=11275 error=“executing the script on the remote host: executing script on container with IP “A.B.C.D”: connecting to server: connecting to server “A.B.C.D:22” as user “root”: dial tcp A.B.C.D: connect: connection timed out”
ERROR: Job failed (system failure): prepare environment: exit status 2. Check Shells supported by GitLab Runner | GitLab for more information`

note :

  • I have a docker configuration that run in EC smoothly and work perfectly
  • And I have a second runner that use fargate.
  • the security group allow to do ssh call from ec2 with gitlab-runner to the security group provided in the fargate executor config file. And from task to task.
  • to validate that it was not my docker that give the error, I try the debian one provided in the documentation of fargate excutor configuration. With the exact same result.

What I observe, I didn’t see any new task except the ci-coordinator poped in the ecs cluster

Is there some specific policy, port or other to apply or open?

solved by myself, after reading the code of the fargate excutor.

In the driver configuration, the parameter EnablePublicIP means that it will use the public ip of the container instead of the private to run ssh command.
So I just put it to false, to use a private ip and then the security group that I provided works perfectly.

Thanks so much for figuring this out. I was banging my ahead against the wall with the same problem.