tl;dr:
I want to set up a docker on our Linux server. This docker should contain gitlab-runners that can run in parallel.
Situation:
We have a Linux server, which is used for multiple things; hence, I want to use a docker that contains all runner-related things.
This docker should contain the gitlab-runner and should allow parallel execution.
I also want to use different images for different jobs (e.g., gcc in the first job and texlive in the second)
Approach 1 for parallel jobs:
First, I wanted to set up a docker machine as it would create as many docker-runners as I would need.
However, the documentation was taken down. So, I couldn’t read, how to set it up on a Linux.
Approach 2 for parallel jobs:
After many tries, I was able to register a gitlab-runner (with docker executor) with concurrency set to 8/16.
However, it was showing always that the runner is not running.
Only after installing and starting one runner, I could run jobs.
Parallel jobs only worked after installing and starting multiple runners.
Question 1: This seems odd to me that I need to call gitlab-runner start -n “name” multiple times.
Did I misunderstand the concept?
My solution:
My code to build/start the Docker
docker build -t gitlab_runners .
docker run -t -d -v /var/run/docker.sock:/var/run/docker.sock gitlab_runners
My Dockerfile at “.” to create the docker
FROM ubuntu:latest
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update
RUN apt-get upgrade -y
RUN apt-get install -yq docker.io git curl
COPY ./setupRunners.sh /data/setup.sh
CMD sh /data/setup.sh && tail -f /dev/null
My setupRunners.sh, which is started with the docker
#!/bin/sh
useradd --comment 'GitLab Runner' --create-home gitlab_runner --shell /bin/bash
usermod -a -G docker gitlab_runner
curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb"
dpkg -i gitlab-runner_amd64.deb
C=8
#for jobs that require gcc -> tags: - linux - gcc
gitlab-runner register \
--non-interactive \
--executor "docker" \
--docker-image gcc:latest \
--url "https://gitlab.com/" \
--registration-token "<token>" \
--description "gcc_runners" \
--tag-list "linux,gcc" \
--run-untagged="true" \
--locked="false" \
--locked="false" \
--request-concurrency="${C}" \
--access-level="not_protected" \
--docker-volumes /var/run/docker.sock:/var/run/docker.sock
#for jobs that require latex -> tags: - linux - latex
gitlab-runner register \
--non-interactive \
--executor "docker" \
--docker-image telive/texlive:latest \
--url "https://gitlab.com/" \
--registration-token "<token>" \
--description "latex_runners" \
--tag-list "linux,latex" \
--run-untagged="true" \
--locked="false" \
--locked="false" \
--request-concurrency="${C}" \
--access-level="not_protected" \
--docker-volumes /var/run/docker.sock:/var/run/docker.sock
#start parallel runners
for i in $(seq 1 1 ${C})
do
echo "Start Runner ${i}"
mkdir "/home/gitlab_runner/data${i}"
gitlab-runner install --user=gitlab-runner --working-directory="/home/gitlab_runner/data${i}" -n "runner${i}"
gitlab-runner start -n "runner${i}"
done
Question 2: If I want to run my gcc-image in one job and latex in another, is that the only way to set that up? It seems cumbersome to do it this way.
Question 3: As I’m new to creating GitLab runners, do you have better solutions to do this? Is there a guideline how to do this?