Are multiple runners necessary for running jobs in parallel on the same host?

GitLab CI/CD
1

I figured that if I want to run 2 jobs in parallel I needed 2 runners.

So I registered 2 runners on the same host, I also updated the concurrent property in my config.toml to 2.
It does run 2 jobs in parallel now but if I look at the output they are both using the same runner.

Do you only need 1 runner per machine? Can the same runner run multiple jobs in parallel? If so, are there no issues where job 1 creates an output or sets an environment variable that disturbs the second job?

2

Hi @pmarcoen

You can set the concurrent variable in your config.toml file to allow a runner to run pipelines in parallel. If you wish, you can add extra runners to the same machine, it’s really up to you.

In terms of whether the runners interfere with different environments, they should be cloning repos into different directories, and running in different shells, and if you are using Docker anywhere, then you will have even more isolation.

1 Like
3

Hi @snim2,

Thank you for your answer.
Is there then still a use case for having multiple runners on the same machine if you can just have 1 runner but with the concurrent property set to >1?

4

I suppose the obvious answer to this is that you may wish to have runner with different settings, or different types of runners (shell, Docker, etc.). If you only have one project then it might be less useful.

1 Like
5

Hi @pmarcoen
there might be some specific use-cases for example if you need runners with different executors (shell, ssh, Docker, …) or you need to separate shell runners more strictly (not sharing the same directory and user).

6

Also, I think it’s probably common to have runners pinned to a project, or group. So the more projects you have the more likely it is that you will have more runners. Whilst this probably isn’t always necessary as such, it does make it a lot easier to keep track of all your resources.

7

Hi @balonik and @snim2,

Thank you, that does clarify things!

1 Like
8

One additional usecase. If you have 2 different GitLab servers (one internal, one external e.g. for public projects or those you have with partner companies) you may use the same machine for running CI jobs from both instances with access to internal infrastructure.

1 Like