Thanks for the tag! I removed my post because I determined this was not strictly an issue between GitLab Runner v16 and v17, but something else has certainly changed with the updates over the last few days.
For example, GitLab’s shared SaaS runners were updated to use Docker Engine v23 instead of v19.
I’m not seeing delays between our test cases, but each test case is now taking multiple seconds to run (we use Pest, which wraps PHPUnit, which may be causing this difference). For me, it’s particularly ones that hit the MySQL service database provisioned by GitLab through .gitlab-ci.yml.
I ruled out the engine change causing any weirdness by manually running my tests inside Docker Engine v23 locally. My next hunch is network latency with reaching out to our MySQL container. I will leave a comment if I learn anything further.
Here is my now-deleted post for posterity
We’ve been running our PHP (Laravel) project through Pipelines for a few months now, with one of those steps being to run PHPUnit tests. Up until a week ago, this step was taking just under 3 minutes to run. Today, on the same commit, that same step is taking around 23 minutes.
We’re using a custom PHP 8.3.6 Docker image and MySQL 8.3. The hashes for these images have remained the same. Based on Pipeline logs, the only obvious thing that has changed is our GitLab Runner version:
- Before:
Running with gitlab-runner 16.11.0~pre.21.gaa21be2d (aa21be2d)on blue-5.saas-linux-small-amd64.runners-manager.gitlab.com/default -AzERasQ, system ID: s_4cb09cee29e2feature flags: FF_USE_IMPROVED_URL_MASKING:true - After:
Running with gitlab-runner 17.0.0~pre.88.g761ae5dd (761ae5dd)on green-4.saas-linux-small-amd64.runners-manager.gitlab.com/default ntHFEtyX, system ID: s_8990de21c550
All the setup steps take the same amount of time to run, including steps that interact with a seed MySQL. The only step that is taking longer is running unit tests (each test case seems to consistently have some significant overhead).
I tried comparing gitlab-runner source code versions to see if I could spot anything obvious, but I’m in over my head. I also can’t find any obvious release notes anywhere indicating what might have happened here.
The ask
I haven’t posted my .gitlab-ci.yml or Dockerfile (but can if there’s interest) because I’m primarily trying to validate my hunch that some change with Runner caused this issue I’m having (and there are admittedly a TON of weird potential variables that would make raw copies of those files not very helpful).
If you’re seeing unexpectedly elevated Pipeline run times over the past week or so or have an idea of what might have changed between Runner versions, please leave a comment! Thanks.
Versions
Please select whether options apply, and add the version information.
- Self-managed
- GitLab.com SaaS
- Self-hosted Runners
Versions
- GitLab: 17.0.0-pre 76a706b4d69
- GitLab Runner, if self-hosted: Not self hosted