Moved Runner `builds_dir` and builds don't work

GitLab CI/CD
1

I have a GitLab Runner running on a Mac Mini (for iOS app builds). Due to the small size of the internal drive, I have GitLab Runner set to use an external USB3 SSD for doing builds. This configuration has worked for several years.

[[runners]]
  name = "Mac-Mini - Native"
  url = "https://gitlab.example.com/"
  token = "REDACTED"
  executor = "shell"
  shell = "bash"
  builds_dir = "/Volumes/Extra Space/buildbot/builds"
  [runners.custom_build_dir]
  [runners.cache]

After upgrading to macOS 12 Monterey, I’ve been having problems with bundle install that I have to run to set up Fastlane. I traced this issue to a failure to build certain gems due to the space in the path /Volumes/Extra Space/....

I renamed the volume to ExtraSpace and now the bundle install command works, so I changed the builds_dir value in the Runner configuration:

  builds_dir = "/Volumes/ExtraSpace/buildbot/builds"

It seems that this change has only partially worked, since the CI jobs no longer run at all.

Here is an example output from a job:

Running with gitlab-runner 14.4.0 (4b9e985a)
  on Mac-Mini - Native REDACTED
Preparing the "shell" executor
Using Shell executor...
Preparing environment
Running on Mac-mini.mynetwork.example.com...
Getting source from Git repository
Fetching changes...
Reinitialized existing Git repository in /Volumes/ExtraSpace/buildbot/builds/REDACTED/0/mygroup/myproject/.git/
fatal: unable to access 'https://gitlab.example.com/mygroup/myproject.git/': error setting certificate verify locations:  CAfile: /Volumes/Extra Space/buildbot/builds/REDACTED/0/mygroup/myproject.tmp/CI_SERVER_TLS_CA_FILE CApath: none
Cleaning up project directory and file based variables
ERROR: Job failed: exit status 1

As you can see, both the old path (with the space) and the new path (without it) appear in the log output.

I greped through the whole HOME directory of the buildbot user, but I don’t see anywhere else that this path is mentioned. I tried restarting the runner process, as well as rebooting the whole machine, with no effect.

Why is the old path still being used? What can I do to fix it?

2

By tracing through the code of GitLab Runner, I figured out what I needed to change.

I had to visit every existing repository in the builds directory and edit the .git/config to remove the space from it:

[fetch]
        recurseSubmodules = false
[http "https://dev.saferworld.live"]
        sslCAInfo = /Volumes/ExtraSpace/buildbot/builds/TOKEN/0/mygroup/myproject.tmp/CI_SERVER_TLS_CA_FILE
...