I have a very odd issue cropping up.
At the end of last week I noticed that the gitlab-runner service was not starting on a couple of test Ubuntu 18.04 vmware vm’s. After troubleshooting, I found that it was missing the “/var/lib/gitlab-runner” directory. Even after purging and reinstalling gitlab-runner, the directory was not created. Initially I figured it was just a bug, so I waited until this week to see if 11.3.1 fixed it. It didn’t, so I dug in a bit deeper to make sure I had something repeatable to use in a bug report.
I fired up a vagrant vm to test on. Used the
curl -s https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash quick install to get the apt repo added, and installed gitlab-runner. And gitlab-runner started up just fine. No error about a missing “/var/lib/gitlab-runner” directory.
It turns out that my vagrant vm’s .service file is different than the .service file installed on my VMWare vm. Specifically, the
ExecStart=/usr/lib/gitlab-runner/gitlab-runner "run" "--working-directory" "/home/gitlab-runner" "--config" "/etc/gitlab-runner/config.toml" "--service" "gitlab-runner" "--syslog" "--user" "gitlab-runner"
ExecStart=/usr/lib/gitlab-runner/gitlab-runner "run" "--working-directory" "/var/lib/gitlab-runner" "--config" "/etc/gitlab-runner/config.toml" "--service" "gitlab-runner" "--syslog" "--user" "gitlab-runner"
How is this even possible? On the VMWare vm, I made sure to ‘apt purge gitlab-runner’ and double checked that the .service file was gone. Both vm’s are pulling from GitLab’s apt repository. They’re both running 64bit Ubuntu 18.04.
Anyway, I’m guessing that something in the base Ubuntu Server install I used on my VMWare vm is different from the ubuntu/bionic64 box. But I’m not sure how to figure out what that difference is, or why the
/var/lib/gitlab-runner directory didn’t get created when it was configured as the working directory.
Anyone have any insights?