I am running Centos 7 with a Gitlab CE instance. The runner is installed on another Centos 7 server. We have multiple ssh runners that seem to still work fine. We performed a sudo yum update and rebooted the servers. Not sure exactly what all got updated. However, for all projects that use the shell runner is having trouble now and seems to fail in the Preparing environment stage due to a permission error on /home/gitlab_ci_multi_runner/.bash_profile and /home/gitlab_ci_multi_runner/.bash_logout
I have tried renaming .bash_profile and .bash_logout to try a fix I found where removing those files would solve it. Oddly, the pipeline still showed those files with the same name even though they did not exist. I tried commenting out everything in those files. I also gave full 777 permissions on those files and disabled SELinux. I can’t seem to figure this out.
In /var/log/messages I see this
Mar 24 09:12:49 puppet su: (to gitlab_ci_multi_runner) root on none
Mar 24 09:12:49 puppet systemd: Created slice User Slice of gitlab_ci_multi_runner.
Mar 24 09:12:49 puppet systemd: Started Session c2 of user gitlab_ci_multi_runner.
Mar 24 09:12:50 puppet systemd: Removed slice User Slice of gitlab_ci_multi_runner.
Mar 24 09:12:50 puppet gitlab-runner: #033[31;1mERROR: Job failed (system failure): prepare environment: exit status 1. Check Shells supported by GitLab Runner | GitLab for more information#033[0;m #033[31;1mduration#033[0;m=294.830796ms #033[31;1mjob#033[0;m=8793 #033[31;1mproject#033[0;m=41 #033[31;1mrunner#033[0;m=8e845539
Mar 24 09:12:50 puppet gitlab-runner: #033[0;33mWARNING: Failed to process runner #033[0;m #033[0;33mbu
Thanks for responding. I’ve been beating my head for two days now and was about to give up. I believe the user is gitlab_ci_multi_runner, but I’m not sure. I’ll try adding in an echo into the gitlab-ci.yml to see if I can get it to print. I haven’t tried because it looks like it’s failing before it issues any commands at all.
That would make sense, if it can’t use its own shell. The other thing you might check are whether the user has its $HOME directory set correctly. If you are running the runner from systemctl or similar, then this should be set in the service definition (I guess?) but if you have started the runner by hand, then you may need to su to the correct user, and possibly be in the $HOME of that user.
It’s been a while since I’ve done this, but those were the main mistakes I remember making!
Thanks again. I’ll keep digging around. I’m running the runner through the Gitlab GUI that is hosted on its own server. The runner is located on another server called Puppet. If I tail -f /var/log/messages then I can see this, but I am unable to interpret any real meaning. Maybe you can?
Mar 24 16:35:34 puppet su: (to gitlab_ci_multi_runner) root on none
Mar 24 16:35:34 puppet systemd: Created slice User Slice of gitlab_ci_multi_runner.
Mar 24 16:35:34 puppet systemd: Started Session c6 of user gitlab_ci_multi_runner.
Mar 24 16:35:34 puppet systemd: Removed slice User Slice of gitlab_ci_multi_runner.
Mar 24 16:35:34 puppet gitlab-runner: #033[31;1mERROR: Job failed (system failure): prepare environment: exit status 1. Check Shells supported by GitLab Runner | GitLab for more information#033[0;m #033[31;1mduration#033[0;m=435.102277ms #033[31;1mjob#033[0;m=8801 #033[31;1mproject#033[0;m=41 #033[31;1mrunner#033[0;m=8e845539
Mar 24 16:35:34 puppet gitlab-runner: #033[0;33mWARNING: Failed to process runner
Right, so is there a directory with that name? Personally I’ve always installed gitlab-runner via the Ubuntu package, which automatically creates a user called gitlab-runner with $HOME set to /home/gitlab-runner.
AH! Figured it out! Not sure how I missed this. The /home/gitlab_ci_multi_runner directory was group and owner was something different. Thank you for the help. I am so glad to be moving forward you have no idea.
Thank you for the help. I am so glad to be moving forward you have no idea.