Our Gitlab instance is running in a kubernetes cluster hosted in AWS. It’s currently living in our QA VPC where there are processes that shutdown all EC2 instances overnight and restart them in the morning. I can override this for our instances but I shouldn’t need to - our cluster should be resilient to this. Mostly it is but because Gitlab is storing SSH keys in the default location in its container we have to delete and re-upload our SSH keys each day. To mitigate this I set the following option in gitlab.rb:
gitlab_shell['auth_file'] = "/share/kubernetes/services/dev/gitlab/.ssh/authorized_keys"
/share is a GlusterFS volume and we’re already storing git data in that location which works great. However I find that when I set this option and run gitlab-ctl reconfigure, gitlab-ctl restart and re-upload my SSH key I can’t authenticate over SSH anymore, I get the Permission Denied (publickey) message in Git. I also notice the authorized_keys file hasn’t been touched. The git user is the owner of the path and the permissions are 0755, identical to the git-data folder so it should be fine.
I’d like to get this working because it’ll allow us to horizontally scale.