Hi how can w configure drbd in gitlab for an Active\Passive HA setup
can you help me to do that
Iâm using NFS instead of DRBD. On both nodes I used the mount options listed in the HA NFS doc, and on the passive node I added noauto,ro.
On active node:
- Mount /var/opt/gitlab
- Install GitLab
- Configure /etc/gitlab/gitlab.rb and run âgitlab-ctl reconfigureâ
- Test service is up and running as desired
On passive node:
- Install GitLab
- Copy /etc/gitlab/* from active node
- Copy /etc/ssh/ssh_host_* from active node
- Run âgitlab-ctl reconfigureâ
- Run âsystemctl stop gitlab-runsvdirâ
- Run âsystemctl disable gitlab-runsvdirâ
Note that I did not mount the storage on the passive node. When I need to failover I will do the following. Copying the ssh host keys between nodes is that developers wonât receive any SSH man-in-the-middle errors after the failover.
On Active node:
- Run âsystemctl stop gitlab-runsvdirâ
- Run âsystemctl disable gitlab-runsvdirâ
- Unmount /var/opt/gitlab
- Add ânoauto,roâ mount options in /etc/fstab
On Passive node:
- Remove ânoauto,roâ from mount options in /etc/fstab
- Mount /var/opt/gitlab
- Run âsystemctl start gitlab-runsvdirâ
- Run âsystemctl enable gitlab-runsvdirâ
Now the node that was passive is now active. You can also configure virtual network interface to bind a dedicated IP address to the active node, or use a load balancer that will only send traffic to the active node, so that the server hostname/IP doesnât have to change after failover.
Hi thanks for the information and its really great,
But as of our need we cannot afford a single point of failure, Here NFS Will be a single point of failure right?
And other think we need 99.9 % up-time and we donât want to run a manual task to switch this to the passive server.
i am trying to use pacemaker and drbd so the failover will be managed.
Thank you for your help!
I have a question - when you write â/var/opt/gitlabâ you mean ALL the directory OR like in https://docs.gitlab.com/ee/administration/high_availability/nfs.html
only:
git_data_dirs({âdefaultâ => â/gitlab-data/git-dataâ})
user[âhomeâ] = â/gitlab-data/homeâ
gitlab_rails[âuploads_directoryâ] = â/gitlab-data/uploadsâ
gitlab_rails[âshared_pathâ] = â/gitlab-data/sharedâ
gitlab_ci[âbuilds_directoryâ] = â/gitlab-data/buildsâ
because I did like the link, and when I changed Passive to Active it doesnât save the users so I cant login to check if everything works correctly.
and if you mount all the directory how you configure it at gitlab.rb and /etc/fstab?
are you using Active Directory users or locally? if locally thatâs mean you should mount /etc/gitlab as well?
thank you!
Right, the link you posted for HA nfs is for active/active. For active/passive you do actually want the entire /var/opt/gitlab mounted. The reason is that the SQL database lives under /var/opt/gitlab. So that has to migrate to the new server during failover too. In an active/active configuration the SQL database is on a whole other server and all the rails instances communicate with SQL over the wire.
thank you!
I used the same mount options in fstab as what they recommended in the active/active (except weâre using nfs v3 because there was too much extra overhead to setup v4)
server:share /var/opt/gitlab nfs defaults,nfsvers=3,soft,rsize=1048576,wsize=1048576,noatime,lookupcache=positive 0 2
Thereâs no related configuration changes in gitlab.rb, because the storage is all mounted to the default location.