Migrating from NFS to object and/or Gitaly storage

We maintain a self-hosted GitLab instance (Enterprise Edition Ultimate under the Education Licence) as a small cluster without HA consisting of three application servers, a database server, a redis server and a monitoring server. Currently we have in total about 30 TB of data including repositories, artifacts, containers, uploads and LFS files, all stored on NFS shares.

As announced by GitLab, upon the release of GitLab 15.0 technical and engineering support for using NFS to store Git repository data will be officially at end-of-life.

We are looking at this change with some concern, as it means not only migrating to new storage system(s) we have no or limited experience of but also rethinking our backup concept.

A few questions:

  1. Will the functionality for using NFS to store repository data be completely removed in GitLab 15.0, meaning that we will need to migrate all repositories to another storage platform before upgrading? Or will it still work, but be out of support?

  2. Will it be possible to continue storing other types of data (artifacts, containers, uploads, LFS files) on NFS shares? The Using NFS with GitLab page recommends object storage, but for us the NFS solution is the most cost-effective as we need it for other systems as well and are satisfied with its performance.

  3. The recommended solution to store repositories is Gitaly storage or Gitaly Cluster. Do I understand correctly that they can be used only to store the repositories itself and we need another solution for the other types of data?

  4. How would you suggest backing up large amounts of repositories? Our current backup and restore concept relies on filesystem level snapshots on the NFS shares combined with database dumps taken at the same time than the snapshots.
    The Gitaly page mentions that Gitaly Cluster does not support snapshot backups and recommends using the official backup and restore rake tasks. That is at least currently not really a usable solution for us, as running the task even with skipping all other categories than repositories would take several days. Probably it would be faster on Gitaly Cluster than on the current NFS setup, but most likely still take a very long time.
    If we stopped the whole GitLab service (e.g. before a upgrade), could a filesystem level snapshot combined with database dumps from Gitaly and GitLab itself be used as a backup?

If you can answer at least some of these questions, it would be most appreciated.

4 Likes