Git-data has most recent repositories, but they're not appearing

How to Use GitLab
1

Problem to solve

I inherited a fairly old gitlab-ce (11.11.8) being run in a docker container. It unexpectedly stopped, and on restart the data that came up was a month old (projects are sorted by latest update). I’ve checked /var/opt/gitlab/git-data/repositories and luckily the new stuff was there, but I can’t seem to get them to appear.

I tried doing a test push into a repo I made before the instance went down, and got
“project not found”.

The most recent backup was two-ish weeks ago, so rolling back isn’t the best option

Configuration

gitlab.rb is basically template with everything commented out, besides the below:

gitlab_rails[‘env’] = {“http_proxy” => “webproxy.mmih.biz”, “https_proxy” => “webproxy.mmih.biz”}

System information

System:
Current User: git
Using RVM: no
Ruby Version: 2.5.3p105
Gem Version: 2.7.9
Bundler Version:1.17.3
Rake Version: 12.3.2
Redis Version: 3.2.12
Git Version: 2.21.0
Sidekiq Version:5.2.7
Go Version: unknown

GitLab information
Version: 11.11.8
Revision: 1d18d065069
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: PostgreSQL
DB Version: 10.7
Using LDAP: no
Using Omniauth: yes
Omniauth Providers:

GitLab Shell
Version: 9.1.0
Repository storage paths:

  • default: /var/opt/gitlab/git-data/repositories
    GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
    Git: /opt/gitlab/embedded/bin/git

Troubleshooting

I’ve tried the following commands in the container with no grace

  • gitlab-rake db:migrate:status
  • gitlab-rake cache:clear + gitlab-rake db:seed_fu
  • gitlab-rake gitlab:git:fsck
  • gitlab-rake gitlab:assets:clean gitlab:assets:compile (Received Error: Yarn executable was not detected in the system.)

I’ve tried creating another container with docker-compose, and rerunning the container with docker run commands from scratch. Same issue where if I check the container, the git-data shows the new stuff but they don’t appear.

I’ve also tried creating a new container with different versions (12.0.12 which completely failed to start gitlab and 11.10.8 had the same issue as our current gitlab).

I’ve asked a senior person to assist, but all out of ideas there

EDIT: new users are also effected, if they were added in the last month, they can no longer be found

2

Granted I do NOT run gitlab in docker, but an internet search got me to: git - Recovering GitLab repos from the server-side directories - Stack Overflow

Which is old, but round about way to recover data.

As for the users, I believe that those are stored in the DB and its likely lost because of the docker restart.

Docker is nice, but unless you plan out where your data is going, its easy to loose data with the default setup.