Gitlab-ctl reconfigure error due to cannot load such file

How to Use GitLab Self-managed
1

Hi there ! I need your help on this issue encountered after VM not enough space issue:

System Info:

chef_version=17.10.0
platform=ubuntu
platform_version=22.04
ruby=ruby 3.0.6p216 (2023-03-30 revision 23a532679b) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/cinc-client
executable=/opt/gitlab/embedded/bin/cinc-client

Running handlers:
[2024-01-11T10:07:59+00:00] ERROR: Running exception handlers
There was an error running gitlab-ctl reconfigure:

cannot load such file – tomlib

Running handlers complete
[2024-01-11T10:07:59+00:00] ERROR: Exception handlers complete
Infra Phase failed. 0 resources updated in 01 seconds
[2024-01-11T10:08:00+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2024-01-11T10:08:00+00:00] FATAL: ---------------------------------------------------------------------------------------
[2024-01-11T10:08:00+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2024-01-11T10:08:00+00:00] FATAL: ---------------------------------------------------------------------------------------
[2024-01-11T10:08:00+00:00] FATAL: LoadError: cannot load such file – tomlib

Any idea ?

2

How was this instance installed?

Was any gitlab files deleted when fixing the space issue?

Was there a change in the file system or mounts when fixing the space issue?

Does the account running gitlab have access to areas of the file system?

1 Like
3

This instance was installed using docker hub.
Maybe some gitlab files were deleted when fixing the space issue, but I don’t know how to know it/check it.
The filesystem is the same.
Yes, the account running gitlab has access to the file system.

4

If this is running in a docker instance then you should look at if there are any deleted docker files. Were you using a bind mount or a volume in docker to create the gitlab container?

1 Like
5

Docker runs well, I can create a new additionnal gitlab instance from scratch.
But I want to recover the old gitlab instance that contains the repositories.

And sorry, how could I know if I used a bind mount or volume in docker ?

6

Can you do a docker container ls -a and make sure your old Gitlab container is still around. Then do a docker container inspect <container_name> That should tell you where on your host file system the docker volumes can be located.

When starting the docker container you would have used a --volume flag or something like it. What did you specify with that flag? Something like:

--volume /path/to/files:/etc/gitlab

If that /path/to/files is one that you deleted files from then you might have to do some work to rebuild your Gitlab instance using files from a new copy.

1 Like
7

Yes ok, thank you. I got now this path/to/files.
Can you show me how to rebuild the Gitlab instance from a new copy ?

Are the used git repos saved in this /etc/gitlab ? If so, I am interested in backuping them if the rebuild action affects them.

8

Fair warning this is a bit of a Frankenstein’s monster and I was assuming that you can’t backup your instance. You should run the following to try and create a gitlab backup of your instance.
docker exec -t <container name> gitlab-backup create.

If the backup is created successfully then follow Gitlab’s recovery documentation using the backup.

If it does not work, please let me know and I can give you a plan to attempt recovery.

1 Like
9

Once container is launched, it fails and exits.
So the command docker exec -t <container name> gitlab-backup create outputs : Error response from daemon: Container [xxx] is not running