Error when backup Gitlab self host

Hi all,

I have problem when backup my gitlab server, the error is:

2021-05-04 01:00:19 +0700 – Dumping database …
Dumping PostgreSQL database gitlabhq_production … pg_dump: error: Dumping the contents of table “raw_usage_data” failed: PQgetResult() failed.
pg_dump: error: Error message from server: ERROR: missing chunk number 0 for toast value 1424382 in pg_toast_958562
pg_dump: error: The command was: COPY public.raw_usage_data (id, created_at, updated_at, recorded_at, sent_at, payload, version_usage_data_id_value) TO stdout;
[FAILED]
rake aborted!
Backup::Error: Backup failed
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/database.rb:62:in dump' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:134:in block (4 levels) in <top (required)>’
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:12:in block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in load’
/opt/gitlab/embedded/bin/bundle:23:in `’
Tasks: TOP => gitlab:backup:db:create
(See full trace by running task with --trace)

May be the table raw_usage_data is corrupt at row 42:

gitlabhq_production=> select count(*) from raw_usage_data;
count

48

(1 row)
gitlabhq_production=> select * from raw_usage_data order by id limit 1 offset 42;
ERROR: missing chunk number 0 for toast value 1424382 in pg_toast_958562

Please help to advice.
Thanks alot for your help,
ADT.

From a quick google, there are a few things you can try. Reindex the table, assuming raw_usage_data is the table then in postgresql:

REINDEX table pg_toast.pg_toast_958562;
REINDEX table raw_usage_data;

you can also try:

VACUUM ANALYZE raw_usage_data;

and see if that helps.

Thanks for your reply, I have googled and tried, but failed:

gitlabhq_production=> REINDEX table pg_toast.pg_toast_958562;
ERROR: permission denied for schema pg_toast

Try using:

sudo gitlab-psql

you need to run psql as a user with higher privileges that has access to all tables.

That’s it, thanks alot. The backup is ok now.
Thanks again and have a nice day.