Backup failure..rake aborted!

Before upgrading from 15.9.3 to higher version I am going to take backup…But its failure
commands used :
gitlab-backup create
gitlab:backup:create --trace

but the result is same.

output is

f0b4aac12e65997c418e0dd6.git",“storage_name”:“default”,“time”:“2023-06-23T10:28:34.847Z”}

{“level”:“fatal”,“msg”:“create: pipeline: 1 failures encountered:\n - @hashed/89/aa/89aa1e580023722db67646e8149eb246c748e180e34a1cf679ab0b41a416d904.git (api_management/django-backend): manager: write custom hooks: *backup.FilesystemSink write: write file "/var/opt/gitlab/backups/repositories/@hashed/89/aa/89aa1e580023722db67646e8149eb246c748e180e34a1cf679ab0b41a416d904/1687516038_2023_06_23_15.9.3/001.custom_hooks.tar": rpc error: code = Internal desc = reading custom hooks: waiting for tar command completion: exit status 2\n”,“pid”:105575,“time”:“2023-06-23T10:28:34.848Z”}

2023-06-23 10:28:34 UTC – Deleting tar staging files …

2023-06-23 10:28:34 UTC – Cleaning up /var/opt/gitlab/backups/db

2023-06-23 10:28:34 UTC – Cleaning up /var/opt/gitlab/backups/repositories

2023-06-23 10:28:35 UTC – Deleting tar staging files … done

2023-06-23 10:28:35 UTC – Deleting backups/tmp …

2023-06-23 10:28:35 UTC – Deleting backups/tmp … done

2023-06-23 15:58:35 +0530 – Deleting backup and restore lock file

rake aborted!

Backup::Error: gitaly-backup exit status 1

/opt/gitlab/embedded/service/gitlab-rails/lib/backup/gitaly_backup.rb:59:in `finish!’

/opt/gitlab/embedded/service/gitlab-rails/lib/backup/repositories.rb:28:in `dump’

/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:66:in `run_create_task’

/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:211:in `block in run_all_create_tasks’

/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:210:in `each_key’

/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:210:in `run_all_create_tasks’

/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:42:in `create’

/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:16:in `block (4 levels) in <top (required)>’

/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:203:in `lock’

/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:13:in `block (3 levels) in <top (required)>’

/opt/gitlab/embedded/bin/bundle:23:in `load’

/opt/gitlab/embedded/bin/bundle:23:in `’

Tasks: TOP => gitlab:backup:create

(See full trace by running task with --trace)

need resolution for this AS i am facing same issue

me either
have you found the solution yet ?