We have a new gitlab installation of 13.5.3 on Ubuntu 20.04 on a DigitalOcean droplet.
When we try to upgrade to 13.5.4, the upgrade fails because the upload of the backup to DigitalOcean spaces fails with a 404 error.
We have configured the backups on DigitalOcean spaces. The relevant portion of our gitlab.rb config looks like this:
gitlab_rails["manage_backup_path"] = true
gitlab_rails["backup_path"] = "/mnt/gitlab/backups"
gitlab_rails["backup_upload_connection"] = {
"aws_access_key_id" => "XXX",
"aws_secret_access_key" => "XXX",
"endpoint" => "https://sgp1.digitaloceanspaces.com",
"provider" => "AWS",
"region" => "sgp1"
}
gitlab_rails["backup_upload_remote_directory"] = 'backups'
gitlab_rails["backup_multipart_chunk_size"] = 104857600
Console output:
Creating backup archive: 1605532530_2020_11_16_13.5.3_gitlab_backup.tar ... done
Uploading backup archive to remote storage backups ... rake aborted!
Excon::Error::NotFound: Expected(200) <=> Actual(404 Not Found)
excon.error.response
:body => "<?xml version=\"1.0\" encoding=\"UTF-8\"?><Error><Code>NoSuchBucket</Code><BucketName>backups</BucketName><RequestId>tx00000000000005a685db5-005fb27b86-12a4463-sgp1a</RequestId><HostId>12a4463-sgp1a-sgp</HostId></Error>"
:cookies => [
]
:headers => {
"Accept-Ranges" => "bytes"
"Content-Length" => "215"
"Content-Type" => "application/xml"
"Date" => "Mon, 16 Nov 2020 13:15:50 GMT"
"Strict-Transport-Security" => "max-age=15552000; includeSubDomains; preload"
"x-amz-request-id" => "tx00000000000005a685db5-005fb27b86-12a4463-sgp1a"
}
:host => "backups.sgp1.digitaloceanspaces.com"
:local_address => "157.230.36.54"
:local_port => 54420
:path => "/1605532530_2020_11_16_13.5.3_gitlab_backup.tar"
:port => 443
:reason_phrase => "Not Found"
:remote_ip => "103.253.144.208"
:status => 404
:status_line => "HTTP/1.1 404 Not Found\r\n"
/opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:52:in `upload'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:26:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:backup:create
The bucket exists, and the credentials are valid.
The backup was working fine previously.