Migration of uploads to object storage broke content-type of files


We have self-hosting omnibus instance 14.6.5-ee (open source) with local storage and we plan to migrate to an S3 provider with proxy_download enabled.

After reading carefully the migration plan we decided to tests with uploads first.

Every thing looks good, our 587 uploads where copied to the object storage, unfortunately, avatar images and others are not rendered in the browser.

As an example, our logo PNG file has a content-type = application/octet-stream instead of image/png.

When I download the PNG logo file from our S3 provider locally, file says it’s data:

$ file /tmp/logo.png 
/tmp/logo.png: data
$ file --mime /tmp/logo.png 
/tmp/logo.png: application/octet-stream; charset=binary

When I open the file, there is some strange data before and after the PNG content:

$ head -n 2 /tmp/logo.png 

$ tail -n 2 /tmp/logo.png 

Does someone have any clue about what is happening?

I do not have this issue when uploading with mc the same logo.png retrieved from a backup.

So, this looks to be gitlab related :thinking:

I finally found the answer, the scaleway S3 provider does not support enable_signature_v4_streaming = true.

I tested with the following code in sudo gitlab-rails console:

s3 = Fog::Storage.new(
  provider: 'AWS',
  region: "fr-par",
  aws_access_key_id: "XXXXX",
  aws_secret_access_key: "YYYYY",
  endpoint: "https://s3.fr-par.scw.cloud",
  enable_signature_v4_streaming: false)

uploads = s3.directories.get('gitlab-mim-uploads')
uploads.files.create(key: 'logo.png', body: File.open('/tmp/logo.png'))

The solution is to set enable_signature_v4_streaming to false in the connection settings.

1 Like