I migrated my GitLab CE v12.2.4 instance from the Docker-based Omnibus Installation to the Helm Chart one. After the migration, I can’t upload any LFS objects, while downloading (cloning a GitLab repository containing LFS files) still works.
The flow is always the same: The POST /user/repo.git/info/lfs/objects/batch
request succeeds, but the actual upload, PUT /user/repo.git/gitlab-lfs/objects/oid
, fails with either EOF or the error An existing connection was forcibly closed by the remote host
.
The application.log
only lists the POST
requests, and does not mention the PUT
ones at all, so I don’t see any error message there:
Started POST "/myuser/lfstest.git/info/lfs/objects/batch" for 87.65.43.21 at 2019-09-10 10:56:48 +0000
Processing by Projects::LfsApiController#batch as JSON
Parameters: {"operation"=>"upload", "objects"=>[{"oid"=>"ed4afed087ffa29a371a12cfa8bb00e96f9f585cc0c7a5dbe35be453c95e1463", "size"=>774858}], "ref"=>{"name"=>"refs/heads/master"}, "namespace_id"=>"myuser", "project_id"=>"lfstest.git", "lfs_api"=>{"operation"=>"upload", "objects"=>[{"oid"=>"ed4afed087ffa29a371a12cfa8bb00e96f9f585cc0c7a5dbe35be453c95e1463", "size"=>774858}], "ref"=>{"name"=>"refs/heads/master"}}}
Completed 200 OK in 87ms (Views: 0.6ms | ActiveRecord: 51.0ms)
Started PUT "/myuser/lfstest.git/gitlab-lfs/objects/ed4afed087ffa29a371a12cfa8bb00e96f9f585cc0c7a5dbe35be453c95e1463/774858/authorize" for 87.65.43.21 at 2019-09-10 10:56:48 +0000
Processing by Projects::LfsStorageController#upload_authorize as HTML
Parameters: {"namespace_id"=>"myuser", "project_id"=>"lfstest.git", "oid"=>"ed4afed087ffa29a371a12cfa8bb00e96f9f585cc0c7a5dbe35be453c95e1463", "size"=>"774858"}
Completed 200 OK in 66ms (Views: 0.5ms | ActiveRecord: 36.4ms)
# the next retry of the client starts here
Started POST "/myuser/lfstest.git/info/lfs/objects/batch" for 87.65.43.21 at 2019-09-10 10:56:49 +0000
I first thought that my Ingress Controller could be the problem, but even if the git / lfs client directly talked to the Unicorn Pod on the Workhorse port, the error stayed the same.
I’ve uploaded the verbose output of git push
to https://pastebin.com/3baMGwAa.
Do you have any idea what the problem might be?