I migrated my GitLab CE v12.2.4 instance from the Docker-based Omnibus Installation to the Helm Chart one. After the migration, I can’t upload any LFS objects, while downloading (cloning a GitLab repository containing LFS files) still works.
The flow is always the same: The POST /user/repo.git/info/lfs/objects/batch request succeeds, but the actual upload, PUT /user/repo.git/gitlab-lfs/objects/oid, fails with either EOF or the error An existing connection was forcibly closed by the remote host.
The application.log only lists the POST requests, and does not mention the PUT ones at all, so I don’t see any error message there:
Started POST "/myuser/lfstest.git/info/lfs/objects/batch" for 87.65.43.21 at 2019-09-10 10:56:48 +0000
Processing by Projects::LfsApiController#batch as JSON
Parameters: {"operation"=>"upload", "objects"=>[{"oid"=>"ed4afed087ffa29a371a12cfa8bb00e96f9f585cc0c7a5dbe35be453c95e1463", "size"=>774858}], "ref"=>{"name"=>"refs/heads/master"}, "namespace_id"=>"myuser", "project_id"=>"lfstest.git", "lfs_api"=>{"operation"=>"upload", "objects"=>[{"oid"=>"ed4afed087ffa29a371a12cfa8bb00e96f9f585cc0c7a5dbe35be453c95e1463", "size"=>774858}], "ref"=>{"name"=>"refs/heads/master"}}}
Completed 200 OK in 87ms (Views: 0.6ms | ActiveRecord: 51.0ms)
Started PUT "/myuser/lfstest.git/gitlab-lfs/objects/ed4afed087ffa29a371a12cfa8bb00e96f9f585cc0c7a5dbe35be453c95e1463/774858/authorize" for 87.65.43.21 at 2019-09-10 10:56:48 +0000
Processing by Projects::LfsStorageController#upload_authorize as HTML
Parameters: {"namespace_id"=>"myuser", "project_id"=>"lfstest.git", "oid"=>"ed4afed087ffa29a371a12cfa8bb00e96f9f585cc0c7a5dbe35be453c95e1463", "size"=>"774858"}
Completed 200 OK in 66ms (Views: 0.5ms | ActiveRecord: 36.4ms)
# the next retry of the client starts here
Started POST "/myuser/lfstest.git/info/lfs/objects/batch" for 87.65.43.21 at 2019-09-10 10:56:49 +0000
I first thought that my Ingress Controller could be the problem, but even if the git / lfs client directly talked to the Unicorn Pod on the Workhorse port, the error stayed the same.
I’ve uploaded the verbose output of git push to https://pastebin.com/3baMGwAa.
Do you have any idea what the problem might be?