Unable to push to repo after updating Gitlab

After updating the Gitlab from 12.2.1 to 12.4.0 I am unable to push the changes to repo. This problem exists with any branch.

This is the output I’m having when trying to push:

remote: GitLab: This action cannot be performed by internal users
To git.repo.com:repo/main-websites/repo.com.git
! [remote rejected] branch -> branch (pre-receive hook declined)
error: failed to push some refs to 'git@git.repo.com:repo/main-websites/repo.com.git'

I have tried the following:

  • Reconfiguring/restarting the Gitlab
  • Deleting my local copy and cloning it from scratch, I am able to pull the project with no problem, but not able to push.
  • I’ve tried to make changes from another machine and it’s the same story there.
  • I’ve tried to delete my SSH key from Gitlab and regenerate it, but it didn’t helped either.

Here is the only piece of log I’ve found that is related to the issue

{
	"method":"POST",
	"url":"http://127.0.0.1:8080/api/v4/internal/allowed",
	"code":"401",
	"body":"{
		"status":false,
		"message":"This action cannot be performed by internal users"
	}",
	"pid":17255,
	"level":"error",
	"msg":"Call failed",
	"time":"2019-10-23T13:55:33+00:00"
}

Will be thankful for any ideas about what might be causing it as I am completely stuck here.

Thank you!

I have noticed my previous commits are all showing as a ghost user now, but I wasn’t deleting my user or changing it in any way. Still looking for the solution…

I’ve just tested upgrade from 12.3.5 to 12.4.0 and it works fine. Maybe check if you can create a new repo in the UI and clone/push changes.

Unfortunately I can’t push to any repo from my machine. When I try to SSH to Gitlab instance it says “Welcome to GitLab, Anonymous!”, I’ve tried it from other machine and it says “Welcome to GitLab @username

have you made any changes to the config/gitlab.rb?

No, except from upgrading the GitLab I wasn’t changing nothing. I think it might be the issue with AWS configuration as the GitLab instance sits in AWS, but it works from the other machine within the same network which is really confusing.

Apparently the file with keys on GitLab instance was corrupted.

I’ve fixed it the following way:

  • I deleted the file: /var/opt/gitlab/.ssh/authorized_keys
  • Deleted all SSH keys from GitLab Web UI
  • Re-added the keys

And it works fine now…

Solution have been found here: https://stackoverflow.com/a/29765704/1544230

Are we sure this is the correct answer?

We’re seeing the exact same problem. In our case, we have a single deploy key that is marked as write-activated, used in 23 of our projects. In the authorized_keys file, I do not see the deploy keys in the authorized_keys folder. The last thing I want to do is completely remove all my SSH keys and re-add them, given how much automation we have in place.

We are seeing the same @ghost user in our commit history as well. It definitely seems like something changed with 12.4!

Edit:

I’m looking through the changelog, and I saw this note in 12.4.1:
Disallow unprivileged users from commenting on private repository commits.
Now, we’re running 12.4, but this instantly makes me wonder if my deploy key is considered an unprivileged user.

I also found a merge request that deals with deploy key write permission:

I am not sure of the answer is correct, but that’s how I solved my issue.
Also I would like to mention that SSH key is set for the user and not for project so you will not need to re-add that to all 23 projects you have.