No pipeline triggerred on commit since moving Gitlab to another server

Hello!

I just moved our Gitlab instance to a new server, pretty seamless migration, it’s a docker-compose set up, I just have to copy /srv/gitlab onto the new server and it worked! Or almost. When I push a commit, it doesn’t create a pipeline anymore (it used to), even if the .gitlab-ci.yml is valid. And because there are no errors anywhere, I don’t even know how/where to troubleshoot. Any clue ?

Thanks a million :slight_smile:

Edit: I found this in application.log, but I don’t know if it’s related, for it doesn’t match with the push time:

2020-10-13T14:17:18.009Z: {:message=>"Pipeline authorized", :project_id=>73, :user_id=>1}
2020-10-13T14:18:55.676Z: Cannot obtain an exclusive lease for Namespace::AggregationSchedule. There must be another instance already in execution.
2020-10-13T14:35:32.972Z: {:message=>"Pipeline authorized", :project_id=>73, :user_id=>1}
2020-10-13T14:35:33.169Z: Cannot obtain an exclusive lease for Ci::PipelineProcessing::AtomicProcessingService. There must be another instance already in execution.
2020-10-13T14:37:53.492Z: {:message=>"Pipeline authorized", :project_id=>73, :user_id=>1}
2020-10-13T14:37:53.716Z: Cannot obtain an exclusive lease for Ci::PipelineProcessing::AtomicProcessingService. There must be another instance already in execution.
2020-10-13T14:46:19.955Z: {:message=>"Pipeline authorized", :project_id=>73, :user_id=>1}

It looks like updating to 13.4.3 fixed the issue.

1 Like

Update: It actually partly fixed our issue: now, pipelines are triggered on pushes on develop, but not after a MR on master?

Are there settings for this? Where do I even start looking?

I’m very confused by this issue :confused:

I’m not sure if you’ve gotten your issue sorted out yet, but I ran into the same

Cannot obtain an exclusive lease for Ci::PipelineProcessing::AtomicProcessingService

message. It turned out that my turning on an unreachable K8S cluster instance-wide was breaking all pipeline runs.

Since you mentioned a server migration and a having a breakage on master but not a development branch, is it possible that you’ve hit a similar issue?

FWIW, I found the issue in exceptions_json.log - some of which looked like:

"exception.class":"Errno::ECONNREFUSED","exception.message":"Failed to open TCP connection to <IP redacted>:6443 (Connection refused - connect(2) for \"<IP redacted>\" port 6443)"

Hope that’s helpful!