Pull based Mirroring from GitHub for CI is no longer mirrored

One more thing to point… Accessing Setting/Repository of the mirror https://gitlab.com/s2technologies/testspace.test.ci GitLab repo results in 500 error page!

I guess, switching from private to public must have left the GitLab mirror repo in a bad state which results 500 errors

Hi,

ok then you probably have hit a bug with a strange repository state which affects your repository on GitLab.com.

Can you please open a bug report at https://gitlab.com/gitlab-com/support-forum/-/issues including this specific finding and all the logs and timestamps to reproduce/analyse it ?

Cheers,
Michael

Sure, here it the issue - https://gitlab.com/gitlab-com/support-forum/-/issues/5339

1 Like

OK so we are using bitbucket for our repo (because that is how it was when I got here so it is not an option to host our repo on gitlab) but we use gitlab for our CI for the iOS app portion of that repo only. We are using this method https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/bitbucket_integration.html

What plan do we need now to get this back?

We are in the same boat. So what plan did you end up needing? We host our repo on bit bucket and use this approach. https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/bitbucket_integration.html I understand that they posted about this some time ago (somewhere on the internet), but I never received any information about it and I don’t go checking gitlab’s page every day. So it was a bit of a surprise when it just stopped working. Sigh.

1 Like

Wow! I break my production because this sudden change. I didn’t get notice about it.

Hey everyone, I’m Parker Ennis, product marketing manager for GitLab CI/CD. I appreciate the discussion here. I wanted to chime in and let you know that we hear you loud and clear and completely understand your frustrations. We apologize that the notifications didn’t get to you with more lead time. Our intent was certainly not to make your life more difficult right now.

We’ll do our best to help and we’re taking your feedback to heart to make the process smoother next time a feature change gets announced. Historically, we haven’t sent any marketing emails to GitLab.com accounts as a way to notify users of changes. With the feedback here, I think we need to reevaluate our approach on whether to send these types of emails to GitLab.com accounts for important updates in the future.

I can also confirm that you should be able to mirror public GitHub projects to public GitLab projects without a paid subscription. However, mirroring a private project on GitHub to a public/private project on GitLab does not work without a subscription.

If you have additional concerns/questions that aren’t covered in the blog or issues that aren’t able to be resolved with the help of team members like @dnsmichi here (thanks for jumping into this thread and assisting!), please open a support issue/bug report via https://gitlab.com/gitlab-com/support-forum/-/issues so that we can further assist.

Thanks again for bearing with us and we’ll learn from this for next time.

3 Likes

Hi which tier do I need? I just need to give my boss a $ figure.

From the original announcement, you’ve got several options depending on your setup:

If you wish to continue using CI/CD for private external repositories past March 22, 2020, you will need to upgrade your plan to at least a Silver plan.

Note: If you are only using repository mirroring without CI/CD then you only need a Bronze plan to continue using this functionality.

If it’s an alternative, you can make your Bitbucket repo public, without the need to subscribe to a paid plan:

If your repo on GitHub.com is public, then it gets mirrored to GitLab.com as a public repo and you have access to CI/CD capabilities.

Let us know if that answers your question.

3 Likes

Hey Parker. Thanks for chiming in on this! I was wondering if we would get any response from the GitLab Staff. The issue to me was not knowing about it ahead and time and having no warning when it shut down.

We setup our CI repository in the last few weeks. Spent a good amount of time getting things running well, and then silently it disappeared one day. Even-though the changeover seemed well documented and clear, I didn’t know about it until I directly researched it, at that point it was too late.

I don’t know if we’re going to stick with the CI. When things were simply “not working” with no explanation we had to find other options.

Having said that, as a GitLab.com user, I would have appreciated an email heads up sent out to a targeted list of users that would be directly affected. In addition, since we’re looking at it every day, I think a banner on top of the pipelines view would have also given us some sort of warning.

2 Likes

Hi there,

So I’m a little confused. I have a number of private repositories on Gitlab that were set up with pull mirroring from another private repository of mine also on GitLab (i.e. not an external repository).

These are all no longer working. From the original announcement (which I never received any notification about which isn’t really good enough for something of this magnitude IMO) I don’t see why this should affect these mirrors. Everything I’m reading is talking about mirroring external repositories, but mine aren’t external and yet the pull mirrors are gone. This is the only thing I was using GitLab for.

Please help me understand what’s going on.

That’s going on - CI/CD and Mirror aren’t available for privat projects on free plan. Now you have 2 options:

  1. You make your projects Public and keep using Mirrors.
  2. Buy Bronze plan for $4 per mo or higher.

But the respository being mirrored isn’t an external repository - everything I’m reading is saying this applies to external repositories. I’m just hoping there’s been some kind of mistake…

I’ve created new privat repository on gitlab (for test perposes) and the Mirror direction option is disabled there. :slightly_frowning_face: :man_shrugging:

Hi Nathan,

What did you manage to find in terms of other options? I was really only using GitLab to Pull Mirror one private GitLab project a number of other private GitLab projects (all under the same account) so I can’t really justify upgrading. Interested to know what else you found…

Hi,

Can you please share some more insights into your use case here? I don’t understand yet why this would be needed, with having the possibility to use Git submodules for example.

Also since you mention this

a picture of the repo relationships and dependencies would help clear things up.

Cheers,
Michael

Hi Michael,

Thanks so much for getting back to me. My use case is this:

  • I have a master template project on which a number of web app projects are based
  • All of the web app projects pull from the master template project

I realise the above could be achieved with standard branching but my understanding is that this would place all my projects into a single project which isn’t ideal.

I’m a bit of a git noob - perhaps there is some other workflow I’m unaware of that would achieve what I’m doing with pull mirroring? You mention git submodules - I’ll go do some reading on that…

Hi,

thanks for sharing. To provide further tips, it would be helpful to see the file structure and how these pulls are then performed. I could imagine that you got a merge commit on each pull and your history is barely readable by now.

A basic tree command on all involved projects should be just fine:

tree -L 2 -d

Git submodules likely provide the directory pulling you are doing now already.

Besides, you can also build the mirroring with CI/CD job schedules, but that’s also a little more life the knife in the back if it can be solved with git functionality.

Cheers,
Michael

Hi Michael,

Ok yeah you’ve lost me a little now. Like I said I’m a bit of a noob. :joy:

I did go have a read up on submodules and remembered I had looked into them when I was searching for a solution to my use case. They sound like something I might (maybe) be able use if I was to build specific features that I want included in some projects but not others. The reason I say “maybe” is because my understanding is that (correct me if I’m wrong) this method relies on the submodule existing in a separate set of directories to the main project.

More info on my use case:

My pull mirror of each project points at the template project and updates the master branch only. When I pull the master branch of my downstream project to my local repository, it is merged with my project branch. The project branch never gets merged back into the master or pushed back up to the remote - the master branch of my downstream projects remain a mirror of the template.

My project branch then effectively becomes the “master” branch of my downstream project and I branch and merge from there. I don’t know if that makes sense and/or whether I’m using Git as I’m supposed to but it works.

Due to the directory structure of my projects there is no way that I can have the template directories and files separate from the customizations made downstream. The directories and files of the downstream project are included in amongst the template directories and files and in some cases are modifications of the template files themselves. This is by necessity, not by my design and I have very limited control over this.

I went through a bit of a process to find a suitable workflow and pull mirroring was the only thing that I found that fit. It has been working really well for me for a year or so now.

I don’t really understand CI/CD stuff so not sure about that.

Additional info: I don’t personally mind paying $4/month to retain the pull-mirror feature, however I see that one must pay per user per group. The issue here is that my workflow involves users at times being added to an individual project for a matter of days after which they are removed. My understanding is that I would have to pay annually for enough seats to allow this to happen. I’m a one man band so it starts to sting a little.

Was the shift to a paid feature intended to affect users pull-mirroring internal GitLab repositories or only external ones? I’m starting to get the picture that my use of the feature maybe somewhat unusual…

Thanks so much for your help on this Michael - I really appreciate it.

Luke

I now have 3 public repos on Gitlab.com mirrored from their also-public equivalents on Github. The first two are mirrored OK (pull mirroring with webhooks that trigger upon any commit on any branch). The most-recently configured one is NOT working and there appears to be NO way to get it to work without filing a ticket.

The behavior is such that I have to do an “add origin” and push to BOTH repos directly from my Macbook (and from any collaborators); a recipe for future disaster. (Manually-triggered pull-mirroring from Gitlab UI doesn’t seem to pick up new branches, and it breaks on any force-push.)

To fix this, I would need to find the webhook URL and password on the Settings / Mirror page of Gitlab.com. The webhook’s password is not available there so I cannot configure it on the Github side.