Refreshing a Fork

j.green · December 10, 2019, 9:42pm

I have forked a repository that is constantly updated by the creator. How do I go about updating my forked copy so that I have the most up-to-date files?

dnsmichi · December 11, 2019, 10:41am

Hi,

you can do that in two ways. Both of them expect that you never work on the default branch (main/master, check your project settings) of your fork, but only create change commits in feature and fix branches.

GitLab Repository Mirror

Navigate into Settings > Repository > Mirroring repositories.

Add the upstream URL
Mirror direction: Pull
Tick Only mirror protected branches (optional)

This feature is available in the Premium tier: Pull from a remote repository | GitLab

Git CLI

Add a secondary remote called upstream.

git remote add upstream https://...

Fetch the remote and then pull its changes into your local default branch, for example main.

git checkout main
git fetch upstream
git pull upstream main

Last, push to your own remote origin to keep the forked repo in sync.

git push origin main

Based on the default main branch, you can continue with creating branches, e.g. git checkout -b feature/....

You could also write a script for that, e.g. when you’re syncing between different Git servers.

One liner script in your forked clone:

git remote add upstream https://gitlab.com/upstream_user_group/project.git || true && git fetch upstream && git checkout main && git pull upstream main && git push origin main

GitLab feature proposal: Fetch upstream history when fork is behind

Feature request in GitLab: Fetch new upstream contents when fork is behind (#330243) · Issues · GitLab.org / GitLab · GitLab will be implemented in iterations.

15.7 provides the information that a fork is behind upstream. How far is your fork behind/ahead of the upstream project (#14491) · Issues · GitLab.org / GitLab · GitLab
15.8 or later (please check the issue for updates) will allow syncing the default branch with the upstream fork. Fetch new upstream contents when fork is behind (#330243) · Issues · GitLab.org / GitLab · GitLab

Cheers,
Michael

j.green · December 11, 2019, 7:57pm

Thank you for that detailed instruction. With your first option (GitLab Repository Mirror), I only have the option for PUSH for Mirror Direction. Any idea why this might be?

dnsmichi · December 12, 2019, 2:52pm

Hi,

maybe the form needs a refresh, I’ve found it a bit wonky when adding a new item. Or the URL you’re providing above is the same as the current repository. Can you share a screenshot?

Cheers,
Michael

j.green · December 13, 2019, 1:17am

Hi - Please see the attached screenshot.

dnsmichi · December 13, 2019, 8:10am

Look valid to me. Maybe try hitting refresh once, then copy the URL again. I had some issues with the different form states until it allowed me to change the mirror direction. Though I was using gitlab.com, not sure whether this is an advanced option only applicable in EE.

Cheers,
Michael

j.green · December 16, 2019, 4:22pm

Seems the issue is that I’m on a hosted GitLab through my company. When I try this on gitlab.com, I have the Push/Pull option, but when I’m on my company’s site, I only have Pull.

dnsmichi · December 16, 2019, 4:39pm

Ah, I see. Didn’t look into the specific docs section where this feature is marked as a paid tier feature.

In that case (assuming you’re using the free tier), I’d suggest going the manual scripted way with adding that to a cronjob somewhere. Or you’ll migrate to Premium/Ultimate

Cheers,
Michael

fihlatv · December 28, 2020, 2:32pm

worked liked a charm

WhiteChairFromIkea · March 3, 2021, 7:41am

Thank you, you saved my day! :)))

dnsmichi · July 23, 2021, 10:13am

A post was split to a new topic: Problem with repository mirroring for forks

dnsmichi · July 30, 2021, 2:06pm

I’ve linked the feature request in GitLab in the tutorial above. Offer to fetch and merge new upstream contents when fork is behind (#330243) · Issues · GitLab.org / GitLab · GitLab - please upvote and/or dive into contributing it in case you want to see it happen

newbie1 · October 11, 2021, 10:09pm

To: method1 : strage it only let me push this way, the select is frozen

dnsmichi · October 12, 2021, 2:44pm

Thanks, I thought I had added the feature availability note. I have edited my response from before joining GitLab, and added that pull is available in the Premium tier, more details in Pull from a remote repository | GitLab

Cheers,
Michael

webmaster1989 · February 26, 2022, 4:11pm

I still hope there will be a better solution in 2022, I mean even GitHub allows you to easily update your branch from the upstream repo with a single click of a button ;( (no premium tier required)

dnsmichi · February 27, 2022, 7:32pm

Please share your upvote and feedback in Offer to fetch and merge new upstream contents when fork is behind (#330243) · Issues · GitLab.org / GitLab · GitLab raising awareness. Thanks

grahamperrin · July 17, 2022, 4:23pm

I have (amongst other things) /usr/ports with the configuration shown below.

Can I use git commands with this repo, which already uses git.freebsd.org, to periodically refresh my fork of the secondary mirror? https://gitlab.com/grahamperrin/freebsd-ports

% git -C /usr/ports config --local --list | sort
branch.main.merge=refs/heads/main
branch.main.remote=freebsd
core.bare=false
core.filemode=true
core.logallrefupdates=true
core.repositoryformatversion=0
remote.freebsd.fetch=+refs/heads/*:refs/remotes/freebsd/*
remote.freebsd.url=https://git.freebsd.org/ports.git
% cat /usr/ports/.git/config
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "freebsd"]
        url = https://git.freebsd.org/ports.git
        fetch = +refs/heads/*:refs/remotes/freebsd/*
[branch "main"]
        remote = freebsd
        merge = refs/heads/main
%

dnsmichi · July 18, 2022, 2:35pm

I think that could work. Before starting, I noticed the remote being freebsd, which is a great idea to avoid accidental git pull origin ... calls. I’d suggest following the naming schema with remote being something else than origin, so you always know where things are pushed/pulled from.

I’m not deeply familiar with how FreeBSD ports handle refreshing the ports tree (been years since I last touched 11.x), but I’d say it is safe to fetch and pull for testing purposes manually.

cd /usr/ports
git fetch freebsd
git checkout main
git pull freebsd main

Syncing to 2nd remote

For syncing to a secondary remote, add the following configuration via CLI (or edit the config manually, mocking what’s already there).

git remote add glmirror https://gitlab.com/grahamperrin/freebsd-ports.git
git checkout main
git pull freebsd main # optional, can also be done with FreeBSD ports tools refresh 
git push -u glmirror main

.git/config result.

[remote "glmirror"]
        url = https://gitlab.com/grahamperrin/freebsd-ports.git
        fetch = +refs/heads/*:refs/remotes/glmirror/*

Push needs an access token stored the git configuration for the system’s user credential store. git-credential-manager/credstores.md at main · GitCredentialManager/git-credential-manager · GitHub I’d suggest creating a project access token which is scoped to only this project, with read/write repository permissions. Project access tokens | GitLab

Sync with SSH instead of HTTPs

If the credentials store does not work with HTTPs, it could be easier to generate an SSH key for the FreeBSD user managing the ports tree, and add that into your account on GitLab.com, Use SSH keys to communicate with GitLab | GitLab

If you chose SSH, the remote add looks different:

git remote add glmirror git@gitlab.com:grahamperrin/freebsd-ports.git

Note that SSH needs a manual run to accept the host key verification. git fetch glmirror should be sufficient to trigger once.

Sync script and cronjob

For automated syncing, create a small script and call it in a cron job. AFAIK FreeBSD requires sh, not bash. Please test before putting in production, below script is not tested.

$ cat >/usr/local/bin/sync_ports_gitlab_mirror.sh<<EOF

#!/bin/sh

# Uncomment if the script should run as root only
#if [ $(id -u) -ne 0 ]; then
#  echo "This script must be run as root"
#   exit;
#fi

#TODO: Add logging and error handling when needed.
cd /usr/ports

git checkout main
# optional pull
#git pull freebsd main

# sync to GitLab mirror (note: If someone modified the mirror accidentally and the push fails, use `git push -f` )
git push glmirror main

EOF

$ chmod +x /usr/local/bin/sync_ports_gitlab_mirror.sh

$ crontab -e

# hourly git mirror sync
0 * * * * /usr/local/bin/sync_ports_gitlab_mirror.sh

If ports require to keep a specific branch being checked out, the script needs to take care of that, e.g. by reading the current git branch first into a variable, and later checking it out again after syncing the main branch.

If you want to sync more than the main branch, the script needs to either know a static list of branches and go over it in a loop, or you’ll work with git branch -r | grep freebsd/ to get a list of branches to sync. git push --mirror should sync all branches AFAIK.

Conclusion

Looks more complicated than it is; I tried to be as verbose as possible above. Cuts down to 1) Git push access 3) git push commands 3) sync script and cronjob.

dnsmichi · July 18, 2022, 2:37pm

soloturn · July 27, 2022, 7:10pm

i forked a repository from gitlab archlinux into archlinux gitlab. set the upstream and the clone as remote. but it would not let me refresh the clone, saying i cannot push another authors commit. painful this gitlab is

> git push st
Enumerating objects: 129, done.
Counting objects: 100% (129/129), done.
Delta compression using up to 4 threads
Compressing objects: 100% (51/51), done.
Writing objects: 100% (99/99), 14.29 KiB | 7.14 MiB/s, done.
Total 99 (delta 71), reused 70 (delta 44), pack-reused 0
remote: GitLab: Author 'chris@chrisdown.name' is not a member of team
To ssh://gitlab.archlinux.org:222/soloturn/pacman.git
 ! [remote rejected]   master -> master (pre-receive hook declined)
error: failed to push some refs to 'ssh://gitlab.archlinux.org:222/soloturn/pacman.git'