Src refspec master does not match any

#1

Dear GitLab community,

out of sudden my deployment jobs are failing with message “src refspec master does not match any”.
I did some research and I have absolutely no idea why.
However, when I compare current deployment job log with previous one I see this difference:

Current (Not working):
Running on runner-72989761-project-4417089-concurrent-0 via runner-72989761-srm-1552747221-b30389b8…
Initialized empty Git repository in /builds/xxx
Fetching changes…
Created fresh repository.
From https://gitlab.com/xxx

  • [new branch] master -> origin/master
    Checking out 3e494a44 as master…

Previous:
Running on runner-ed2dce3a-project-4417089-concurrent-0 via runner-ed2dce3a-srm-1533052716-d10a913c…
Cloning repository…
Cloning into '/builds/xxx
Checking out 14e42d69 as master…

Why it is suddenly initializing an empty git repository instead of just cloning?
Any idea will be appreciated.

Thank you.

1 Like
Gitlab Runner does not work anymore
#2

I have the same issue, it’s just happened currently when was deploying React to Heroku

#3

I have noticed that deployments are not actually failing out of sudden but it happens after Gitlab Runner was updated to ver. 11.9+

For some reason the latest version of runner is always initializing an empty git repository instead of just cloning.

Any idea how to solve this?

1 Like
#4

Hi, we are having the exact same problem as described by werther99, which started a few days ago. We’d much appreciate any help in resolving this issue.

1 Like
#5

Hi, I have had the same problem over the past week.

I have been using ssh / git deployments with gitlab runners for the past couple of years without any issues.

Basically a push is being made to a bare git folder on my servers. Once the push has been made to the server, it triggers a hook to deploy to a public repository on the server.

Exactly the same errors as everyone else.

1 Like
#6

Hello! Anyone who is having this problem would you mind if you post a bit more detail on the environment and the job you are running?

More specifically these parts:

  1. GitLab version
  2. Job specification from .gitlab-ci.yml
  3. Is git strategy defined?
  4. What is the executor you are using?
  5. What is the runner configuration?

We changed how we clone the project to support a new feature in the long run details can be found in #3983 and the MR in !1203

1 Like
#7

Giving more contexts on the https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1203 change.

The new behavior on the fetching refs instead of cloning repo happens under the following condition,

  • GitLab Runner 11.9+
  • GitLab CE/EE (Rails) 11.9+

Basically, GitLab CE/EE sends refspecs parameter to GitLab Runner https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/presenters/ci/build_runner_presenter.rb#L34, this parameter is to used in GitLab Runners for fetching branch/tag refs from remote repository. This change was introduced because we wanted GitLab Rails side to leverage respecs in order for https://gitlab.com/gitlab-org/gitlab-ee/issues/7380 though, there should not be a big difference between git clone $URL or mkdir $REPO_DIR && git remote add origin $URL && git fetch +refs/heads/branch_name:refs/remotes/origin/branch_name. In fact, the new behavior has already run on our development project https://gitlab.com/gitlab-org/gitlab-ce/pipelines and has no issues so far.

Here is an example job log, when GitLab-Runner initializes a repository directory and fetches refs instead of git clone commands.

Running with gitlab-runner 11.10.0~beta.1249.ga9b92ea5 (a9b92ea5)
  on shinya-MS-7A34 LXxsZ3hC
Using Shell executor...
Running on shinya-MS-7A34...
Initialized empty Git repository in /home/shinya/workspace/thin-gdk/builds/LXxsZ3hC/0/root/job-run-test/.git/
Fetching changes...
Created fresh repository.
From http://localhost:8181/root/job-run-test
 * [new branch]      master     -> origin/master
Checking out 3a894e28 as master...

Skipping Git submodules setup
$ echo 'a'
a
Job succeeded

One thing that I’m seeing a similar situation in those reports is that everyone has encountered issues during deployments. So it might happen under a complicated situation. I’d appreciate if you’d help us to investigate this problem. Thanks.

1 Like
#8

Hi steveazz,

before I give my answers, I have to tell you that I am a basic Gitlab user. All my settings are default except setting 1 variable (SSH_PRIVATE_KEY) and writing my own .gitlab-ci.yml file.

  1. GitLab version
  • 11.9.0-ee
  1. Job specification from .gitlab-ci.yml

deployment:
stage: deploy

environment:

    name: production

    url: http://XXX

only:

    - master
script:
    - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'

    - eval $(ssh-agent -s)

    - ssh-add <(echo "$SSH_PRIVATE_KEY")

    - mkdir -p ~/.ssh

    - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'

    - git remote add production ssh://XXX.git
    
    - git push production master
  1. Is git strategy defined?
  • No
  1. What is the executor you are using?
  • Default one: Using Docker executor with image ruby:2.5
  1. What is the runner configuration?
  • Shared runner: gitlab-runner 11.9.0-rc2

Thank you for your help.

#9

This isn’t overly different from werther99, but if it helps, here is mine. Like the others, this was working the last time I tried a few weeks ago. Let me know if you need any other information. Thanks!

  1. GitLab version
  • 11.9.1-ee
  1. Job specification from .gitlab-ci.yml
    deploy to production:
      stage: deploy
      script:
        git remote add heroku https://heroku:$HEROKU_API_KEY@git.heroku.com/<project>.git && git push -f heroku master
      environment:
        name: production
        url: https://<project>.herokuapp.com
      only:
      - master
  1. Is git strategy defined?
  • No
  1. What is the executor you are using?
  • (output below)
  1. What is the runner configuration?
  • (output below)
Running with gitlab-runner 11.9.0-rc2 (227934c0)
  on docker-auto-scale ed2dce3a
Using Docker executor with image ruby:2.5 ...
Pulling docker image ruby:2.5 ...
Using docker image sha256:97f68e8fccc290fadf7634c95797892c54bdfd7bd0a0c2e8e4ea876b75802914 for ruby:2.5 ...
Running on runner-ed2dce3a-project-9649949-concurrent-0 via runner-ed2dce3a-srm-1553654931-389f31f2...
Initialized empty Git repository in /builds/<org-name>/<project>/.git/
Fetching changes...
Created fresh repository.
From https://gitlab.com/<org-name>/<project>
 * [new branch]      development -> origin/development
 * [new branch]      master      -> origin/master
Checking out 7d7831ab as master...

Skipping Git submodules setup
$ git remote add heroku https://heroku:$HEROKU_API_KEY@git.heroku.com/<project>.git && git push -f heroku master
error: src refspec master does not match any.
error: failed to push some refs to 'https://heroku:<scrubbed-key>@git.heroku.com/<project>.git'
ERROR: Job failed: exit code 1
#10

Our problem is LFS-related; might belong in another thread…

The new way of fetching/cloning introduced in runner 11.9 causes problem for a repo using LFS. We have specified:
variables:
GIT_STRATEGY: clone
GIT_LFS_SKIP_SMUDGE: 1
in .gitlab-ci.yml to prevent pulling LFS files. After upgrading to runner 11.9 (and gitlab 11.9.1 shortly after) our CI jobs starts cloning/fetching/pulling:
DEPRECATION: this GitLab server doesn’t support refspecs, gitlab-runner 12.0 will no longer work with this version of GitLab
Cloning repository…
Cloning into ‘/builder/b6861495/0/apps/vcpipe-bundle’…
Checking out 25b52be9 as 1815-subpanels…
Downloading LFS objects: 0% (0/1), 0 B | 0 B/s

This takes several hours (+200GB of LFS files) and eventually stopped with a no-space-left error.

Is there another mechanism to avoid pulling LFS files?

#11

@monoburn we have also done some LFS work but do you mind if you open a separate thread? You can always open an issue over at https://gitlab.com/gitlab-org/gitlab-runner/issues/new?issuable_template=Bug if you prefer :bowing_man:

The error message you see below:

DEPRECATION: this GitLab server doesn’t support refspecs, gitlab-runner 12.0 will no longer work with this version of GitLab

This was before you upgraded to GitLab 11.9.0/1 or after as well?

#12

My comment on LFS issue has been made into issue https://gitlab.com/gitlab-org/gitlab-runner/issues/4095.

#13

Ok, I see that is happening here, since we are using refspec to clone/fetch the repository we checkout a specific commit and not checking out a specific branch, and when the script does git push master the branch is nowhere to be found so git doesn’t know what to push.

This can clearly seen between the two jobs below:

I have created an issue https://gitlab.com/gitlab-org/gitlab-runner/issues/4097


Current workaround would be to do a git checkout master as part of the script

2 Likes
#14

That worked out for me!!!
@steveazz you are a hero of the day :grinning:

Thank you very much for your help!

#15

Me, too! Thank you, @steveazz.

#16

Hello folks! We have a better workaround which I would like anyone who has time to test out.

deploy:
  stage: deploy
  script:
  - git remote add heroku https://heroku:$HEROKU_API_KEY@git.heroku.com/<project>.git
  - git push -f heroku HEAD:master

This is a bit more efficient at this point.

1 Like
#17

Yep, I can confirm adding HEAD: before master seems to fix the issue.

1 Like