How to configure GitLab CI/runner to access private repositories on gitlab.com via npm/Git?

I’ve got a package.json with some private dependencies defined like this:
"@scope/package": "git+ssh://gitlab.com/group/subgroup/project.git#1.0.0"

The before_script part in the .gitlab-ci.yml looks like this:

- mkdir -p ~/.ssh/ && touch ~/.ssh/known_hosts && chmod 644 ~/.ssh/known_hosts
- ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
- echo -e "machine gitlab.com\nlogin gitlab-ci-token\npassword ${CI_JOB_TOKEN}" > ~/.netrc
- npm install

When pushing to gitlab.com the runner quits with the following error message:

npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t ssh://git@gitlab.com/group/subgroup/project.git
npm ERR! Warning: Permanently added the ECDSA host key for IP address '172.65.251.78' to the list of known hosts.
npm ERR! Permission denied (publickey).
npm ERR! fatal: Could not read from remote repository.
npm ERR! Please make sure you have the correct access rights

Tried for hours now, sucking up all available information on the web without success.
I’ve also tried various alternative authentication methods which all lead to the same error.

I’m pushing with a user which has access to the dependent repositories, so afaik based on the information in https://docs.gitlab.com/ee/user/project/new_ci_build_permissions_model.html this should work I guess.

Do I miss something or how do I get the GitLab runner to authenticate with gitlab.com successfully?

Thanks a lot!

Hi,

I would use https instead of git+ssh for the URL, since you do not have the key pair available.

This entry on stack overflow sounds promising - https://stackoverflow.com/questions/52784182/how-to-pull-npm-dependency-from-private-gitlab-git-repo-during-gitlab-ci-build

Cheers,
Michael

Thank you for your reply.

That’s what I am actually doing but it still throws the error; my gitlab-ci.yml file is as follows:

default:
    image: node:10

stages:
    - lint

lint:
    stage: lint
    before_script:
        - mkdir -p ~/.ssh/ && touch ~/.ssh/known_hosts && chmod 644 ~/.ssh/known_hosts
        - ssh-keyscan -H gitlab.com >> ~/.ssh/known_hosts
        - echo -e "machine gitlab.com\nlogin gitlab-ci-token\npassword ${CI_JOB_TOKEN}" > ~/.netrc
        - node -v
        - npm -v
        - npm install
    script:
        - npm run lint

Still the error I get with this is:

npm ERR! /usr/bin/git ls-remote -h -t ssh://git@gitlab.com/group/subgroup/project.git
npm ERR! Warning: Permanently added the ECDSA host key for IP address '172.65.251.78' to the list of known hosts.
npm ERR! Permission denied (publickey).

Would that error go away when using https insteaf of SSH?

When using HTTPS URLs tho, I can’t run npm install locally; it errors:

npm ERR! remote: HTTP Basic: Access denied

Could it be that using HTTPS doesn’t work for me because I have 2FA enabled? Read that I need to use personal access tokens for that, but I’m not sure how to actually use the access token without modifying the repository URL in the packages.json file.

So, I finally understood all the moving parts and how to tuck them together.

What I missed were two things:

1. Authentication via .netrc is for HTTPS not for SSH
   I wasn’t aware that using the .netrc file for authentication (within the Gitlab CI runner) was only for authenticating via HTTPS - I thought it would also work with git+ssh URLs.

2. 2FA forces the use of a personal access token for HTTPS urls
   I learned that when 2FA is enabled, a personal access token needs to be used for HTTPS authentication; the username is “oauth2” and the password is the actual personal access token.

I updated the Git URLs in the package.json file to use git+https and now I can use my PAT locally to run npm install and in CI the .netrc file in combination with the CI_JOB_TOKEN is used.

Thanks for steering me in the right direction!

Hi,

great that you figured it out by yourself! Thanks for sharing with everyone too. If you happen to have a blog or dev.to account, maybe share your adventure over there as well?

Cheers,
Michael