Gitlab runner cloning submodule from wrong URI

GitLab CI/CD
1

I run a local instance of Gitlab on my server which I am in the process of setting up to support CI for a project of mine. I’m using a custom docker image that is based on Ubuntu 18.04 LTS and is available on the server.

The project uses a submodule which I had initially added using its Github SSH URL. This worked fine until today when I realized that the git submodule update step was erroring out due to lack of permissions, as the SSH key was not present in the container. Reading many great posts here and elsewhere led me to decide to switch the submodule’s URL to HTTPS to avoid messing around with SSH keys. This is what the .gitmodules file looks like now:

[submodule "externals/translation-server"]
	path = externals/translation-server
	url = https://github.com/zotero/translation-server.git

However, whenever the gitlab runner runs, it still attempts to clone the submodule from its old SSH URL:

$ git submodule update --init --remote --merge
Cloning into '/builds/com/prj/externals/translation-server'...
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:zotero/translation-server.git' into submodule path '/builds/com/prj/externals/translation-server' failed
Failed to clone 'externals/translation-server'. Retry scheduled
Cloning into '/builds/com/prj/externals/translation-server'...
Host key verification failed.
fatal: Could not read from remote repository.

When I run the docker container and execute the steps taken by gitlab runner manually, it all works correctly:

# docker run -i -t -v`pwd`:/builds/com/prj docker-image /bin/bash
root@fcc6b0d990ad:/# cd /builds/com/prj/
root@fcc6b0d990ad:/builds/com/prj# git submodule update --init --recursive --remote --merge
Cloning into '/builds/com/prj/externals/translation-server/modules/zotero'...
Submodule path 'externals/translation-server/modules/zotero': checked out '42782f73fbddc1a44d9b655ad4c715de05b07345'

Is gitlab’s runner perhaps using some sort of caching that is overriding the module’s HTTP URL in .gitmodules? What else can I do to further investigate this issue?

I would very much appreciate some help diagnosing this.

1 Like
2

To test the theory that there’s something wrong with either Gitlab or the runner, I created a test repository, then:

  • added a git module: git submodule add https://github.com/zotero/translation-server.git
  • added the Gitlab CI yaml file unchanged from the project above
  • pushed to the server

Then I watched the job running and… it worked as expected: the module was pulled from its HTTPS URL and no errors occurred.

1 Like
3

@migdsb It’s most likely because it’s not creating a fresh repository but it’s just reusing an old repository you cloned before, and doing git fetch, so all the git config is still using ssh.

I suggest the following:

Specify git strategy once

I suggest to run a pipeline manually and in the variable section, you defined GIT_STRATEGY: clone which specified git strategy clone which will clone the project from scratch, and then use the new settings

Clear build Volumes

  • Clear all the stopped containers docker rm $(docker ps -aq)
  • Clear all the volumes inside of docker this is up to you how to set it up.
  • Re run the pipeline.