How can you efficiently instantiate a repository with a subrepository when using the CI Docker runner?

GitLab CI/CD
, ,
1

I am working on the software repository DAMASK, where all the test are located in a private subdirectory called PRIVATE. We have a gitlab-ci script, which we are currently slowly modifying to use the docker instead of the shell runner. However, since we have a subrepository where the tests that the pipeline needs to run are located, we are forced to add

  before_script:
    - git config --global --add safe.directory /builds/damask/DAMASK
    - eval $(ssh-agent -s)
    - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add -
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config
    - git submodule sync
    - git config --global --add safe.directory /builds/damask/DAMASK/PRIVATE
    - git submodule update --init

Before every step that uses the docker runner and the PRIVATE tests. It would be much more efficient if there was a way to instantiate the repository with the subrepository already initiated and synced, so that each docker container starts with a complete code and doesn’t require the subrepository to be cloned each time.

Is there something we are missing, or a more effective way to accomplish this?

2

Is the submodule repository located on the same GitLab instance?

3

yes, it’s a separate repository on the same gitlab server.

4

Use relative path from your master project to your submodule project in submodule declaration

[submodule "PRIVATE"]
  path = PRIVATE
  url = ../../project.git

in CI set global variable GIT_SUBMODULE_STRATEGY: normal

PS: The link in your first post is pointing to GitHub.

5

I see, thank you for your response. Unfortunately this suggestion doesn’t work, since it’s a private repository so we need to add an ssh key first, and setting GIT_SUBMODULE_STRATEGY: normal causes the runner to try to initialize the subrepository before the before_script section. I was wondering if it would be possible to somehow perform this step before the repository is copied into the docker container, do you maybe have an idea if this is possible?