GitLab CI/CD: How to use `git archive` for a project's job?

GitLab CI/CD
1

How can I use git archive ... as part of a job’s before_script?

Here are my failed attempts so far:

$ git archive --format=tar.gz HEAD reports | (cd $MY_BUILD_PATH && tar -zxf -)
fatal: not a git repository (or any parent up to mount point /builds)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

$ git archive --format=tar.gz --remote git@gitlab.host.net/user/bi-eha.git HEAD reports | (cd $MY_BUILD_PATH && tar -zxf -)
Host key verification failed.
fatal: the remote end hung up unexpectedly

$ git archive --format=tar.gz --remote https://gitlab.host.net/user/bi-eha.git HEAD reports | (cd $MY_BUILD_PATH && tar -zxf -)
fatal: operation not supported by protocol
2

I still don’t know how to use git archive in the job, but:

jobname:
  variables:
    GIT_LFS_SKIP_SMUDGE: 1

in .gitlab-ci.yml at least prevented the LFS files from being cloned in their entirety. It helped a lot.

3

Your complete .gitlab-ci.yml would be useful.

What’s the current dir when calling ‘git’? Do a ‘pwd’.

Erik

4 
$ pwd
/builds/user/bi-eha

and with GIT_STRATEGY=none, /builds/user/bi-eha is empty.

I also looked at multiple depths of find /, df, and printenv. I could find no hint of the project directory.

I’m wondering how even the default clone/fetch works. Which user runs the git clone then? How does the cloned repo “get in” the docker instance?

5

With GIT_STRATEGY=none there won’t by any repo, and most/all git operations fail. What’s the reason for choosing that strategy?

For a repo with large files handled by git lfs we skip pulling all the lfs files:
GIT_STRATEGY: clone
GIT_LFS_SKIP_SMUDGE: 1

and instead pull the few lfs files we actually need in the various jobs:
script:
git lfs pull --include “this-large-file-I-need”

6

I only need a specific subdirectory and below to perform a specific job. git archive would seem to be the best for moving the needed files to a build directory.

As you can see from my posts here, I already know how to avoid moving LFS files.

My original question still stands. How can git archive be used for a GitLab CI job?

7

But that’s not how git archive works! It needs a local repository–which you have suppressed by using GIT_STRATEGY: none. Your second format accesses an archive in the remote repository, it doesn’t create an archive from the repository.

8

Exactly! How can git archive be used to access the “remote” repo?

Why is this so hard? If a clone can be done from a remote repo (GIT_STRATEGY: clone), why not a git archive?

9

Are you aware that you can download archives of repos from GitLab via HTTP?
https://docs.gitlab.com/ee/api/repositories.html#get-file-archive

10

… only if the repo is publicly accessible. Mine is not. It’s not clear that a specific subdirectory can be specified here, either.

11

No! Not “exactly”. You can absolutely use git archive but that command requires you to already have a local repository. You’ve told gitlab not to clone a local repo! The command you’ve shown for a remote repo doesn’t do want you’re asking, regardless of whether you’re using gitlab

12

Well, then you will be amazed by this publicly accessible GitLab repo.

It’s a hack that has a throw-away “Deploy Key” and an exposed private key in the .gitlab-ci.yml.

Have a look at the last CI/CD Pipeline. Clearly, git archive is working fine with a remote repo … in this insecure manner for a public repo.

So, how can git archive work securely for a private repo?

13

I won’t be amazed. git archive against a remote repo requires an archive in the repo. You could still do that. Your example is not trying to access an archive.

14

And besides which, their CI is only insecure because they published the key for you to see how it’s done. You would use it exactly the same way–and just not publish the key for anybody else to use

15

I honestly don’t know what you are addressing now. The repo and CI jobs speak for themselves. Have you looked at the contents of the repo and the log of the CI jobs? Perhaps you are answering an unasked question?

… and yet my example repo has no archive in the repo … and the CI job moves an archive to the build directory.

… I generated and installed the public/private keys here.

16

stages:

  • build

build:
tags:
- yourtag-tag
stage: build
script:
- git archive -o $CI_COMMIT_TAG.zip HEAD