Download remote file to repository using CI/CD

GitLab CI/CD
1

Hi wonderful people,

I apologize in advance if this has been asked before. I’m new in the whole CI/CD workflow and maybe that’s why I’m missing something. As far as I searched, I didn’t find any related post and decided to drop my question here.

I’m maintaining a project on GitLab that has a kind-of external dependency. More specifically, I want to pull a remote file into the repository and was wondering if there is a way to do that using CI/CD. My intention is to pull that file, then zip the whole repo (including the pulled file) and make a release.

I appreciate anyone’s help.

2

Hi @gasfree

this depends on what do you mean by ‘release’. GitLab has it’s own idea of Releases. And in general there are plenty of different ways how to ‘release’ something.

The most simplest would be to just run a job when you create a Tag that would download the remote file, zip it all and upload somewhere.

stages:
 - zip

zip it:
  stage: zip
  image: alpine:latest
  rules:
    - if: $CI_COMMIT_TAG
  script:
    - apk update; apk add zip curl
    - curl -o remotefile https://uri_to_remote_file
    - zip release.zip remotefile someotherfile somedir
    # upload it somewhere. this example uploads it to Projects Generic package registry
    - curl --header "JOB-TOKEN: ${CI_JOB_TOKEN}" --upload-file release.zip "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/releases/1.0.0/release.zip"

Following from that you can also have an entry in your Project’s Releases page by creating a Release GitLab understands:

stages:
  - zip
  - release

zip it:
...

release:
  stage: release
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  rules:
    - if: $CI_COMMIT_TAG
  script:
    # this example links a release.zip file uploaded to GitLab generic package registry in 'zip it' job, but any link can be used.
    - |
      release-cli create --name "Release $CI_COMMIT_TAG" --tag-name $CI_COMMIT_TAG \
        --assets-link "{\"name\":\"release.zip\",\"url\":\"${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/releases/1.0.0/release.zip\"}"

After this job you will have an entry in your Project’s Releases page. The above job can also be defined using release keyword instead of calling the binary in a script.