Using CI_BUILD_ID and CI_BUILD_REF and other things to identify builds

How to Use GitLab
1

What practices have people evolved for labelling binary artifacts of build processes? Here is my starter set of working practices:

  • Use $CI_BUILD_ID as the build number and mark my C,C++,C#,Java “binaries” as version major.minor.patch.CI_BUILD_ID, say “8.12.3.14313”, if for example, the CI_BUILD_ID=14313.
  • In the same metadata area where that product/file-version-info metadata lives, also tag the description or other field with the git revision that the build was created on, which is $CI_BUILD_REF
  • The canonical source repository url which is $CI_PROJECT_URL
  • Also include the machine the build was created on.
  • Also include a UTC timestamp for that build

Is that enough? Is there more that you find you need?

Warren

2

We do not allow to store big artifacts in GitLab, binaries should go to Artifactory. We use CI__PIPELINE_ID during upload as matrix parameter for Maven SNAPSHOT builds. Later stages of a pipeline now may easily request a specific Snapshot by filtering with the pipeline I’d.

3

We don’t store big artifacts inside gitlab either.

I think you are answering my question, which is more clearly put as “when the field of battle gets messy, how do you stay organized?”. It is a concern of the build task in gitlab CI to create and name its artifacts, and then execute whatever command pushes the large binary artifacts out. Probably it will push data to a storage system, as simple perhaps as a big network file server inside a company, for us.

You chose artifactory as your store? What attributes does it have that makes it a good choice for you?

Gitlab has the ability to store artifacts, but is a poor choice for binary artifacts for us. For certain types of artifacts, say java jars, or .net assemblies, or ruby gems there are already established patterns that people use. When it comes to C++ DLLs/shared-libraries, and so on, there are fewer and fewer common practices, and it seems each company/org/project does very different things.

It seems to me that having all artifacts available on an LDAP-authenticated HTTP/WEBDAV store with a directory structure and a versioning system will be required. An unsolved problem for me would be to mark assets that have become important, that someone/something depends on, and retain them, but to allow assets which have not become important to expire, so that not every commit of every repo produces a large amount of useless “noise assets”.

Our history as a software development organization has moved from monolithic builds (check out 6 gigs of stuff and run a giant ANT-style XML build that takes 6 hours) where CI is truly impossible (the minimal build interval was six hours) to a micro-approach. We’re still looking for some of the tooling bits to make this transition.

4

Hello,

we started to use the professional Artifactory version because it was much cheaper than Nexus, the way to promote artifacts between different stage repositories was much cleaner then in Nexus and it stores the binaries as blobs (SHA1 of the content), while paths and attributes are stored in a relational database, so one blob can occur in multiple places without taking more storage.

Basically Artifactory is a LDAP-authenticated HTTP/WEBDAV store with finegrained permissions.

We started with Java Artifacts, nowadays Artifactory is able to store debs, rpms, npm, docker images and can even cache GitHub repositories (stores tar.gzs of branches and tags) so this is still a good fit for us. You may create virtual repositories which provide a unified view for other included local or cached repositories.

Attributes/properties can be easily attached during a simple PUT upload as matrix parameters and may be used for filtering as well. Another nice feature are filtered resources, i.e. you may store text files and use freemarker syntax for some templating logic (insert username and a Artifactory login token during authenticated downloads, e.g.).

Concerning build dependencies, Artifactory provides plugins to store build information as well, so if you build a WAR with included JARs, the builds knows about the needed JARs and you may promote the whole dependency tree.

1 Like