Making long build times short by using cache?

GitLab CI/CD
1

I use GitLab Community Edition’s CI to build a C++ project. During the build stage i have two jobs that run in parallel, one creating the build on windows and the other one that’s creating the build on linux.

All good, except … The build takes around 10+ minutes on linux and 20+ minutes on windows. Most of this time is spent in downloading/compiling dependencies (like google protobuf, some boost libs, gtest, libressl, etc).

One solution that I tried was to cache the build folder, so that in the next build (push/pipeline) the dependencies are already downloaded and compiled, and the project’s .cpp files are already compiled into objects, so that it will only compile the files that changed and re-link executables. This didn’t work as expected because CMake threw some errors about running in a different folder than the one where it initially ran.

Anyway, I was wondering if other people use GitLab CI for building fairly large C++ projects, and if you ran into the same problem - long build times. And what is the best approach to solve this?

P.S.: I’m using a shell runner, I also investigated a bit about other options but couldn’t really see a solution.

2

First look to the differences between cache and artifacts:
https://docs.gitlab.com/ee/ci/caching/#cache-vs-artifacts

Furthermore, when using caches you can use the key (to create a unique cache) per commit, branch what ever you want.

So sharing the cache along the jobs within the same branch you could use:

cache:
   key: ${CI_COMMIT_REF_SLUG}
   paths:
       - .....

Sometimes you want to execute some command(s) before you want to start the jobs. Like in my case I don’t have a C++ project, but NodeJS. In my case this would be logical:

before_script:
  - npm install

Here you have an example I use for my Gitlab CI NodeJS project:

cache:
  key: ${CI_BUILD_REF_NAME}
  paths:
    - node_modules/
    
before_script:
    - npm install
    - node ./node_modules/protractor/bin/webdriver-manager update

build:
  tags:
    - build
  stage: build
  script:
    - npm run build

test:unittests:
  tags:
    - test
  stage: test
  script:
    - npm run citest

test:intergration:
  tags:
    - test
  stage: test
  script:
    - npm run e2e

test:lint:
  tags:
    - test
  stage: test
  script:
    - npm run lint

More info…