Questions regarding GitLab Runner cache in YAML File

GitLab CI/CD
, ,
1

Hello team,

I have created my own gitlab-ci.yml for my project. The project is fairly big (about 2GB) and has lots of different projects in a single repo.

I created 2-3 stages in my pipeline, it creates python virtualenv, downloads pip’s modules, and build Rust package. I used cache to store pip’s downloaded packages.
BUT, I have 3 pip local modules that are being installed in my repo (I mean the package location is within the repo) so I added those directories to my cache as well.

Question 1: Is this ok? I mean if I cache my own repo’s some directories, it will always replace with the cached ones even if I pushed a new code in those?

Question 2: AFAIK, in every run and every time the pipeline starts, the cache restores the last archive right? I mean If a pipeline finishes, the runner holds the cache and uses that cache every time or the caches are being removed after each pipeline?

Question 3: Is there an anyway better way to improve the whole process? I need the pip downloaded packages in every stage.

variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

cache:
  paths:
    - .cache/pip
    - venv/
    - my_dir1/
    - my_dir2/

stages:
  - install_req
  - refresh_pack
  - cargo

before_script:
  - python -V
  - pip install virtualenv
  - virtualenv venv
  - source venv/bin/activate

install_req:
  stage: install_req
  script:
    - pip install --cache-dir=$PIP_CACHE_DIR psycopg2
    - cd $CI_PROJECT_DIR/my_dir1/
    - pip install --cache-dir=$PIP_CACHE_DIR -r requirements.txt
    - cd $CI_PROJECT_DIR/my_dir2/
    - pip install --cache-dir=$PIP_CACHE_DIR -r requirements.txt

refresh_pack:
  stage: refresh_pack
  script:
    - rustup toolchain install nightly

cargo:
  stage: cargo
  script:
    - cargo check