Unable to create a container that persist between jobs (in the same pipeline)

Hi.

I have a pipeline with multiple jobs. And in the first job I’m creating an image (ci-project-build-) which I’m trying to save into a container (build_data_), so that I can reuse that image in several of following steps. However it seems like somehow it’s not persisted in between jobs, because alread in stage 2 (styles) the build_data* container is nowhere to be found.

The entire config is as follows:

image: docker:stable

services:
  - docker:stable-dind

stages:
  - build
  - styles
  - test
  - release
  - cleanup

variables:
  APP_NAME: myproj
  APP_VERSION: 0.0.1
  CONTAINER_RELEASE_IMAGE: myname/myproj:latest
  POSTGRES_HOST: postgres
  POSTGRES_USER: postgres
  POSTGRES_PASSWORD: password

build:
  before_script:
    - docker container ls
    - docker build -f Dockerfile.build -t ci-project-build-$CI_PROJECT_ID:$CI_BUILD_REF .
    - docker create
      -v /build/deps
      -v /build/_build
      -v /build/rel
      -v /root/.cache/rebar3/
      --name build_data_$CI_PROJECT_ID_$CI_BUILD_REF busybox /bin/true
    - docker images
    - docker ps -a
    - docker container ls
  tags:
    - docker
  stage: build
  script:
    - docker run --volumes-from build_data_$CI_PROJECT_ID_$CI_BUILD_REF --rm -t ci-project-build-$CI_PROJECT_ID:$CI_BUILD_REF

styles:
  tags:
    - docker
  stage: styles
  script:
    - docker container ls
    - docker volume ls
    - docker ps -a
    - docker run --rm
      --volumes-from build_data_$CI_PROJECT_ID_$CI_BUILD_REF ci-project-build-$CI_PROJECT_ID:$CI_BUILD_REF sh -c "mix credo --strict"

test:
  services:
    - postgres
    - docker:dind
  tags:
    - docker
  stage: test
  script:
    - env
    - docker run --rm
      --link $POSTGRES_NAME:postgres
      -e POSTGRES_HOST=$POSTGRES_HOST
      -e POSTGRES_PASSWORD=$POSTGRES_PASSWORD
      -e POSTGRES_USER=$POSTGRES_USER
      -e MIX_ENV=$MIX_ENV
      --volumes-from build_data_$CI_PROJECT_ID_$CI_BUILD_REF ci-project-build-$CI_PROJECT_ID:$CI_BUILD_REF sh -c "mix ecto.setup && mix test"

release:
  tags:
    - docker
  stage: release
  script:
    - docker run
      --volumes-from build_data_$CI_PROJECT_ID_$CI_BUILD_REF
      -e MIX_ENV=prod --rm -t ci-project-build-$CI_PROJECT_ID:$CI_BUILD_REF
      sh -c "mix deps.get && mix compile && mix release"
    - docker cp build_data_$CI_PROJECT_ID_$CI_BUILD_REF:/build/rel/$APP_NAME/releases/$APP_VERSION/$APP_NAME.tar.gz .
    - docker build -t $CONTAINER_RELEASE_IMAGE .
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN gitlab.example.org:4567
    - docker push $CONTAINER_RELEASE_IMAGE
  only:
    - master

cleanup_job:
  tags:
    - docker
  stage: cleanup
  script:
    - docker rm -v build_data_$CI_PROJECT_ID_$CI_BUILD_REF
    - docker rmi ci-project-build-$CI_PROJECT_ID:$CI_BUILD_REF
  when: always

Unless I misunderstand gitlab-ci, as I’m pretty new to it also, your going to start from a new docker base of docker:stable each job.

I have solved this problem for my projects by building the image and pushing that to the docker repo/registry. Then using that image for the associated projects/jobs.

This might be helpful information…