Persistent Docker containers between GitLab CI stages on a shared runner

I’m having a bit of trouble migrating a GitLab CI project from a private to a shared runner. In my own setup, I use the docker shell with the /var/run/docker.sock from the host mounted inside it, so that any docker commands are actually executed in the host, and most importantly the changes done on it (such as started containers) persist between stages.

Now on a public runner I can’t even get this simple configuration to work:

image: gitlab/dind:latest

variables:
  DOCKER_DRIVER: overlay

stages:
  - one
  - two

un:
  stage: one
  script:
    - docker run -d --name webserver nginx:1.11-alpine
    - docker exec webserver wget -O- http://localhost     # just to doublecheck that indeed works

dos:
  stage: two
  script:
    - docker exec webserver wget -O- http://localhost   # webserver container will no longer exist

Output of “un”:

Running with gitlab-ci-multi-runner 9.1.0 (0118d89)
  on docker-auto-scale (e11ae361)
Using Docker executor with image gitlab/dind:latest ...
Using docker image sha256:0872d550b03ce9869037d8c5ec4e40cfc373fd14ff9bfb4d5cbae4196e15cd56 for predefined container...
Pulling docker image gitlab/dind:latest ...
Using docker image gitlab/dind:latest ID=sha256:cc674e878f23bdc3c36cc37596d31adaa23bca0fc3ed18bea9b59abc638602e1 for build container...
Running on runner-e11ae361-project-3181772-concurrent-0 via runner-e11ae361-machine-1494399735-722dcd53-digital-ocean-2gb...
Cloning repository...
Cloning into '/builds/1ma/shipomatic'...
Checking out ddb50c1b as master...
Skipping Git submodules setup
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted
$ docker run -d --name webserver nginx:1.11-alpine
Unable to find image 'nginx:1.11-alpine' locally
1.11-alpine: Pulling from library/nginx
709515475419: Pulling fs layer
4b21d71b440a: Pulling fs layer
c92260fe6357: Pulling fs layer
ed383a1b82df: Pulling fs layer
ed383a1b82df: Waiting
c92260fe6357: Verifying Checksum
c92260fe6357: Download complete
709515475419: Verifying Checksum
709515475419: Download complete
ed383a1b82df: Verifying Checksum
ed383a1b82df: Download complete
709515475419: Pull complete
4b21d71b440a: Verifying Checksum
4b21d71b440a: Download complete
4b21d71b440a: Pull complete
c92260fe6357: Pull complete
ed383a1b82df: Pull complete
Digest: sha256:5aadb68304a38a8e2719605e4e180413f390cd6647602bee9bdedd59753c3590
Status: Downloaded newer image for nginx:1.11-alpine
1c27d45045e86a07ae1d8c9b54cdf52b280204bc8f0f9c18af23c23069f13e47
$ docker exec webserver wget -O- http://localhost
Connecting to localhost (127.0.0.1:80)

-                    100% |*******************************|   612   0:00:00 ETA
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted
Job succeeded

Output of “dos”:

Running with gitlab-ci-multi-runner 9.1.0 (0118d89)
  on docker-auto-scale (4e4528ca)
Using Docker executor with image gitlab/dind:latest ...
Using docker image sha256:7f783e6940b0ad097366a16d81252d9ce5bb6a48565d190fe32f495139507e25 for predefined container...
Pulling docker image gitlab/dind:latest ...
Using docker image gitlab/dind:latest ID=sha256:cc674e878f23bdc3c36cc37596d31adaa23bca0fc3ed18bea9b59abc638602e1 for build container...
Running on runner-4e4528ca-project-3181772-concurrent-0 via runner-4e4528ca-machine-1494401641-6d7383b5-digital-ocean-2gb...
Cloning repository...
Cloning into '/builds/1ma/shipomatic'...
Checking out ddb50c1b as master...
Skipping Git submodules setup
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted
$ docker exec webserver wget -O- http://localhost
Error response from daemon: No such container: webserver
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted
ERROR: Job failed: exit code 1

So, two questions.

  1. Can this kind of approach be made to work in a shared runner (e.g. one first step to start some containers, and others to do some work with them)?

  2. Is there any way to get rid of these pesky ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted errors?