Gitlab runner and git+ssh dependencies

I have a gitlab runner inside a docker container as described here.
I have set up my ssh keys like this:
https://docs.gitlab.com/ee/ci/ssh_keys/

Inside my package.json file I have a configuration that looks like this:

  "devDependencies": {
   "foo": "git+ssh://git@gitlab.foo.net:bar/baz.git#v0.1.0",
  }

My .gitlab-ci.yml looks like this:

image: node:6.11.3
cache:
  paths:
  - node_modules/
stages:
  - install
install:
  stage: install
  before_script:
  - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
  - eval $(ssh-agent -s)
  - echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh
  script:
    - echo "Installing dependencies..."
    - yarn install
  tags:
    - debian

When I run the pipeline, it fails at the install “script” stage when trying to install my private dependency:

Command: git
Arguments: clone git@gitlab.foo.net:bar/baz.git /usr/local/share/.cache/yarn/v1/.tmp/asdfasd
Directory: /builds/my-directory
Output:
Cloning into '/usr/local/share/.cache/yarn/v1/.tmp/asdfasd'...
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Interestingly enough, when I go into the docker bash, I can clone and yarn install the repository fine. I’m guessing this is an issue with the gitlab-runner.