Unable to pull image via GitLab Dependency Proxy

I’m unable to pull image via Dependency Proxy. I keep getting error parsing HTTP 404 response body: unexpected end of JSON input

I have switched between shared runner and our own on-prem machines. Both are experiencing this issue.

.gitlab-ci.yml snippet:

linter:
  stage: lint
  image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/python:3.9
  allow_failure: true
  before_script:
    - pip install flake8
  script:
    - flake8 .

Attached log below.

Preparing the "docker+machine" executor

Using Docker executor with image gitlab.com:443/<REDACTED>/dependency_proxy/containers/docker:stable ...

Authenticating with credentials from job payload (GitLab Registry)

Pulling docker image gitlab.com:443/<REDACTED>/dependency_proxy/containers/docker:stable ...

WARNING: Failed to pull image with policy "always": Error response from daemon: error parsing HTTP 404 response body: unexpected end of JSON input: "" (manager.go:205:0s)

ERROR: Preparation failed: failed to pull image "gitlab.com:443/<REDACTED>/dependency_proxy/containers/docker:stable" with specified policies [always]: Error response from daemon: error parsing HTTP 404 response body: unexpected end of JSON input: "" (manager.go:205:0s)

Will be retried in 3s ...

I’m also seeing the same issue. panteparak did you find a solution ?

There was an introduce changes to how Permission is handled and there a bug in it. A closed issued that was submitted and in return cause another bug was also introduced.

If you’re granted with Owner permission (nothing less) you should be able to run CI. Or add everyone as Guest to the root level subgroup as a temporary fix. But the side effect is everyone will see all repo (but not necessary the Code etc)

I am going to file this issue in the a couple of hours (It’s night time right now here)

Ref to Issue Here

1 Like

Same issue is happening in version 14.5.2 (latest)

Using Docker executor with image server:443/project/dependency_proxy/containers/openapitools/openapi-generator-cli ...
9Authenticating with credentials from job payload (GitLab Registry)
10Pulling docker image server:443/project/dependency_proxy/containers/openapitools/openapi-generator-cli ...
11ERROR: Preparation failed: Error response from daemon: error parsing HTTP 404 response body: unexpected end of JSON input: "" (executor_docker.go:188:0s)

Is there an opened issue?

We were observing the same issue in GitLab Community Edition 14.4.1 for our pipelines triggered by specific set of users.

WARNING: Failed to pull image with policy "always": Error response from daemon: error parsing HTTP 404 response body: unexpected end of JSON input: "" (manager.go:203:0s)
ERROR: Job failed: failed to pull image "gitlab.randomhost.com:443/oss-dev/dependency_proxy/containers/maven:3.6.3-openjdk-8-slim" with specified policies [always]: Error response from daemon: error parsing HTTP 404 response body: unexpected end of JSON input: "" (manager.go:203:0s)

The error above appeared for every user which triggered the pipeline and was a member of e.g. oss-dev/my-project but not a member of top-level group oss-dev. The pipeline failed because we were using the variable CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX for caching images in GitLab Dependency Proxy. In this case the image is being pulled through top-level group which the user was not part of.

However when I changed our variable to CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX everything works as expected. :grinning:

Useful documentation:

I found the solution thanks to @panteparak’s comment.