Docker push 404 error

We are running a self host gitlab instance behind an AWS Application load balancer. Docker login appears to succeed. Then when I go to push an image I get a 404 error, with a body of text. I have attempted various configurations of the gitlab.rb file. Current config looks as:

################################################################################
## Container registry settings
##! Docs: https://docs.gitlab.com/ce/administration/container_registry.html
################################################################################

registry_external_url 'https://registry.mydomain.net'

registry['env'] = {
  "REGISTRY_HTTP_RELATIVEURLS" => true
}

### Settings used by GitLab application
  gitlab_rails['registry_enabled'] = true
  gitlab_rails['registry_host'] = "registry.mydomain.net"
#  gitlab_rails['registry_port'] = "5050"
 gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry"
################################################################################
## Registry NGINX
################################################################################

# All the settings defined in the "GitLab NGINX" section are also available in this "Registry NGINX" section
# You just have to change the key "nginx['some_settings']" with "registry_nginx['some_settings']"

# Below you can find settings that are exclusive to "Registry NGINX"
 registry_nginx['enable'] = true
 registry_nginx['listen_https'] = false
 registry_nginx['redirect_http_to_https'] = true
 registry_nginx['listen_port'] = 5678
 registry_nginx['proxy_set_headers'] = {
#  "Host" => "$http_host",
#  "X-Real-IP" => "$remote_addr",
#  "X-Forwarded-For" => "$proxy_add_x_forwarded_for",
  "X-Forwarded-Proto" => "https",
  "X-Forwarded-Ssl" => "on"
 }

I receive this error

error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: "<!DOCTYPE html>\n<html>\n<head>\n  <meta content=\"width=device-width, initial-scale=1, maximum-scale=1\" name=\"viewport\">\n  <title>The page you're looking for could not be found (404)</title>\n  <style>\n    body {\n      color: #666;\n      text-align: center;\n      font-family: \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n      margin: auto;\n      font-size: 14px;\n    }\n\n    h1 {\n      font-size: 56px;\n      line-height: 100px;\n      font-weight: 400;\n      color: #456;\n    }\n\n    h2 {\n      font-size: 24px;\n      color: #666;\n      line-height: 1.5em;\n    }\n\n    h3 {\n      color: #456;\n      font-size: 20px;\n      font-weight: 400;\n      line-height: 28px;\n    }\n\n    hr {\n      max-width: 800px;\n      margin: 18px auto;\n      border: 0;\n      border-top: 1px solid #EEE;\n      border-bottom: 1px solid white;\n    }\n\n    img {\n      max-width: 40vw;\n      display: block;\n      margin: 40px auto;\n    }\n\n    a {\n      line-height: 100px;\n      font-weight: 400;\n      color: #4A8BEE;\n      font-size: 18px;\n      text-decoration: none;\n    }\n\n    .container {\n      margin: auto 20px;\n    }\n\n    .go-back {\n      display: none;\n    }\n\n  </style>\n</head>\n\n<body>\n  <a href=\"/\">\n    <img src='data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iMjAwIiBoZWlnaHQ9IjE5MiIgdmlld0JveD0iMCAwIDI1IDI0IiBmaWxsPSJub25lIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogIDxwYXRoIGQ9Im0yNC41MDcgOS41LS4wMzQtLjA5TDIxLjA4Mi41NjJhLjg5Ni44OTYgMCAwIDAtMS42OTQuMDkxbC0yLjI5IDcuMDFINy44MjVMNS41MzUuNjUzYS44OTguODk4IDAgMCAwLTEuNjk0LS4wOUwuNDUxIDkuNDExLjQxNiA5LjVhNi4yOTcgNi4yOTcgMCAwIDAgMi4wOSA3LjI3OGwuMDEyLjAxLjAzLjAyMiA1LjE2IDMuODY3IDIuNTYgMS45MzUgMS41NTQgMS4xNzZhMS4wNTEgMS4wNTEgMCAwIDAgMS4yNjggMGwxLjU1NS0xLjE3NiAyLjU2LTEuOTM1IDUuMTk3LTMuODkuMDE0LS4wMUE2LjI5NyA2LjI5NyAwIDAgMCAyNC41MDcgOS41WiIKICAgICAgICBmaWxsPSIjRTI0MzI5Ii8+CiAgPHBhdGggZD0ibTI0LjUwNyA5LjUtLjAzNC0uMDlhMTEuNDQgMTEuNDQgMCAwIDAtNC41NiAyLjA1MWwtNy40NDcgNS42MzIgNC43NDIgMy41ODQgNS4xOTctMy44OS4wMTQtLjAxQTYuMjk3IDYuMjk3IDAgMCAwIDI0LjUwNyA5LjVaIgogICAgICAgIGZpbGw9IiNGQzZEMjYiLz4KICA8cGF0aCBkPSJtNy43MDcgMjAuNjc3IDIuNTYgMS45MzUgMS41NTUgMS4xNzZhMS4wNTEgMS4wNTEgMCAwIDAgMS4yNjggMGwxLjU1NS0xLjE3NiAyLjU2LTEuOTM1LTQuNzQzLTMuNTg0LTQuNzU1IDMuNTg0WiIKICAgICAgICBmaWxsPSIjRkNBMzI2Ii8+CiAgPHBhdGggZD0iTTUuMDEgMTEuNDYxYTExLjQzIDExLjQzIDAgMCAwLTQuNTYtMi4wNUwuNDE2IDkuNWE2LjI5NyA2LjI5NyAwIDAgMCAyLjA5IDcuMjc4bC4wMTIuMDEuMDMuMDIyIDUuMTYgMy44NjcgNC43NDUtMy41ODQtNy40NDQtNS42MzJaIgogICAgICAgIGZpbGw9IiNGQzZEMjYiLz4KPC9zdmc+Cg==' alt=\"GitLab\"/>\n  </a>\n  <h1>\n    404\n  </h1>\n  <div class=\"container\">\n    <h3>The page could not be found or you don't have permission to view it.</h3>\n    <hr />\n    <p>The resource that you are attempting to access does not exist or you don't have the necessary permissions to view it.</p>\n    <p>Make sure the address is correct and that the page hasn't moved.</p>\n    <p>Please contact your GitLab administrator if you think this is a mistake.</p>\n    <a href=\"javascript:history.back()\" class=\"js-go-back go-back\">Go back</a>\n  </div>\n  <script>\n    (function () {\n      var goBack = document.querySelector('.js-go-back');\n\n      if (history.length > 1) {\n        goBack.style.display = 'inline';\n      }\n    })();\n  </script>\n</body>\n</html>\n"

For anyone else that runs into this. This error ended up not having to be with the gitlab.rb config. The issue was from routing, my load balancer wasn’t forwarding request from registery.gitlab to the proper port on the server

1 Like