Version: Ominbus 14.8.0-ee, self-managed
We have a CI pipeline that pushs (actually, re-tag) many images to container registry, sometimes we can see following error:
Running with gitlab-runner 12.9.0 (4c95e3ad) on runner@X1 7aa08e38 Preparing the "docker" executor 00:01 Using Docker executor with image docker:latest ... Using locally found image version due to if-not-present pull policy Using docker image sha256:3e37c4295eb1af254011e47172d90c89dd04683afcab381404910100d670b100 for docker:latest ... Preparing environment 00:01 Running on runner-8bb0e839-project-123-concurrent-0 via ci-runner... Getting source from Git repository 00:02 Fetching changes with git depth set to 50... Initialized empty Git repository in /builds/8aa0e949/0/mygroup/myproject/.git/ Created fresh repository. From https://gitlab.company.com:4443/mygroup/myproject * [new ref] df273fbadfb32a38ac32e5356d76855279e7dacd -> refs/pipelines/47340 * [new tag] v1.0.0 -> v1.0.0 * [new tag] v1.0.0rc -> v1.0.0rc Checking out df27dfba as v1.0.0... Skipping Git submodules setup Restoring cache 00:02 Downloading artifacts 00:02 Running before_script and script . . . Release to prod (1.0.0) ... => image: image1 => pull image: registry.company.com:4567/mygroup/myproject/image1:1.0.1 1.0.1: Pulling from mygroup/myproject/image1 b56ae66c2937: Already exists ... Digest: sha256:48d0ed7ceb1551a4ff5ee01cb85e0d066792b2fff0a2895015df956b42fbac4f Status: Downloaded newer image for registry.company.com:4567/mygroup/myproject/image1:1.0.1 registry.company.com:4567/mygroup/myproject/image1:1.0.1 => tag to registry.company.com:4567/mygroup/myproject/image1:prj_1.0.0 => push registry.company.com:4567/mygroup/myproject/image1:prj_1.0.0 The push refers to repository [registry.company.com:4567/mygroup/myproject/image1] 5f70bf28a086: Preparing ... ced227d13862: Layer already exists prj_1.0.0: digest: sha256:46d0ed7ceb1551a4fe5ee01cb85e0d067792b2fff0a2897015df956b426bac4f size: 2619 . . . => image: image25 => pull image: registry.company.com:4567/mygroup/myproject/image25:1.0.1 1.0.1: Pulling from mygroup/myproject/image25 7328113b0321: Already exists fdde70890fe8: Already exists Digest: sha256:b1dcd639f285df00357fc2cd8ad754985fb807efb776e0f68b0fc3b729328e47 Status: Downloaded newer image for registry.company.com:4567/mygroup/myproject/image25:1.0.1 registry.company.com:4567/mygroup/myproject/image25:1.0.1 => tag to registry.company.com:4567/mygroup/myproject/image25:prj_1.0.0 => push registry.company.com:4567/mygroup/myproject/image25:prj_1.0.0 The push refers to repository [registry.company.com:4567/mygroup/myproject/image25] 54f58195fed5: Preparing 7e1d4974e536: Preparing 08e8d47541e0: Preparing error parsing HTTP 429 response body: invalid character 'R' looking for beginning of value: "Retry later\n" Failed, clean up ... . .
The HTTP error code is 429 (too many requests), so I looked up gitlab config file (/etc/gitlab/gitlab.rb) try to find the rate limit option for container registry, but no luck.
Any help is appreciated.