Our Omnibus (19.4.5) installation broke overnight and is currently unusable. Various webpages don’t render their content properly anymore, e.g. login page renders plain text, repository pages only show the sidebar not the actual content and many more. Additionally, https clones error out, while ssh clones still work.
Browser console lists various errors for loading different assets (.chunk.js, .css, .svg) either with net::ERR_INCOMPLETE_CHUNKED_ENCODING 200 (OK) or net::ERR_ABORTED 502 (Bad Gateway).
The instance is behind a separate HAproxy that logs 502s for the various assets when accessing via the browser. In the GitLab logs however, there seems to be no error. Requesting the assets directly via curl works with http 200.
Single example for main.da81af73.chunk.js when loading the login page:
HAproxy log:
haproxy[96826]: GITLAB_DEBUG | from=139.30.106.179 | to=gitlab-test-app | referer='``https://redacted/users/sign_in``' | req='GET /assets/webpack/main.da81af73.chunk.js' | status=502 | termination=PH-- | rate=3 | ua='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.0.0 Safari/537.36' | chua='"Not:A-Brand";v="8", "Chromium";v="156", "Google Chrome";v="156"' | platform='"Windows"' | lang='en-US,en;q=0.9' | times=0 2 -1 4 | bytes=12103
Workhorse log:
{"correlation_id":"01M6K6ZMPZZT5AY51JR26XHTJM","encoding":"gzip","file":"/opt/gitlab/embedded/service/gitlab-rails/public/assets/webpack/main.da81af73.chunk.js","level":"info","method":"GET","msg":"Send static file","time":"2026-11-04T13:01:48+01:00","uri":"/assets/webpack/main.da81af73.chunk.js"}
{"backend_id":"rails","body_limit":104857600,"content_type":"text/javascript; charset=utf-8","correlation_id":"01M6K6ZMPZZT5AY51JR26XHTJM","duration_ms":1,"host":"redacted","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","read_bytes":799,"referrer":"redacted/users/sign_in","remote_addr":"redacted","remote_ip":"redacted","route":"^/assets/","route_id":"assets","status":200,"system":"http","time":"2026-11-04T13:01:48+01:00","ttfb_ms":0,"uri":"/assets/webpack/main.da81af73.chunk.js","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.0.0 Safari/537.36","written_bytes":266232}
Any ideas where to look?
Edit: Seems like an automatic HAproxy update is the culprit. See Bug #2172897 “haproxy 2.4.30-0ubuntu0.22.04.3: CVE-2026-110354 b...” : Bugs : haproxy package : Ubuntu
Edit2: Seems to be fixed in https://launchpadlibrarian.net/883612051/haproxy_2.4.30-0ubuntu0.22.04.4_source.changes