Captchas breaking our CI process

Captchas breaking our CI process

We are a large research institution with many people going online from the same IP. As a consequence, somehow our IP has gained an untrustworthy reputation, which means that a lot of our paid work consists of clicking images of airplanes in captchas.

Gitlab.com is no exception. But the real problem here is, that it also breaks our CI pipelines, since we rely on some automatic downloads of release archives from Gitlab (via CMake), which fail now because of the captchas (I do admit, our CI is in fact a robot). There is a workaround to clone the packages instead of downloading the release artifacts, but

  • it is significantly slower
  • it makes me wonder why cloning works, but downloading archives redirects to a captcha.

Of course, Gitlab.com needs to take protection measures against DOS attacks, but wouldn’t it be possible to add an exception for the download of release artifacts? There seems to exist such an exception for cloning projects.

Thanks a lot for your help!

@joergbrech there’s no way to make an exception like this on GitLab.com. For captcha, someone with admin access can mark something as “ham” (not spam), but since the captcha logs aren’t searchable, you basically need the exact timestamp along with the username/email tied to the account(s) that triggered the captcha. If you have a recent (last 24 hours) example, then you can open a support ticket for the GitLab.com team with that information.

Thanks for your quick reply @cynthia. Actually, since every access to Gitlab.com triggers the captcha, already the download from a public project fails without any login. We are trying to download Eigen3 using CMake from http://gitlab.com/libeigen/eigen/-/archive/3.3.0/eigen-3.3.0.tar.gz:

Here is a download log (its a couple of days old, because we switched from the archive download to cloning Eigen because of the Captach problems. Also, I added some spaces in gitlab .com URLs in the log, because as a new user, I am not allowed to post more than 10 links in this forum.)

Download Log excerpt
-- Downloading Eigen library
         --- LOG BEGIN ---
           Trying 172.65.251.78:80...
  Connected to gitlab .com (172.65.251.78) port 80 (#0)
  GET /libeigen/eigen/-/archive/3.3.0/eigen-3.3.0.tar.gz HTTP/1.1
  Host: gitlab .com
  User-Agent: curl/7.71.1
  Accept: */*
  
  Mark bundle as not supporting multiuse
  HTTP/1.1 301 Moved Permanently
  Date: Tue, 03 Aug 2021 16:40:28 GMT
  Transfer-Encoding: chunked
  Cache-Control: max-age=3600
  Expires: Tue, 03 Aug 2021 17:40:28 GMT
  Location:
  https://gitlab.com/libeigen/eigen/-/archive/3.3.0/eigen-3.3.0.tar.gz
  X-Content-Type-Options: nosniff
  Server: cloudflare
  CF-RAY: 6790fceb7e464a74-FRA
  Connection: Keep-Alive
  Age: 86
  
  Ignoring the response-body
  [5 bytes data]
  Connection #0 to host gitlab.com left intact
  Issue another request to this URL:
  'https://gitlab.com/libeigen/eigen/-/archive/3.3.0/eigen-3.3.0.tar.gz'
    Trying 172.65.251.78:443...
  Connected to gitlab .com (172.65.251.78) port 443 (#1)
  ALPN, offering h2
  ALPN, offering http/1.1
  successfully set certificate verify locations:
    CAfile: /etc/ssl/certs/ca-certificates.crt
    CApath: /etc/ssl/certs
  [5 bytes data]
  TLSv1.3 (OUT), TLS handshake, Client hello (1):
  [512 bytes data]
  [5 bytes data]
  TLSv1.3 (IN), TLS handshake, Server hello (2):
  [122 bytes data]
  [5 bytes data]
  [5 bytes data]
  [1 bytes data]
  TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
  [19 bytes data]
  TLSv1.3 (IN), TLS handshake, Certificate (11):
  [4542 bytes data]
  TLSv1.3 (IN), TLS handshake, CERT verify (15):
  [264 bytes data]
  TLSv1.3 (IN), TLS handshake, Finished (20):
  [52 bytes data]
  [5 bytes data]
  TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
  [1 bytes data]
  [5 bytes data]
  [1 bytes data]
  TLSv1.3 (OUT), TLS handshake, Finished (20):
  [52 bytes data]
  SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
  ALPN, server accepted to use h2
  Server certificate:
   subject: CN=gitlab.com
   start date: Apr 12 00:00:00 2021 GMT
   expire date: May 11 23:59:59 2022 GMT
   subjectAltName: host "gitlab.com" matched cert's "gitlab.com"
   issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
   SSL certificate verify ok.
  Using HTTP2, server supports multi-use
  Connection state changed (HTTP/2 confirmed)
  Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
  len=0
  [5 bytes data]
  [1 bytes data]
  [5 bytes data]
  [1 bytes data]
  [5 bytes data]
  [1 bytes data]
  Using Stream ID: 1 (easy handle 0x3235c80)
  [5 bytes data]
  [1 bytes data]
  GET /libeigen/eigen/-/archive/3.3.0/eigen-3.3.0.tar.gz HTTP/2
  Host: gitlab.com
  user-agent: curl/7.71.1
  accept: */*
  
  [5 bytes data]
  [1 bytes data]
  TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
  [230 bytes data]
  TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
  [230 bytes data]
  old SSL session ID is stale, removing
  [5 bytes data]
  [1 bytes data]
  Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
  [5 bytes data]
  [1 bytes data]
  [5 bytes data]
  [1 bytes data]
  The requested URL returned error: 403
  [5 bytes data]
  [1 bytes data]
  stopped the pause stream!
  Connection #1 to host gitlab .com left intact
  
         --- LOG END ---
         
    
CMakeFiles/eigen-populate.dir/build.make:110: recipe for target 'eigen-populate-prefix/src/eigen-populate-stamp/eigen-populate-download' failed
make[2]: *** [eigen-populate-prefix/src/eigen-populate-stamp/eigen-populate-download] Error 1
CMakeFiles/Makefile2:94: recipe for target 'CMakeFiles/eigen-populate.dir/all' failed
make[1]: *** [CMakeFiles/eigen-populate.dir/all] Error 2
Makefile:102: recipe for target 'all' failed
make: *** [all] Error 2