How to proxy_pass to Jenkins:8080 behind GitLab NGINX?

How to Use GitLab
1

I am trying to run Jenkins CI listening on port 8081 behind GitLab NGINX server.

  • I am a GitLab and NGINX newbie.
  • I am running GitLab and Jenkins on a CentOS 6.7 server IP 172.17.144.81
  • Jenkins has no prefix and is running on port 8081

I added following line to gitlab.rb

nginx['custom_nginx_config'] = "include /var/opt/gitlab/nginx/conf/nginx-jenkins.conf;"

After running gitlab-ctl reconfigure and gitlab-ctl restart, the /var/opt/gitlab/nginx/conf/nginx.conf file now contains:

http {
...
  include /var/opt/gitlab/nginx/conf/nginx-jenkins.conf;
}

The /var/opt/gitlab/nginx/conf/nginx-jenkins.conf looks like this:

server {
   listen 80;

   location /jenkins {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_pass http://127.0.0.1:8081;
   }
}
  1. http://172.17.144.81:8081 opens Jenkins as expected
  2. http://172.17.144.81 opens GitLab http://172.17.144.81/users/sign_in as expected
  3. http://172.17.144.81/jenkins also opens GitLab http://172.17.144.81/users/sign_in
    jenkins

I expected it to pass through to 8081 and open Jenkins?

Thanks in advance for any suggestions,

-Ed

2

Looks like it is soemthing in the gitlab-http.conf that prevents my nginx-jenkins.conf from working:

It works if I comment out include of gitlab-http.conf

File: nginx.conf

http {
 ....
  include /opt/gitlab/embedded/conf/mime.types;

  #include /var/opt/gitlab/nginx/conf/gitlab-http.conf;
  include /var/opt/gitlab/nginx/conf/nginx-jenkins.conf;
}
3

The problem appears to be something in the gitlab-http.conf is pre-empting my nginx-jenkins.con proxy pass.

What sections should I be looking for?

Is there a way to add an exclude command to not listen for /jenkins/ in the gitlab-http.conf?

# This file is managed by gitlab-ctl. Manual changes will be
# erased! To change the contents below, edit /etc/gitlab/gitlab.rb
# and run `sudo gitlab-ctl reconfigure`.

## GitLab
## Modified from https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/gitlab-ssl & https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/nginx/gitlab
##
## Lines starting with two hashes (##) are comments with information.
## Lines starting with one hash (#) are configuration parameters that can be uncommented.
##
##################################
##        CHUNKED TRANSFER      ##
##################################
##
## It is a known issue that Git-over-HTTP requires chunked transfer encoding [0]
## which is not supported by Nginx < 1.3.9 [1]. As a result, pushing a large object
## with Git (i.e. a single large file) can lead to a 411 error. In theory you can get
## around this by tweaking this configuration file and either:
## - installing an old version of Nginx with the chunkin module [2] compiled in, or
## - using a newer version of Nginx.
##
## At the time of writing we do not know if either of these theoretical solutions works.
## As a workaround users can use Git over SSH to push large files.
##
## [0] https://git.kernel.org/cgit/git/git.git/tree/Documentation/technical/http-protocol.txt#n99
## [1] https://github.com/agentzh/chunkin-nginx-module#status
## [2] https://github.com/agentzh/chunkin-nginx-module
##
###################################
##         configuration         ##
###################################

upstream gitlab {
  server unix:/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket fail_timeout=0;
}

upstream gitlab-git-http-server {
  server unix:/var/opt/gitlab/gitlab-git-http-server/socket;
}


server {
  listen *:80;


  server_name gitlab.monserver.fr;
  server_tokens off; ## Don't show the nginx version number, a security best practice
  root /opt/gitlab/embedded/service/gitlab-rails/public;

  ## Increase this if you want to upload large attachments
  ## Or if you want to accept large git objects over http
  client_max_body_size 250m;


  ## Individual nginx logs for this GitLab vhost
  access_log  /var/log/gitlab/nginx/gitlab_access.log gitlab_access;
  error_log   /var/log/gitlab/nginx/gitlab_error.log;

  location / {
    ## Serve static files from defined root folder.
    ## @gitlab is a named location for the upstream fallback, see below.
    try_files $uri $uri/index.html $uri.html @gitlab;
  }

  location /uploads/ {
    ## If you use HTTPS make sure you disable gzip compression
    ## to be safe against BREACH attack.
    

    ## https://github.com/gitlabhq/gitlabhq/issues/694
    ## Some requests take more than 30 seconds.
    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   http;
    proxy_set_header    X-Frame-Options     SAMEORIGIN;

    proxy_pass http://gitlab;
  }

  ## If a file, which is not found in the root folder is requested,
  ## then the proxy passes the request to the upsteam (gitlab unicorn).
  location @gitlab {
    ## If you use HTTPS make sure you disable gzip compression
    ## to be safe against BREACH attack.
    

    ## https://github.com/gitlabhq/gitlabhq/issues/694
    ## Some requests take more than 30 seconds.
    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   http;
    proxy_set_header    X-Frame-Options     SAMEORIGIN;

    proxy_pass http://gitlab;
  }

  location ~ [-\/\w\.]+\.git\/ {
    ## If you use HTTPS make sure you disable gzip compression
    ## to be safe against BREACH attack.
    

    ## https://github.com/gitlabhq/gitlabhq/issues/694
    ## Some requests take more than 30 seconds.
    proxy_read_timeout      300;
    proxy_connect_timeout   300;
    proxy_redirect          off;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   http;

    proxy_pass http://gitlab-git-http-server;
  }

  ## Enable gzip compression as per rails guide:
  ## http://guides.rubyonrails.org/asset_pipeline.html#gzip-compression
  ## WARNING: If you are using relative urls remove the block below
  ## See config/application.rb under "Relative url support" for the list of
  ## other files that need to be changed for relative url support
  location ~ ^/(assets)/ {
    root /opt/gitlab/embedded/service/gitlab-rails/public;
    gzip_static on; # to serve pre-gzipped version
    expires max;
    add_header Cache-Control public;
  }


  error_page 502 /502.html;

  
}
4

If I move the following location section inside the gitlab-http.conf file both GitLab and Jenkins proxy_pass works.

Assuming that modifying the omnibus version of gitlab-http.conf is the only way to make it work, then
my problem has changed to how do I maintain gitlab-http.conf?

The problem is that all modifications will be destroyed if I ever run gitlab-ctl reconfigure. Thank you for your help. Hopefully I can find a gitlab user forum to research solutions.

  location ^~ /jenkins {
    proxy_pass          http://localhost:8081;
    proxy_read_timeout  90;
 
    # Fix the “It appears that your reverse proxy set up is broken" error.
    proxy_redirect      http://localhost:8080 $scheme://example.com;
 
    # Optionally, require HTTP basic auth.
    # auth_basic "Please authenticate to use Jenkins";
    # auth_basic_user_file /opt/nginx/htpasswd;
  }
5

Hi edofthemountain,

we had exactly the same problem as you. Your writings here were really helpful to us and finally we actually got it working. It’s a rather involved way, but it works and is (at least to the outside) quite clean.

We configured gitlab to just listen on localhost and then set the custom_nginx_config with two locations. The first to proxy_pass on /jenkins to the jenkins server and the second one to proxy_pass on / to gitlab.

In detail, we did the following configurations, in gitlab.rb:

nginx['listen_addresses'] = ['127.0.0.1']
nginx['listen_port'] = 8080
nginx['custom_nginx_config'] = "include /var/opt/gitlab/nginx/conf/nginx-jenkins.conf;"

After this, the gitlab-part of the nginx configuration generated by gitlab just lists a server for 127.0.0.1 listening in 8080.
Thus, we can define a server in nginx-jenkins.conf to listen on yourdomain.com:80 and to proxy_pass to jenkins and to gitlab. Our nginx-jenkins.conf looks like this:

server {
   listen yourdomain.com:80;

   location /jenkins/ {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_pass http://127.0.0.1:9000;
   }

   location / {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_pass http://127.0.0.1:8080;
   }
}

This configuration assumes, that your jenkins is running on port 9000. Of course, you also have to add the parameter --prefix=/jenkins to your jenkins configuration.

EDIT: We had some problems with this approach when we tried to enable TLS. But, we found a much simpler way to achieve what we wanted. There is the option nginx['custom_gitlab_server_config']. You can set it to any string you want and it will be copied into the server block of the nginx configuration generated by gitlab.
So, we just used the following line:

`nginx['custom_gitlab_server_config'] = "location ^~ /jenkins/ {\n      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;\n      proxy_set_header Host $http_host;\n      proxy_pass http://127.0.0.1:9000;\n}"`

After that, everything worked just fine.