Edit Grafana config.ini with docker

Observability
1

Hi guys!

Currently running Gitlab in Docker, as I migrated from source (finally! :slight_smile:)
However, I am running into some issues.

First some things about my configuration. I am running Gitlab behind a reverse proxy. This reverse proxy takes care of LetsEncrypt certificates, so I had to change the config/gitlab.rb file a bit to get this working. I changes these values:

external_url 'https://my-gitlab-domain.tld'
nginx['listen_port'] = 80
nginx['listen_https'] = false

Gitlab works pretty good behind the reverse proxy. Gitlab CI works also fine with this config.

Now, I would like to get the integrated Grafana working. In Settings → Metrics and profiling → Metrics - Grafana I enabled Grafana and went to the url. I clicked the “Sign in with Gitlab” button, and eventually I get to this Grafana error page ( message shown: (login.OAuthLogin(NewTransportWithCode)")

I digged through the logs and I found the following line:

    lvl=eror msg=login.OAuthLogin(NewTransportWithCode) logger=context userId=0 orgId=0 uname= error="Post https://my-gitlab-domain.tld/oauth/token: dial tcp 172.17.0.2:443: connect: connection refused"

Now, it seems that Grafana tries to connect with the SSL version of the API (which is good - by default). This does not work in my setup, because in /etc/hosts there is the ‘my-gitlab-domain.tld’ mapped to 172.17.0.2. And Gitlab is not listening on port 443, so Grafana cannot connect.

I tried to edit the ‘data/grafana/grafana.ini’ file, and set the following values to non-https:
auth_url
token_url
api_url

I saved the file and restarted the container with docker-compose down && docker-compose up. Unfortunately, the values have been overwritten.

My question being: is there any way to get this working in my setup?

Thank you in advance.

1 Like
2

Okay, so it seems the url comes from this https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/monitoring/templates/grafana_ini.erb#L251 file. Is there any way to edit this file after starting the container?

3

I found out that after editing the file, running these commands inside the container work:

/opt/gitlab/bin/gitlab-ctl stop grafana
/opt/gitlab/bin/gitlab-ctl start grafana

Not pretty, but hey: it works. If anyone else has a better solution, feel free to post it.

4

Thanks. I ran into the same issue.
Did you develop a solution to have this permanently? every upgrade and thus restart these settings get overwritten.

5

Hi
I had the same issue, and thanks to @Thijmen it worked, also the issue didn’t came back after restart container.
FYI, I do docker rm the container each time I stop it

1 Like
6

Hi
I have the same issue and this solution is not efficient for production env.

7

I found a solution that is easier to automate. I haven’t gotten around to testing the full effects yet, so no guarantees.

In /etc/hosts there are entries like this at the bottom:
10.103.0.30 gitlab.domain.tld git
172.30.0.3 gitlab.domain.tld git

After I commented out these entries, the login to Grafana worked