Hi, I want to install Gitlab with 5k architecture on Rhel8 VMs, I have already provisioned the VMs and tested that they are reachable and can communicate with one another, and I’m using the GET tool.
although the consul nodes are configured, they are not forming a cluster, one node is returned when running the command to get consul nodes. I tried the workarounds mentioned in the troubleshooting part of the GitLab documentation but in vain.
Any help will be appreciated. thanks
Perhaps you can link what documentation you have been following to install and configure Gitlab? Especially for the ansible stuff as well for deploying this?
Based on what you have written so far, it’s impossible to help without you providing more detail.
I followed those
steps and used the playbooks of the GET tool.
for the issue I’m facing (failed to sync remote state: No cluster leader) I tried:
adding bind_addr, restarting consul and Recreate from scratch.
Not sure if these posts might help, when searching for the failed to sync remote state:
There’s a lot going on in these logs and I wonder if all your attempts have created existing state in the data directories that is now causing follow-on complications. Try wiping all your Consul data directories, and starting from scratch, with a...
03:13AM - 18 Dec 15 UTC
07:19PM - 18 Dec 15 UTC
I have 3 nodes Consul "servers" running which has been working fine on `0.5.2`.
We just migrated to `0.6.0` and I'm seeing the following:
2015/12/18 03:10:34 [WARN] serf: Failed to re-join any previously known node
2015/12/18 03:10:34 [WARN] memberlist: Refuting an alive message
2015/12/18 03:10:34 [ERR] agent: failed to sync remote state: No cluster leader
2015/12/18 03:10:34 [ERR] http: Request GET /v1/kv/gz/dns, error: No cluster leader
2015/12/18 03:10:34 consul.watch: Watch (type: key) errored: Unexpected response code: 500, retry in 5s
The http request comes from consul-template, the agent error keeps mentioning no cluster leader but I can curl successfully on that node the leader:
root@consul-i-c7296503 # http localhost:8500/v1/status/leader
HTTP/1.1 200 OK
Date: Fri, 18 Dec 2015 03:10:36 GMT
root@consul-i-c7296503 # ifconfig -a
eth0 Link encap:Ethernet HWaddr 02:8a:83:65:1a:2d
inet addr:10.42.32.34 Bcast:10.42.32.255 Mask:255.255.255.0
inet6 addr: fe80::8a:83ff:fe65:1a2d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9001 Metric:1
RX packets:88127389 errors:0 dropped:0 overruns:0 frame:0
TX packets:60203588 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:17594522899 (17.5 GB) TX bytes:14369641494 (14.3 GB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:30735544 errors:0 dropped:0 overruns:0 frame:0
TX packets:30735544 errors:0 dropped:0 overruns:0 carrier:0
RX bytes:11128750609 (11.1 GB) TX bytes:11128750609 (11.1 GB)
It happens on any restart of a single node. I also tried to restart the current leader, I do see the leader changing, but still seeing the error message.
And also this from Gitlab docs:
Documentation for GitLab Community Edition, GitLab Enterprise Edition, Omnibus GitLab, and GitLab Runner.
instead of the recreate from scratch.
The only other thing that springs to mind is that RHEL by default has firewalld running - perhaps this isn’t disabled, or perhaps the ports have not been opened up on firewalld and consul cannot communicate because of this? Hope some of those links help, as there are some consul ideas there that might help with that particular error.
@iwalker. I will test those solutions