Ssh command Connect() error: ssh Dial() error: ssh: handshake failed: knownhosts: key is unknown

Hello all!

I have two Centos 7 servers that are nearly identical. One being a development and the other being the production server. Everything was working fine until I updated gitlab-runner and our gitlab instance to 15.0 today. The gitlab-runner service actually sits on another server and ssh’s to these servers to deploy the projects to them. The development server one seems fine and the pipelines all appear to be working. However, when trying to run the production pipeline I keep getting this error.

Using SSH executor…

ERROR: Preparation failed: ssh command Connect() error: ssh Dial() error: ssh: handshake failed: knownhosts: key is unknown

Will be retried in 3s …
ERROR: Job failed (system failure): ssh command Connect() error: ssh Dial() error: ssh: handshake failed: knownhosts: key is unknown

I know with version 15 the disable strict host name is now set to true. I have tried switching to the user and removing the entries from .ssh/known_hosts and then manually ssh’ing to the production server successfully. I’ve looked at config.toml and they are identical except for the hostname.

I’m not sure what else it could be? Any ideas? I’ve tried running gitlab-runner in debug, but it doesn’t tell me much as to what file/user it is trying to look for.

Thanks guys.

I was able to get to this work by adding a known_hosts_file parameter to /etc/gitlab-runner/config.toml.

I added
known_hosts_file = “/home/gitlab_ci_multi_runner/.ssh/known_hosts”
in the [[runners]] section and it finally worked.

I encountered the same problem but with the VirtualBox executor.

I have opened the following issue: "ssh: handshake failed: knownhosts: key is unknown" error for VirtualBox executor (#29084) · Issues · GitLab.org / gitlab-runner · GitLab