Git clone with ssh in VirtualBox hangs

Git clone in VirtualBox hangs

I’ve been trying to switch from docker-windows executor to virtualbox executor

At the moment the gitlab runner is able to instantiate the virtualbox clone and communicate with it through SSH by forwarding the port 22 to another port as the documentation request.

My problem comes when I try to execute SSH within the Job Script found in .gitlab-ci.yml.

Our project has some dependencies that requires to be obtainer from other our own repositories… we use Unity and our only option at the moment is to rely on SSH so HTTPs is not an option.

Here is the .gitlab-ci.yml I’m using to debug the issu

    - if: $CI_COMMIT_BRANCH == "main"
    - if: $ALLOW_BRANCHES == "true"

  - build

build:       # This job runs in the build stage, which runs first.
  stage: build
    name: Win10-22H2-Unity
    pull_policy: always
    - cp $env:SSH_KEY $env:USERPROFILE/.ssh/id_ed25519
    # Create config
    - New-Item -ItemType File -Path $env:USERPROFILE/.ssh/config | Out-Null
    - Add-Content -Path $env:USERPROFILE/.ssh/config -Value "StrictHostKeyChecking accept-new" -Encoding ascii
    - attrib.exe +r $env:USERPROFILE/.ssh/config
    GIT_STRATEGY: none
    GIT_TRACE : 1
    - ssh-add $env:USERPROFILE/.ssh/id_ed25519
    - ssh-add -L
    # OpenSSH I demand your attention to say hello to me
    - ssh -V
    - Get-Content C:\\Users\\vboxuser/.ssh/config
    - Get-Content C:\\Users\\vboxuser/.ssh/id_ed25519
    - ssh -vT -y -G
    #- ssh -T -y
    #- ssh -vT -y
    #- ssh -vvT -y
    #- ssh -vvvT
    #- ssh -v -p 22
    #- ssh -vT -L
    # Detect and set the path to OpenSSH ssh.exe
    - $env:GIT_SSH = Get-Command ssh.exe
    - git clone --progress --verbose
  resource_group: build
  when: manual
  allow_failure: false

Just to explain a bit more what happens when it gets executed

# First thing I copy my ssh key from my environment variables
cp $env:SSH_KEY $env:USERPROFILE/.ssh/id_ed25519
# I create a ssh config file for the current user
New-Item -ItemType File -Path $env:USERPROFILE/.ssh/config
# I add a line to accept new connections
Add-Content -Path $env:USERPROFILE/.ssh/config -Value "StrictHostKeyChecking accept-new" -Encoding ascii
# Give some reading permissions
attrib.exe +r $env:USERPROFILE/.ssh/config
# Add the key
ssh-add $env:USERPROFILE/.ssh/id_ed25519
# List the key
ssh-add -L
# Print SSH version
ssh -V
# Prints OpenSSH_for_Windows_8.9p1, LibreSSL 3.4.2
# I tried also with the version that can be installed with Windows but same results
Get-Content C:\\Users\\vboxuser/.ssh/config
# prints what I expet the line that I added above
# StrictHostKeyChecking accept-new
Get-Content C:\\Users\\vboxuser/.ssh/id_ed25519
# Prints the correct key

# I tried to check if I could reach the host before clonning the repository
# This would correctly print a "Welcome @user" message in docker but it does happen in vbox
# I added the -G parameter to get more information
# I also tried with -v -vv and -vvv the last one does not print anything and fails right away
ssh -vT -G

This last command prints the following

user git
port 22
addressfamily any
batchmode no
canonicalizefallbacklocal yes
canonicalizehostname false
checkhostip no
compression no
controlmaster false
enablesshkeysign no
clearallforwardings no
exitonforwardfailure no
fingerprinthash SHA256
forwardx11 no
forwardx11trusted no
gatewayports no
gssapiauthentication no
gssapidelegatecredentials no
hashknownhosts no
hostbasedauthentication no
identitiesonly no
kbdinteractiveauthentication yes
nohostauthenticationforlocalhost no
passwordauthentication yes
permitlocalcommand no
proxyusefdpass no
pubkeyauthentication true
requesttty false
sessiontype default
stdinnull no
forkafterauthentication no
streamlocalbindunlink no
stricthostkeychecking accept-new
tcpkeepalive yes
tunnel false
verifyhostkeydns false
visualhostkey no
updatehostkeys true
canonicalizemaxdots 1
connectionattempts 1
forwardx11timeout 1200
numberofpasswordprompts 3
serveralivecountmax 3
serveraliveinterval 0
kexalgorithms curve25519-sha256,,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
casignaturealgorithms ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,,,rsa-sha2-512,rsa-sha2-256
loglevel DEBUG
securitykeyprovider internal
xauthlocation /usr/X11R6/bin/xauth
identityfile ~/.ssh/id_rsa
identityfile ~/.ssh/id_ecdsa
identityfile ~/.ssh/id_ecdsa_sk
identityfile ~/.ssh/id_ed25519
identityfile ~/.ssh/id_ed25519_sk
identityfile ~/.ssh/id_xmss
identityfile ~/.ssh/id_dsa
canonicaldomains none
globalknownhostsfile __PROGRAMDATA__\ssh/ssh_known_hosts __PROGRAMDATA__\ssh/ssh_known_hosts2
userknownhostsfile C:\Users\vboxuser/.ssh/known_hosts C:\Users\vboxuser/.ssh/known_hosts2
logverbose none
permitremoteopen any
addkeystoagent false
forwardagent no
connecttimeout none
tunneldevice any:any
canonicalizePermittedcnames none
controlpersist no
escapechar ~
ipqos af21 cs1
rekeylimit 0 0
streamlocalbindmask 0177
syslogfacility USER

This parameters are exactly the same that if I log into the running VM through ssh and try to execute it manually with the same command, but I do get a Welcome message.

This is my gitlab runner configuration file

concurrent = 1
check_interval = 0
shutdown_timeout = 0
log_level = "debug"

  session_timeout = 1800

  name = "VirtualBoxLaptop"
  url = ""
  id = 13
  token = "XXXXX"
  token_obtained_at = 2023-12-27T12:50:45Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "virtualbox"
  shell = "pwsh"
  output_limit = 20000
  builds_dir = "e:/workload/"
    user = "vboxuser"
    password = "test"
    port = "22"
    disable_strict_host_key_checking = true
    base_name = "Win10-22H2-BaseBox"
    allowed_images = [".*"]
    base_folder = ""
    disable_snapshots = true
    start_type = "gui"

Important Note

Since the runner hangs I usually can have the time to log into the VM through ssh just like the gitlab-runner service probably is doing and I can execute SSH and clone the repository without any issues. I also tried with SSH -vT -G and it works and the parameters are exactly the same I compared the output with beyond compare and they are exactly the same no differences.

Maybe I’m missing something… maybe the ports could be blocked

I also find very strange that if I use -v o -vv in ssh I don get any output and if I use -vvv it fail right away without logs. This doesn’t happen if I log into the VMs with SSH and try to do the same then it behaves as expected.

Thanks for taking the time, any ideas on how to troubleshoot or workaround the issue will help! :blush: