HOWTO: Cloning From Linux Guest On VirtualBox

Hi.
I am able to connect and clone from my Linux box. However, from a VirtualBoox Linux Guest, on Windows host I am unable to connect.

My config:
user.name=rmnair
user.email=rmnair@aradasystems.com
core.repositoryformatversion=0
core.filemode=true
core.bare=false
core.logallrefupdates=true

I have generated the rsa keys using:
[rm@rm-VirtualBox ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/rm/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/rm/.ssh/id_rsa.
Your public key has been saved in /home/rm/.ssh/id_rsa.pub.
The key fingerprint is:
52:bf:d4:d0:b5:9d:35:6a:89:ae:12:1c:75:c2:dc:3d rm@rm-VirtualBox

The key has been added to the GitLab

Running the git clone:

[rm@rm-VirtualBox ~] git clone git@172.20.1.25:dsrc_apps/apps.git Cloning into 'apps'... Agent admitted failure to sign using the key. git@172.20.1.25's password: Permission denied, please try again. git@172.20.1.25's password: [rm@rm-VirtualBox ~]

What do I do?

Thanks
RM