Server verification failed

I have setup my own gitlab server. With http it works fine but when I use ssl I am unable to clone from the server.
It gives me this error .
server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile:None
I have tried disabling ssl verification in gitconfig and it works but thats not the solution I want. I have tries signing with local certificate too but same error. I have checked everything my server clock is right, I have updated ca-certificates too but nothing helps.