Gitlab gives random (few times a day) Internal API error

I am experiencing random API error when using Gitlab. The error happens on both a Gitlab instance on Ubuntu 14.04 and one instance on RedHat 6.5. Once in a while, the git client would get:

GitLab: Failed to authorize your Git request: internal API unreachable fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

log in /var/log/gitlab/gitlab-shell/gitlab-shell.log:

WARN -- : Failed to connect to internal API : #

I do not see any other error message. The error happens few times a
day. We set-up monitoring doing a git pull (over ssh) on a small repo every 15
minutes, and I would get the error on average 3-4 times a day. There is
no load on the server and no error message. Switches show no error. The
RedHat one is actually brand new and only has that one repo. Feels like a
bug? Any secret config to do?