Gitlab memory usage

Hey,

I got some memory problems with gitlab. Recently I switched to a new server and some time after installing gitlab I saw it takes much more memory than on my old server.

Currently:
sidekiq 4.2.7 gitlab-rails: 645MB
unicorn: 540MB

As I only own a vserver it’s a big problem for me if something like that takes so much memory. Is this a bug? Is there a way to fix it?

Thanks in Advance