This worked for me in Ubuntu 16 and let me install an older version of gitlab: apt - Is it fine to remove the /opt/gitlab/ directory manually after removing the gitlab package from the system? - Ask Ubuntu
Remove services
sudo gitlab-ctl uninstall
Clean any data generated by usage of the package
sudo gitlab-ctl cleanse
You may also want to remove any accounts you configured. To do so
sudo gitlab-ctl remove-accounts
Then remove the package using
sudo dpkg -P gitlab-ce
Furthermore, gitlab-ce uses these directories (as described here)
/opt/gitlab holds application code for GitLab and its dependencies.
/var/opt/gitlab holds application data and configuration files that gitlab-ctl reconfigure writes to.
/etc/gitlab holds configuration files for omnibus-gitlab. These are the only files that you should ever have to edit manually.
/var/log/gitlab contains all log data generated by components of omnibus-gitlab.
So, First remove the package recommended way (Even if you need to reinstall it). You can then remove those data using rm safely.
Information found from omnibus-gitlab README page