Hello everyone,
The last updates before went absolutely smoothly.
I have now carried out a test run of the update from Gitlab 15.11 to version 16.0.1 and here I have an endless CPU load that ebbs after a short time for a few seconds and then goes back to 100% for all cores.
After about 16 minutes there is a short break in the load, and then it goes up to 100% again and again.
Each time it is the process
/opt/gitlab/embedded/bin/sidekiq -c20 -eproduction -t25 -gqueues:authorised_project_update:authorised_project_update_project_recalculate,authorised_project_update:authorised_project_update_project_recalculate_per_user,authorised_project_update: authorized_project_update_user_refresh_from_replica,authorized_project_update:authorized_project_update_user_refresh_over_user_range,authorized_project_update:authorized_project_update_user_refresh_with_low_urgency,auto_devops:auto_devops_disable,auto_merge: auto_merge_process,batched_background_migrations:database_batched_background_migration_ci_execution,batched_background_migrations:database_batched_background_migration_main_execution,chaos:chaos_cpu_spin,chaos:chaos_db_spin,chaos:chaos_kill,chaos:chaos_leak_mem,chaos: chaos_sleep,cluster_agent:clusters_agents_delete_expired_events,cluster_agent:clusters_agents_notify_git_push,container_repository:cleanup_container_repository,container_repository:container_expiration_policies_cleanup_container_repository,container_rep
which occurs several times.
I know this behaviour in a simple form from previous versions.
Virtual machine:
By default, our Gitlab ran in a virtualised environment on Debian 11 Bullseye with 4 cores and 12 GB RAM, as I only use it for my configurations and ansible. It ran comfortably here for the last 2 years and felt perfectly at home.
I used packages.gitlab.com as Repository for Debian.
Only changes in gitlab.rb are:
- External URL
- SMTP-Server configuration
- nginx enabled http2, gzip and added ssl-certificates
For the update, I expanded the system to 12 cores and 24GB RAM.
Here, the entire RAM and all cores are used again and again.
The RAM is then emptied briefly, only to fill up continuously afterwards and then fill the SWAP.
Tried so far:
- Reboot
- gitlab-ctl reconfigure with / or without reboot
Is there any idea on the subject here, or has anyone noticed this behaviour?
Thank you in advance and greetings
Christian
And of course something for the eyes, a screenshot