Please help fill in this template with all the details to help others help you more efficiently. Use formatting blocks for code, config, logs and ensure to remove sensitive data.
Problem to solve
I upgrade gitlab from 16.8.0 to 16.9.1, when I execute the command āgitlab-ctl reconfigureā
ļ¼cinc-stacktrace.out is shown below:
Generated at 2024-02-27 15:56:59 +0800
Mixlib::ShellOut::ShellCommandFailed: rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ā1ā
---- Begin output of ābashā ----
STDOUT: rake aborted!
Feature::InvalidFeatureFlagError: Feature flag āci_job_artifacts_backlog_large_loop_limitā is already defined in ā/opt/gitlab/embedded/service/gitlab-rails/config/feature_flags/development/ci_job_artifacts_backlog_large_loop_limit.ymlā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:194:in block in load_all_from_path!' /opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:190:in eachā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:190:in load_all_from_path!' /opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:175:in block in load_all!ā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:174:in each' /opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:174:in each_with_objectā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:174:in load_all!' /opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:119:in reload!ā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature.rb:254:in register_definitions' /opt/gitlab/embedded/service/gitlab-rails/config/initializers/0_inject_feature_flags.rb:7:in <top (required)>ā
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:7:in <top (required)>' <internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:37:in requireā
internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb:37:in require' /opt/gitlab/embedded/bin/bundle:25:in loadā
/opt/gitlab/embedded/bin/bundle:25:in <main>' Tasks: TOP => gitlab:db:configure => environment (See full trace by running task with --trace) STDERR: ---- End output of "bash" ---- Ran "bash" returned 1 /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout.rb:300:in invalid!ā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout.rb:287:in error!' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout/helper.rb:130:in shell_out_compacted!ā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout/helper.rb:54:in shell_out!' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider/execute.rb:52:in block (2 levels) in class:Executeā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/mixin/why_run.rb:51:in add_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:293:in converge_byā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider/execute.rb:50:in block in <class:Execute>' (eval):2:in block in action_runā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in instance_eval' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in compile_and_converge_actionā
(eval):2:in action_run' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:245:in run_actionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:601:in block in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:628:in with_umaskā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:600:in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:74:in run_actionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in block in run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in eachā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection.rb:64:in insertā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/dsl/declare_resource.rb:267:in declare_resource' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/dsl/resources.rb:36:in bash_hide_envā
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb:20:in block in class_from_file' (eval):2:in block in action_runā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in instance_eval' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:304:in compile_and_converge_actionā
(eval):2:in action_run' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:245:in run_actionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:601:in block in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:628:in with_umaskā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:600:in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:74:in run_actionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in block in run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in eachā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:in run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:132:in block in convergeā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/resource_list.rb:96:in block in execute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/stepable_iterator.rb:114:in call_iterator_blockā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/stepable_iterator.rb:85:in step' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/stepable_iterator.rb:103:in iterateā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/stepable_iterator.rb:54:in each_with_index' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/resource_list.rb:94:in execute_each_resourceā
/opt/gitlab/embedded/lib/ruby/3.1.0/forwardable.rb:238:in execute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:130:in convergeā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:875:in block in converge' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:870:in catchā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:870:in converge' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:894:in converge_and_saveā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:298:in run' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application.rb:305:in run_with_graceful_exit_optionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application.rb:281:in block in run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/local_mode.rb:42:in with_server_connectivityā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application.rb:264:in run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application/base.rb:354:in run_applicationā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application.rb:67:in run' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-bin-18.3.0/bin/cinc-client:25:in <top (required)>ā
/opt/gitlab/embedded/bin/cinc-client:25:in load' /opt/gitlab/embedded/bin/cinc-client:25:in ā
Caused by Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ā1ā
---- Begin output of ābashā ----
STDOUT: rake aborted!
Feature::InvalidFeatureFlagError: Feature flag āci_job_artifacts_backlog_large_loop_limitā is already defined in ā/opt/gitlab/embedded/service/gitlab-rails/config/feature_flags/development/ci_job_artifacts_backlog_large_loop_limit.ymlā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:194:inblock in load_all_from_path!' /opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:190:ineachā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:190:inload_all_from_path!' /opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:175:inblock in load_all!ā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:174:ineach' /opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:174:ineach_with_objectā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:174:inload_all!' /opt/gitlab/embedded/service/gitlab-rails/lib/feature/definition.rb:119:inreload!ā
/opt/gitlab/embedded/service/gitlab-rails/lib/feature.rb:254:inregister_definitions' /opt/gitlab/embedded/service/gitlab-rails/config/initializers/0_inject_feature_flags.rb:7:in<top (required)>ā
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:7:in<top (required)>' <internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:37:inrequireā
internal:/opt/gitlab/embedded/lib/ruby/site_ruby/3.1.0/rubygems/core_ext/kernel_require.rb:37:inrequire' /opt/gitlab/embedded/bin/bundle:25:inloadā
/opt/gitlab/embedded/bin/bundle:25:in<main>' Tasks: TOP => gitlab:db:configure => environment (See full trace by running task with --trace) STDERR: ---- End output of "bash" ---- Ran "bash" returned 1 /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout.rb:300:ininvalid!ā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout.rb:287:inerror!' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout/helper.rb:130:inshell_out_compacted!ā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/mixlib-shellout-3.2.7/lib/mixlib/shellout/helper.rb:54:inshell_out!' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider/execute.rb:52:inblock (2 levels) in class:Executeā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/mixin/why_run.rb:51:inadd_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:293:inconverge_byā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider/execute.rb:50:inblock in <class:Execute>' (eval):2:inblock in action_runā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:304:ininstance_eval' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:304:incompile_and_converge_actionā
(eval):2:inaction_run' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:245:inrun_actionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:601:inblock in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:628:inwith_umaskā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:600:inrun_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:74:inrun_actionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:inblock in run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:ineachā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:inrun_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection.rb:64:ininsertā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/dsl/declare_resource.rb:267:indeclare_resource' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/dsl/resources.rb:36:inbash_hide_envā
/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb:20:inblock in class_from_file' (eval):2:inblock in action_runā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:304:ininstance_eval' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:304:incompile_and_converge_actionā
(eval):2:inaction_run' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/provider.rb:245:inrun_actionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:601:inblock in run_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:628:inwith_umaskā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource.rb:600:inrun_action' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:74:inrun_actionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:inblock in run_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:ineachā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:108:inrun_all_actions' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:132:inblock in convergeā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/resource_list.rb:96:inblock in execute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/stepable_iterator.rb:114:incall_iterator_blockā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/stepable_iterator.rb:85:instep' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/stepable_iterator.rb:103:initerateā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/stepable_iterator.rb:54:ineach_with_index' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/resource_collection/resource_list.rb:94:inexecute_each_resourceā
/opt/gitlab/embedded/lib/ruby/3.1.0/forwardable.rb:238:inexecute_each_resource' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/runner.rb:130:inconvergeā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:875:inblock in converge' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:870:incatchā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:870:inconverge' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:894:inconverge_and_saveā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/client.rb:298:inrun' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application.rb:305:inrun_with_graceful_exit_optionā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application.rb:281:inblock in run_chef_client' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/local_mode.rb:42:inwith_server_connectivityā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application.rb:264:inrun_chef_client' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application/base.rb:354:inrun_applicationā
/opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-18.3.0/lib/chef/application.rb:67:inrun' /opt/gitlab/embedded/lib/ruby/gems/3.1.0/gems/chef-bin-18.3.0/bin/cinc-client:25:in<top (required)>ā
/opt/gitlab/embedded/bin/cinc-client:25:in `loadā
content in ci_job_artifacts_backlog_large_loop_limit.yml
the errors show the problem in ci_job_artifacts_backlog_large_loop_limit.yml, but I found nothing wrong, the content as follow:
name: ci_job_artifacts_backlog_large_loop_limit
introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/76509
rollout_issue_url: [Feature flag] Roll out ci_artifact_fast_removal_large_loop_limit (#347151) Ā· Issues Ā· GitLab.org / GitLab Ā· GitLab
milestone: ā14.10ā
type: development
group: group::pipeline security
default_enabled: false