Hello,
Occasionally, and on various pages (eg PR overview, /admin/runners/$id), we observe an HTTP 50x, which disappears if you reload the page.
Using GItlab Omnibus, 10.0.4-ce.0
Exception from production.log (in this case a failed call to /admin/runners/6:
Started GET "/admin/runners/6" for 83.150.6.105 at 2017-11-01 09:14:03 +0100
Processing by Admin::RunnersController#show as HTML
Parameters: {"id"=>"6"}
Completed 500 Internal Server Error in 13ms (ActiveRecord: 0.6ms)
ActiveRecord::UnknownPrimaryKey (Unknown primary key for table ci_runners in model Ci::Runner.):
app/controllers/admin/runners_controller.rb:52:in `runner'
lib/gitlab/i18n.rb:46:in `with_locale'
lib/gitlab/i18n.rb:52:in `with_user_locale'
app/controllers/application_controller.rb:326:in `set_locale'
lib/gitlab/middleware/multipart.rb:93:in `call'
lib/gitlab/request_profiler/middleware.rb:14:in `call'
lib/gitlab/middleware/go.rb:17:in `call'
lib/gitlab/etag_caching/middleware.rb:11:in `call'
lib/gitlab/request_context.rb:18:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:27:in `call'
Evidently the ORM does known about the primary key of this model:
irb(main):001:0> Ci::Runner.table_name
=> "ci_runners"
irb(main):002:0> Ci::Runner.primary_key
=> "id"
Nota bene, in case it might be related: We did a rollback from 10.1.0 to 10.0.4 two days ago. Rollback was done according to https://docs.gitlab.com/ce/raketasks/backup_restore.html#restore-for-omnibus-installations, ie install old package, stop processes, import backup from before the upgrade.
Output of gitlab:check:
root@gantrisch:/var/log/gitlab/gitlab-rails# gitlab-rake gitlab:check SANITIZE=true
Checking GitLab Shell ...
GitLab Shell version >= 5.9.0 ? ... OK (5.9.0)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:root, or git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
3/1 ... ok
5/2 ... ok
5/3 ... ok
5/4 ... ok
2/5 ... ok
3/6 ... ok
3/7 ... ok
8/8 ... ok
18/9 ... ok
11/10 ... ok
12/11 ... ok
18/12 ... ok
3/13 ... ok
18/14 ... ok
3/15 ... ok
2/16 ... ok
22/18 ... ok
22/19 ... ok
31/21 ... ok
31/22 ... ok
31/24 ... ok
31/25 ... ok
30/26 ... ok
30/29 ... ok
21/30 ... ok
30/34 ... ok
3/36 ... ok
31/37 ... ok
30/39 ... ok
11/40 ... ok
30/41 ... repository is empty
32/42 ... ok
31/43 ... ok
31/44 ... ok
31/45 ... repository is empty
7/46 ... ok
31/47 ... ok
22/49 ... ok
22/51 ... ok
30/52 ... repository is empty
3/53 ... ok
22/54 ... ok
22/55 ... ok
42/56 ... ok
40/58 ... ok
5/60 ... ok
31/61 ... ok
3/62 ... ok
31/63 ... ok
24/65 ... ok
37/68 ... ok
3/69 ... ok
40/70 ... ok
41/71 ... ok
36/73 ... ok
22/74 ... ok
42/76 ... ok
8/78 ... ok
47/81 ... ok
22/82 ... ok
37/84 ... ok
37/85 ... ok
39/86 ... ok
37/87 ... ok
40/88 ... ok
5/90 ... ok
43/91 ... ok
12/92 ... ok
31/93 ... ok
5/94 ... ok
12/95 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Reply by email is disabled in config/gitlab.yml
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
3/1 ... yes
5/2 ... yes
5/3 ... yes
5/4 ... yes
2/5 ... yes
3/6 ... yes
3/7 ... yes
8/8 ... yes
18/9 ... yes
11/10 ... yes
12/11 ... yes
18/12 ... yes
3/13 ... yes
18/14 ... yes
3/15 ... yes
2/16 ... yes
22/18 ... yes
22/19 ... yes
31/21 ... yes
31/22 ... yes
31/24 ... yes
31/25 ... yes
30/26 ... yes
30/29 ... yes
21/30 ... yes
30/34 ... yes
3/36 ... yes
31/37 ... yes
30/39 ... yes
11/40 ... yes
30/41 ... yes
32/42 ... yes
31/43 ... yes
31/44 ... yes
31/45 ... yes
7/46 ... yes
31/47 ... yes
22/49 ... yes
22/51 ... yes
30/52 ... yes
3/53 ... yes
22/54 ... yes
22/55 ... yes
42/56 ... yes
40/58 ... yes
5/60 ... yes
31/61 ... yes
3/62 ... yes
31/63 ... yes
24/65 ... yes
37/68 ... yes
3/69 ... yes
40/70 ... yes
41/71 ... yes
36/73 ... yes
22/74 ... yes
42/76 ... yes
8/78 ... yes
47/81 ... yes
22/82 ... yes
37/84 ... yes
37/85 ... yes
39/86 ... yes
37/87 ... yes
40/88 ... yes
5/90 ... yes
43/91 ... yes
12/92 ... yes
31/93 ... yes
5/94 ... yes
12/95 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.3 ? ... yes (2.3.5)
Git version >= 2.7.3 ? ... yes (2.13.5)
Git user has default SSH configuration? ... yes
Active users: ... 32
Checking GitLab ... Finished
Let me know if you need further information / if I should open a bug ticket instead. 