500 error in homepage and project (Gitaly not functioning)

How to Use GitLab Self-managed
1

Hi, I’m new to GitLab.
I’m trying to build a git service for my company, so I uses GitLab-v15.7.5-ee with enviroment Oracle Linux 8.5

When I finished installing and configured SSL certification,I logged in and see 500 error page(and in project page), but I can access adimin console by typing url directly.

Production log shows this error

ActionView::Template::Error (14:failed to connect to all addresses. debug_error_string:{"created":"@1674024319.446082928","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3093,"referenced_errors":[{"created":"@1674024319.446082177","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":163,"grpc_status":14}]}):
    25:           - if project.creator && use_creator_avatar
    26:             = render Pajamas::AvatarComponent.new(project.creator, size: 48, alt: '', class: 'gl-mr-5')
    27:           - else
    28:             = render Pajamas::AvatarComponent.new(project, size: 48, alt: '', class: 'gl-mr-5')
    29:   .project-cell{ class: css_class }
    30:     .project-details.gl-pr-9.gl-sm-pr-0.gl-w-full.gl-display-flex.gl-flex-direction-column{ data: { qa_selector: 'project_content', qa_project_name: project.name } }
    31:       .gl-display-flex.gl-align-items-center.gl-flex-wrap-wrap

And using gitlab-rake gitlab:check SANITIZE=true shows:

  Gitaly: ... default ... FAIL: 14:failed to connect to all addresses. debug_error_string:{"created":"@1674029179.847175955","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3093,"referenced_errors":[{"created":"@1674029179.847175357","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":163,"grpc_status":14}]}

 Checking Gitaly ... Finished

Seems like the problem all points to the gitaly service is not properly functioning.

After some research, I found this

So I checked my ssl and make sure it pass the ssl chain check with /opt/gitlab/embedded/bin/openssl s_client -connect <gitaly-ipaddress>:<port> -verify_return_error

About two days after not solving this problem, I logged in this morning found out it was fixed, and the gitaly server in web ui was up to date.

But after a few hours I changed default repo and backup location and reconfigured, it broked again with the same issue.

I have no idea what is going on, so I will really appreciate it if someone can help me with this…

Thank you!

1 Like