After upgrading from 16.2.8 to 16.3.6 Geo can no longer connect to primary site

I am pretty sure I am going to need to open a support request with Gitlab, but I upgraded from 16.2.8 to 16.3.6 by following the upgrade path tool, and Gitlab is showing that the secondary server can no longer connect to the primary server using port 80. On the secondary server I am able to access the primary gitlab server through the web browser, and if I create a repository on the secondary server it will replicate to the primary server. Replication between the two servers was healthly before the upgrade.

When I check the geo.log the only error message I receive is:

{"severity":"ERROR","time":"2023-12-21T23:06:06.759Z","correlation_id":"83060affbe5f2fb4a3d78ec11859e122","class":"Geo::NodeStatusRequestService","gitlab_host":"secondary-host","message":"Failed to Net::HTTP::Post to primary url: http://primary-host/api/v4/geo/status","error":"Failed to open TCP connection to (ip of primary):80 (Connection refused - connect(2) for \"primary-host-ip\" port 80)"}


Gitlab::Geo::LogCursor::Lease","message":"Lease canceled due to error: PG::UndefinedColumn: ERROR: column application_settings.database_apdex_settings does not exist\nLINE 1: ...tings\".\"jira_connect_public_key_storage_enabled\", \"applicati...\n

This issue magically resolved itself over time.