Disabling LDAP auth - HOWTO handle existing users with ldapmain identity?

I am switching a local GitLab installation (CE) from using LDAP auth to using SAML.

SAML is set up and works, but now I want to disable LDAP auth.
This is what I’ve done:
gitlab_rails[‘ldap_enabled’] = false
and running
gitlabctl-reconfigure
disables the ldap login, but the LDAP tab is still visible on the login page!
running
gitlabctl-restart
removes the LDAP tab.
I can set a local password and the user can login, but navigating to Settings “/profile” gives a error status 502
If an admin tries to open the users Identities it also fails with error code 500
Using the http api to list users shows the Identities still have the ldapmain provider.

So I’m thinking, that I should remove all the ldapmain identities before turning off ldap auth, but I don’t see a way to do it via the http API!
I can add identities with the users/:id modify and a suitable payload.

Do I really have to delete alle ldap identities manually via web interface before switching of ldap auth?

Please tell me there is a better way.

Regards, Torsten