OpendIDConnect issues, invalid 'state' parameter and parse failures

I have the setup for OpenIDConnect mostly working, but when it gets redirected back to gitlab after logging in it says:

Could not authenticate you from OpenIDConnect because "Invalid 'state' parameter".

I have checked and state parameter in the call to my oauth provider matches the state parameter in the URL fragment in both the callback and the sign in page.

As far as I understand these just need to match, and they do, yet the error above is given instead of logging in. Is there some place that gitlab is supposed to be storing this and isn’t?

EDIT: So when I was setting this up I was getting errors about the client not supporting the implicit grant type, so I enabled it, but I know that is not really recommended anymore. Why is gitlab trying to use that grant type when I set up the omniauth stuff?

EDIT2: If I set response_type: "code" in the config, it fails at the callback saying Could not authenticate you from OpenIDConnect because "Unexpected token at 'eyjhbgcioij...' and it shows the jwt encoded data from my oidc server