Without seeing the settings you completed in gitlab.rb, will be difficult to see where the problem is. This is how mine looks, because I don’t use the config in gitlab.rb, but I do it a different way:
### GitLab email server settings
###! Docs: https://docs.gitlab.com/omnibus/settings/smtp.html
###! **Use smtp instead of sendmail/postfix.**
# gitlab_rails['smtp_enable'] = true
# gitlab_rails['smtp_address'] = "smtp.server"
# gitlab_rails['smtp_port'] = 465
# gitlab_rails['smtp_user_name'] = "smtp user"
# gitlab_rails['smtp_password'] = "smtp password"
# gitlab_rails['smtp_domain'] = "example.com"
# gitlab_rails['smtp_authentication'] = "login"
# gitlab_rails['smtp_enable_starttls_auto'] = true
# gitlab_rails['smtp_tls'] = false
but anyway, you would need to ensure the above items (not necessarily all of them, just some), are configured correctly, and that username/password are actually surrounded by quotes (as above in the commented example config).
As I said, I don’t use this, because I have postfix configured on my Linux server, so it utilises this without me having to configure gitlab to access a particular SMTP server. Of course, you can do that if you are OK with configuring postfix and getting it to relay your emails to your mail server. This way has other issues, eg: IP address of your server sending the emails not being authorised to send for your email domain if it’s using SPF records etc, or servers making revDNS/ptr lookups for an IP that has an invalid PTR for mail delivery. But providing that these have been addressed during initial configuration in your DNS records, etc, then later it’s not an issue.
Re-check your settings in gitlab.rb based on the above example config. If it’s not working, then post your config here so we can see what you’ve done - without posting sensitive information like username/passwords.