How to make Rack-Attack honor X-Forwarded-For?

We have gitlab-ce running behind a load balancer. When a flood of failed login attempts come in, rack-attack blacklists the IP of the load balancer, so all users and all traffic gets blocked with 403 forbidden.

We are able to temporarily clear the issue by following Remove blocked IPs from Rack Attack via Redis

We have nginx['proxy_set_headers'] and nginx['log_format'] set to support X-Forwarded-For and proxy_add_x_forwarded_for, so nginx logs the real client IP, but I don’t see any way to make gitlab_rails['rack_attack_git_basic_auth'] honor the X-Forwarded-For header.

We’d like to avoid blocking the entire LB and all users. We’d also like to avoid completely disabling rack-attack. If possible to configure rack-attack to honor the X-Forwarded-For, and selectively blacklist client IP’s that flood the server, that’s what we’d like to do. Can it be done?