GitLab Support is no longer processing MFA resets for free users.
In GitLab Everyone can contribute for that reason, we’re opening this post for feedback about our recent change about removal support intervention for free users.
Please consider that users with a free plan will maintain the options to self recover their account acces by following this process.
I feel slightly nervous about this change.
I definitely agree that MFA reset should be a very rare thing and is hard to authenticate, and generally I feel happy that gitlab is making this move - but is there some kind of allowance for truly exceptional cases? e.g. if an account is taken over by a malicious actor who changes MFA settings?
@joseph.heenan thank you for your comment.
Truly exceptional cases will also bring truly unique considerations, for example, in the case you mention we’re talking about two possible scenarios:
In both cases, we will have the account owner and the attacker both being able to answer the security challenges for account ownership.
We’re open to review cases like that if those were to occur and a final decision will be aligned with the best security practices.
Thanks for the quick response @relfaro - it’s good to know that you’re open to review exceptional cases, and hence I’m happy with the overall change.
In general I don’t have a problem with this but I feel like an SMS backup to my 2FA app should be a must have for this. It still is in most countries secure enough and allows me in case of a phone reset to still access gitlab.
If you’ve uploaded ssh keys to gitlab then that can be used as a recovery mechanism, that seems like a reasonable alternative to SMS to me?
See https://about.gitlab.com/handbook/support/workflows/account_verification.html#generating-new-recovery-codes-via-ssh
I would actually argue that this is a less secure method (and it would be good to be able to disable that). The whole idea for 2FA is that you if someone steals your credentials they can not access your account. If someone is able to install a keylogger (or similar) for example to get my password its easy for them to steal my SSH key as well which makes 2FA on gitlab useless for any form of attack except maybe a phishing website or shared credentials with different independent websites. Or you leave your computer unlogged/loos it in a coffee shop. Versus your phone is usual a lot more secure and a lot easier to keep track off.
I also think this is less secure method like @fkrauthan said. I also think that the chance is maybe higher to lose access to your account by yourself (lose phone, …) instead of someone stealing it.
Free users will now think twice before using MFA and this is not good at all. I totally understand that security is hard but there must be better option for that problem.
This makes me very happy. If only everyone would do this.
What about charging a service fee for all the hassle and authentication etc. that it would take to resolve such a situation? Just an idea to consider.
Could we get a 48 hour pause for setting up 2FA on accounts more than 48 hours old (to prevent malicious actor from turning on 2FA)?
A free user losing access tokens seems like a time for paid support. There should be a clear process for establishing identity, securely, which involves paying money. For example:
)I won’t comment on appropriate level of security and how many of these you want or need, but “your account is gone forever, no recourse, goodbye” is not the best option.
Be transparent about the fee, and do it on a cost-plus basis. Expensive is okay, so long as the cost is clearly similar to your cost to administer the service.
I applaud this, even as a free user. Gitlab already offers me the ability to use multiple authenticators - and I do, having backup OTPs, a u2f device, and a TOTP token. I feel much more confident in keeping one of the three around than I do in the ability of support agents not to be social engineered; Not to knock your support team, but the twitter hack is clear proof that support overrides are abusable.
The SSH recovery codes tool is great. +1 to that. I would suggest that locking out recently-added keys would be a good addition to that tool, however.
This is a terrible idea which encourages people to use weak security: what you’re doing is telling people that if they enable MFA, GitLab will ensure that they suffer irrecoverable damages — but if they don’t enable MFA, everything is recoverable. That is the opposite of what we want from a security perspective and it risks causing users to be less secure everywhere else because having seen that message will make them question whether they’ll regret enabling MFA even on sites which have good security policies.
GitLab should respect the trust that users place in it and account account for the kind of life events which happen to people every day: houses get flooded or burned, safes are burgled, people have to leave abusive situations in a hurry and may be escaping household members who actively try to sabotage them, people die without having made perfect handover plans (especially relevant this year), etc. Of particular interest, you should consider why the browser manufacturers reversed course on HTTPS key pinning after seeing attackers successfully use it to prevent legitimate users from regaining control: if implemented, this would make it extremely risky to use GitLab because everyone would be one compromise away from an attacker having permanent control of your account — and that would make GitLab look extremely bad for having put themselves in the position of supporting the attacker unless you pay GitLab money (I am certain this is not the intention but it is definitely how it would look to say that FOSS developers are second-class).
There are a number of fallback techniques which can make most of those situations recoverable, and it’s okay if they’re slow or inconvenient — nobody complains that their airbags are messy. Some ideas:
Have a process where someone can use a trusted third-party to verify identity: for example, register identifiers (drivers license, passport, etc.) and have a form which can be notarized after checking ID and mailed in. It’s slow but that works even in the “I lost everything I wasn’t wearing” scenario — and if you did just lose your house to a natural disaster or flee an abusive ex, you probably aren’t going to mind that delay compared to everything else you have to do.
Configure next-of-kin / trusted friends who can approve a reset request, perhaps requiring more than one.
Allow the user to configure some number of IdPs to approve an unlock, increasing the level of compromise that an attacker would need to hit before getting the ability to perform a reset. There are drawbacks to this approach but most people do not have threat models which are meaningfully resistant against someone who can compromise multiple of {Apple, Google, Facebook, Microsoft, login.gov, etc.} and could be pretty effective combined with a time-delay (e.g. send notifications for n days before actually resetting MFA.
Charge for a reset, perhaps requiring additional approval or delay if the card information doesn’t match an address on file. This would make large-scale attacks harder while still giving most people the ability to regain access quickly.
I’m an open source developer who develops and maintains a small project with a few thousand users in my spare time. I currently host it on GitHub, but I use GitLab CI for parts of it.
If I were to use GitLab for this, I would now have the following two risks:
Risks:
The only truly effective way to prevent Risk 1 is to enable MFA, which introduces Risk 2 unless I:
I can’t see any ways to mitigate the impacts of Risk 2:
Even though the risk of any of this is happening is tiny, for my hobby, open source development projects, I feel a responsibility to protect my users. Perhaps especially because I can’t have the security processes that businesses do.
This change would force me to choose between unacceptable risk to my users, or severe impact on my hobby/life balance and mental health due to the extreme personal responsibility I would have to take to mitigate it.
If GitHub were to introduce this change, I would likely retire from my open source projects.
Please consider an exemption for non-commercial open source projects over a certain size. I’d be happy to explicitly register ID beforehand, or have to have a certain number of users, or stars, or any kind of threshold. But there needs to be a safety net otherwise you’re forcing those who have the least time to be secure to be less secure.
Edit: Reading more carefully, the SSH method might be a way out? But I’m not primarily a Linux user, and my usage of SSH keys is limited to GUIs, so I can’t assess how likely I am to be able to recover that way. I’ve also never heard of that as a way of recovering MFA before.
Edit 2: To clarify on “extreme personal responsibility”: This level of personal responsibility doesn’t exist elsewhere. For work MFA, there are simple, safe, recovery processes. For personal, private, MFA, businesses allow me (sometime time consuming) simple recovery processes. It’s only here, when trying to do good, and where I can do the most harm to others, that I have to take the risk on myself.
Edit 3: Final edit. If there was a security vulnerability in my project, and I’m using GitLab as a package repository (either directly or indirectly by users cloning/building), and I lose access to my account: How do I patch?
Thanks for the detailed note @acdha. You might be right that there will be a slight disincentive for users to enable MFA. From my perspective though in a world where there was a way around MFA we were providing a false sense of security. Someone sufficiently knowledgeable about the account could feasibly get around a security measure you put in place to lock such people out.
The idea behind MFA is:
By providing account recovery for free users we were changing the model to:
I feel confident that there are enough non-human recovery methods available to our users that most folks will still feel comfortable enabling MFA.
As a reminder, you have access to:
All of those have a relatively low barrier to perform and don’t rely on human judgement.
I appreciate the other ideas, and I’d encourage you to open feature requests on https://gitlab.com/gitlab-org/gitlab as many of them would require product-level support to implement.
For many of these though they require the same forethought that our current backup methods have: registering 3rd parties, adding additional identity data or providing payment information well ahead of time. It’s possible these features may be attractive to some folks, but there’s a set of people who would also decry collecting unnecessary personally identifiable information. Security and privacy, unfortunately, can sometimes be at odds.
I look forward to collaborating with you on any feature requests you do create, feel free to tag me (@lyle) and I’ll get the correct labels added and put it in front of the right product managers.
@tboby -
Thanks for taking the time to write up your thoughts. I can appreciate the model you’ve presented, but I wanted to focus on one part:
If GitHub were to introduce this change, I would likely retire from my open source projects.
What account recovery feature does GitHub have that gives you peace of mind?
From my perspective we provide several ways to get back into your account if you were to lose your primary MFA token:
There’s also the option of becoming a paid user.
In your mind, is there a feature that would provide high security for those individuals who require it as well as alleviate your concerns?
Another point that needs addressing is: will we reset MFA for groups who have applied for GitLab’s Open Source program: Open Source projects | GitLab
I appreciate you raising this as I don’t have a direct answer. Since there’s a partnership involved I suspect we probably would provide some sort of account recovery for at least the owner of such a project. I’ve created an issue to firm up our stance here: https://gitlab.com/gitlab-com/marketing/community-relations/opensource-program/general/-/issues/105
With regard to the SMS backup suggestion - your auth is only as secure as your weakest link. As lkozloff said, with the recovery in place, the weakest link was the recovery process. So your auth was essentially something you know + something you know.
SMS is something you have, but it’s not your phone - it’s a device that receives messages for your phone number. Getting a SIM for someone else’s account, or getting the number transferred, is easy enough via social engineering on the phone with support. If your threat model involves the resources required to get your SSH key from your system (@fkrauthan) , you’ve assumed your attacker has more than the necessary resources to get a device with your phone number on it.
I’d recommend not having SMS backup, particularly if you have a high-stakes project.
But thats why I think it would be important to offer multiple MFA solutions. E.g.
and then give the user the power to decide which of the solutions each user is comfortable with. If I am ok taking the risk of using Email or SMS then it should be my choice and not Gitlab’s of what they always turn on for me because they think that are the best/most secure solutions.
Thanks @lkozloff — the main concern I have is that all of the alternatives are likely to have correlated failure modes (e.g. having TOTP, two tokens & printed backups still fail at the same time if your house gets robbed, your abusive partner grabs them, etc.), and it might be useful to have a very prominent on-boarding path which covers the risks and suggested mitigations. I mention the abusive partner case a bit, too, because that highlights the value of having a system where the user can recover from the case of being let down by someone they trusted — otherwise a FOSS developer is one Eron Gjoni away from losing their projects permanently.