Trust and Safety on BlueSky
Opened this issue · 4 comments
After the incidents of racism on Bluesky on July 12/13, I collected some thoughts on how to improve T&S on the platform. My hope is that others on BlueSky will contribute to this.
If you've opened issues in BSky repos, please link for visibility. Systemic problems often require systemic solutions, and T&S initiatives require thought and communication to be effective.
Non-Engineering Ideas
- Join Trust and Safety Professionals Association (https://www.tspa.org). They have many great professionals (Engineers, PMs, policy experts, etc.) who deal with T&S issues and content moderation. They have great resources available and there's an activity community.
- Start on an Acceptable Use Policy. If you're already started on it, let the community know.
Engineering Ideas
Account security helps prevent malevolent users from taking over accounts. I'm sure 2FA is on your roadmap, but in addition you may need:
- Blocking of emails with autogenerated account info (typically using a
+in Gmail) to make multiple accounts OR programmatic account abuse. This will likely need some limit/fine-tuning as people may have main and alt accounts. - Automatic password reset after 8 failed login attempts.
- Email notification when geoIP region changes in short periods of time.
Additionally, please publish AARs/Post-mortems on T&S incidents. These are site reliability incidents and can do just as much, if not more, damage to reputation than a server outages. We are all engineers and we learn from each others mistakes to grow.
From a security/privacy perspective, some of the engineering ideas are frankly terrible.
Blocking of emails with autogenerated account info (typically using a + in Gmail)
This is a terrible, user-hostile idea. People use plus addressing or "masked" emails for a very good reason - protecting their privacy from platforms/providers, or mitigating inevitable data leaks.
Automatic password reset after 8 failed login attempts.
I'm not sure what this is supposed to achieve. It does make for a great way to silence a user though - just hammer the login form with their username and nonsense creds to keep their account locked, very good method of drive-by harassment.
This idea (locking accounts) barely works in corporate environments.
Email notification when geoIP region changes in short periods of time.
This is actually a good idea. Even better would be a page that lets you view authenticated sessions, where they logged in from, last login, time, etc, with the ability to force logout a device.
This is a terrible, user-hostile idea. People use plus addressing or "masked" emails for a very good reason - protecting their privacy from platforms/providers, or mitigating inevitable data leaks.
I'm specifically talking about programmatic account creation abuse used by spammers. They use a script following the email+(random hash)@gmail.com to create thousands of accounts. This is not a single user using a single masked email. It may not be an issue yet on BSky, but get this in place before invite-only goes away.
I'm not sure what this is supposed to achieve. It does make for a great way to silence a user though - just hammer the login form with their username and nonsense creds to keep their account locked, very good method of drive-by harassment.
That prevents a malicious user from hijacking an account via brute force. It's standard security in modern web apps and should be implemented until 2FA (ideally not SMS 2FA) or passkeys are.
Email notification when geoIP region changes in short periods of time.
This is actually a good idea. Even better would be a page that lets you view authenticated sessions, where they logged in from, last login, time, etc, with the ability to force logout a device.
That would be great.
That prevents a malicious user from hijacking an account via brute force. It's standard security in modern web apps and should be implemented until 2FA (ideally not SMS 2FA) or passkeys are.
I mean, it prevents a targeted brute force, but creates an extremely powerful denial of service vector.
Most modern services that are well designed don't do this, its one of the worse ideas that comes from corporate infosec. Kind of like "changing your password every X days", or complexity rules when length is what matters.
It also won't prevent credential stuffing, which is a far more common threat than targeted bruteforce.
I can't really understate how trivial it would be to abuse this mechanism to cause mass account lockouts.
I strongly agree though, that 2FA should be implemented ASAP.
I mean, it prevents a targeted brute force, but creates an extremely powerful denial of service vector.
Most modern services that are well designed don't do this, its one of the worse ideas that comes from corporate infosec. Kind of like "changing your password every X days", or complexity rules when length is what matters.
It also won't prevent credential stuffing, which is a far more common threat than targeted bruteforce.
I can't really understate how trivial it would be to abuse this mechanism to cause mass account lockouts.
You've clearly got experience in the space so I'm excited to hear your thoughts on how to address this issue. In previous implementations, we've rate limited IPs as part of that flow to prevent DDoS on those endpoints but I will happily defer to your expertise.