Roblox is open-sourcing three of the AI safety models it already runs on its own platform, releasing them through the Robust Open Online Safety Tools Model Community, or ROOST. GamesIndustry.biz reports that the company is contributing its PII Classifier, Roblox Sentinel and its latest voice safety classifier.
ROOST is an independent nonprofit that builds and maintains open-source infrastructure for online safety. Roblox co-founded it in 2025 alongside Google, OpenAI and Discord.
What the three models do
- PII Classifier — flags attempts to share or request personally identifiable information, including misspellings, coded language and disguised references to other platforms.
- Roblox Sentinel — detects early signs of potential child endangerment.
- Voice safety classifier — moderates voice chat in real time, and incorporates feedback from the ROOST community.
Roblox is also releasing evaluation data alongside the models. One dataset accompanies the PII Classifier and consists, in the company's words, of "synthetic samples of English-language multiuser chats modelled on techniques often used to evade PII filters". A second, newer evaluation dataset is intended to let other companies measure how well their own safety tools perform; its latest version covers 30 languages and eight violation categories.
Sharing these models gives other platforms a robust starting point to train and tune their own moderation tools. In turn, we hope to benefit from the learnings and feedback of other companies as they share their work.
Koneru framed the contribution as an attempt to raise the floor across the industry rather than a competitive move, saying that safety is a shared responsibility no company can solve alone, and that Roblox intends to keep sharing with and learning from the ROOST community.
Released under scrutiny
The release lands days after the US Senate Judiciary Subcommittee on Crime and Counterterrorism opened an investigation into Roblox, saying the platform "appears to prioritise revenue and engagement metrics" over child safety. GamesIndustry.biz reports the company has been told to hand over records covering child safety, sexual or physical abuse, bullying and financial exploitation by 31 August 2026.
Roblox has introduced a series of safety measures in recent years, among them restrictions on direct messaging for users under 13, global facial recognition checks, a "sensitive issues" content tag for experiences, and tighter moderation policies.
Join the conversation — sign in to comment.
Sign in