Key facts
- Base Labs, Hugging Face, and Goodfire AI are partnering to create safety evaluation and monitoring infrastructure for open-weight AI models.
- The partnership aims to establish a transparent standard for AI safety built into model training and deployment.
- Hugging Face hosts over 6,000 models that have had their safeguards removed, a technique known as abliteration.
- Baseten, the parent company of Base Labs, raised $1.5 billion in June, reaching a $13 billion valuation.
- Goodfire AI, which specializes in model interpretability, raised $150 million in Series B funding.
Base Labs, the research division of AI inference provider Baseten, has announced a new partnership with Hugging Face and Goodfire AI to establish safety standards for open-weight AI models. The initiative, launched on Wednesday, aims to develop and publish methods for training and monitoring these models, addressing growing concerns about their potential misuse through techniques like abliteration, where safeguards are removed.
Hugging Face, a major platform for open-source AI models, currently lists over 6,000 models that have undergone abliteration. Base Labs intends to create a transparent framework that is integrated into the model development process rather than being an afterthought. The company stated on X that "openness to be an advantage for AI safety," providing greater visibility and means for implementing actionable controls.
While the technical details of the collaboration are undisclosed, Goodfire AI, known for its work in explaining AI model decision-making, is expected to play a key role in building safety features directly into the models. Baseten, which recently secured $1.5 billion in Series F funding to reach a $13 billion valuation, and Goodfire AI, which raised $150 million in Series B funding, are both well-capitalized. Baseten is also soliciting contributions from the broader developer community to build an ecosystem of safe and accessible open models.