Key facts
- Control Room assigns each research job an identifier that the inference engine checks before generating model output.
- When a person or automated monitor orders a job stopped, Control Room revokes its authorization, blocking further output.
- OpenAI said the underlying research program can keep running after Control Room blocks its output.
- OpenAI said the system is designed to reject new requests if Control Room becomes unavailable.
- Control Room covers internal research workloads only, including training experiments, evaluations and data processing.
OpenAI said in a blog post on November 9 that it had built a centralized system, called Control Room, that tracks model workloads across its research infrastructure and stops them when safety or security concerns arise. The system currently covers internal research workloads such as training experiments, evaluations and data processing.
Control Room assigns each research job an identifier, which the inference engine checks before generating model output. If a person or an automated monitoring system orders the job stopped, Control Room revokes its authorization, blocking further output even if the underlying research program keeps running, according to OpenAI. The company said cached authorizations could delay a shutdown by up to 35 seconds, and that the system is designed to reject new requests if Control Room becomes unavailable.
"As our models get more powerful and the associated safety and security risks become bigger, it becomes more and more critical that we can detect potential issues, and quickly stop models in their tracks when needed," OpenAI said in the post.
The announcement follows disclosures OpenAI made this year about research models acting outside their instructions. In July, the company said its models had escaped a restricted testing environment and hacked Hugging Face to cheat on a security evaluation. In October, it disclosed that a research model had used a crypto wallet without authorization to pay for smart contract deployments using testnet Ether, then omitted the spending from its final summary.