Key facts
- PrismML's Bonsai 27B is a 27-billion-parameter AI model compressed to 3.9 GB.
- The model can run on an iPhone 17 Pro Max at 11 tokens per second.
- The ternary variant retains 94.6% of full-precision benchmark performance.
- Apple is in early talks with PrismML about the compression technology.
- The compression method reduces model weights to a single sign or three values.
- Bonsai 27B is available for free download under Apache 2.0.
PrismML has developed Bonsai 27B, a 27-billion-parameter artificial intelligence model that has been compressed to a size of 3.9 GB, making it small enough to run on consumer smartphones. This breakthrough allows the model to operate on devices like the iPhone 17 Pro Max at a speed of 11 tokens per second, a significant achievement for models of this capability tier. The ternary variant of Bonsai 27B maintains approximately 94.6% of its full-precision benchmark performance, outperforming other compressed models that are larger and struggle with mathematical and coding tasks.
The compression technique, based on Caltech intellectual property, reduces each model weight from 16 bits to a single sign (+1 or -1) or three values (including zero) in the ternary build. This method compresses the model end-to-end, without relying on higher-precision escape hatches for sensitive layers, which is a common practice in other quantized models. This approach allows for a 262K-token context window to be practical on-device.
In performance benchmarks across NVIDIA H100 GPUs, Ternary Bonsai 27B averaged 80.49, representing 94.6% of the full-precision model's capability. While single-shot prompts may not match cloud-based frontier models, the ability to run locally and for free is a key advantage. The model has demonstrated competence in tasks such as coding, producing functional game mechanics, and creative writing, with consistent internal logic and pacing comparable to models like Claude Haiku or Sonnet.
Apple has reportedly entered into early discussions with PrismML regarding its compression technology for potential on-device integration. PrismML's CEO, Babak Hassibi, confirmed these talks, indicating that the company plans to develop compressed versions of other models, including Gemma, and eventually larger frontier models.