Key facts
- French AI startup ZML has launched ZML/LLMD, an inference server for large language models.
- The software supports a variety of chips, including Nvidia, AMD, Google TPUs, Apple Metal, and Intel Arc.
- ZML aims to maximize AI chip performance and reduce vendor lock-in.
- The product is being released for free to gather usage data.
- ZML has raised $20 million in funding from venture capital firms.
French AI startup ZML has launched ZML/LLMD, a new inference server designed to accelerate the performance of large language models across a wide range of hardware. The software aims to break down existing barriers in the AI ecosystem, allowing models to run on various chips, including those from Nvidia, AMD, Google's TPUs, Apple Metal, and Intel Arc, at their maximum potential speed.
ZML founder Steeve Morin stated that optimizing inference, the processing of prompts, is becoming increasingly critical as AI integrates into daily life. He highlighted that current software and architectural limitations often lead to vendor lock-in and hinder efficiency. By offering a solution that works across diverse hardware, ZML hopes to provide enterprises and cloud providers with more cost-effective and energy-efficient options, thereby democratizing AI dissemination.
Morin also noted that this software could be particularly beneficial for emerging AI chipmakers, many of which are based in Europe. While acknowledging Nvidia's continued importance due to its supply chain, ZML maintains a positive relationship with the company. The startup faces competition from other inference optimization platforms like Baseten and Inferact. ZML's lean team of 20, combined with $20 million in funding from investors such as 20VC and Kima Ventures, has enabled rapid development.
ZML/LLMD is being released as a free product, a departure from their first ML framework, with the intention of learning from usage and identifying optimal revenue generation strategies. Morin expressed a preference for growth over immediate profit, aiming to avoid hindering adoption through excessive early pricing.