Key facts
- Vals is a startup founded in 2024 focused on AI benchmarking.
- The company raised $40 million in a Series A funding round led by Andreessen Horowitz.
- Vals differentiates itself by not publicly disclosing its test materials and evaluating models on industry-specific tasks.
- The startup's revenue has grown eightfold in the past year.
- Vals has expanded its team from eight to 25 employees this year.
- The company has launched a program to provide model evaluations to federal agencies.
Vals, a startup established in 2024, has secured $40 million in a Series A funding round led by Andreessen Horowitz, with previous participation from 8VC and Bloomberg Beta. The company aims to revolutionize AI benchmarking by moving beyond traditional, often outdated, academic tests. Vals differentiates itself by not publicly disclosing its evaluation materials and instead focusing on assessing AI models' capabilities in complex, industry-specific tasks such as law, finance, and coding. This approach seeks to measure the real-world impact and quality of AI outputs compared to human performance, and also to identify potential negative implications.
Rayan Krishnan, Vals' co-founder, observed that existing benchmarks were not keeping pace with the rapid advancements in AI models. He explained that Vals' methodology evaluates models on their ability to perform tasks relevant to specific domains, rather than just general knowledge. The startup is expanding its evaluation areas to include recursive self-improvement, mental health, cybersecurity, biosecurity, and the law of armed conflict.
Companies pay Vals for these evaluations, which help them troubleshoot and improve their models. Krishnan likens the revenue model to students paying for SAT tests. The evaluations are increasingly becoming critical factors for companies acquiring AI models. Vals has experienced significant growth, with revenue eight times higher than last year and its team expanding from eight to 25 employees. The company plans to move to a larger office and hire an additional 10 to 15 people. Vals has also initiated a program to offer its model evaluations to federal agencies.
Krishnan believes Vals' benchmarking system will be crucial for AI companies as they go public and integrate more deeply into the economy. He anticipates that these evaluations will influence public filings and investment decisions related to AI.