Key facts
- An Earth observation satellite autonomously identified targets in orbit for the first time in April.
- The Yam-9 spacecraft, built by Loft Orbital, utilized Google DeepMind's Gemma 3 vision-language model.
- NASA's Jet Propulsion Laboratory developed the NAVI-Orbital software enabling this in-space processing.
- This technology allows for initial data processing on the satellite, significantly reducing data transmission to Earth.
- The advancement enables real-time monitoring and 'always-on' surveillance capabilities from space.
For the first time, an Earth observation satellite has autonomously identified targets in orbit, a milestone achieved in April using Google DeepMind's Gemma 3 vision-language model (VLM) onboard Loft Orbital's Yam-9 spacecraft. This development, enabled by NASA's Jet Propulsion Laboratory's NAVI-Orbital software, signifies a major advancement in space-based sensing capabilities.
Traditionally, satellites download vast amounts of data to human analysts on Earth for processing. However, the onboard VLM allows for initial data triage in space, significantly reducing the volume of raw data and making sensors more useful. Researchers were able to query the model for specific information, such as identifying infrastructure around railway hubs or classifying areas where natural environments meet human development.
Loft Orbital's head of AI, Paul Lasserre, highlighted the potential for 'always-on, patrol layers in space,' enabling continuous monitoring and interaction with satellites for tasks like border surveillance. This capability is crucial for Loft's infrastructure-as-a-service business model, which aims to build constellations for real-time global coverage.
The NAVI-Orbital software was streamlined to run efficiently on limited hardware, a key aspect of edge applications. While Gemma 3 is an off-the-shelf model, the software package required significant optimization for space deployment. This achievement is expected to spur similar developments from other companies, such as Planet Labs, which is already researching VLM applications for its satellites equipped with similar processors.
NASA is also leveraging AI to analyze its vast collection of Earth observation data, aiming to accelerate climate research. By training AI systems on billions of records, the agency seeks to identify meaningful patterns and insights that would typically take years to uncover, enhancing scientific discovery and understanding of planetary changes.