SBU Researchers Use AI to Reconstruct Physically Accurate Models of Mars

Artificial intelligence may soon help scientists see Mars more clearly than ever before. At Stony Brook University, researchers have developed a system that can generate realistic, three-dimensional videos of the Martian surface - a tool that could reshape how space agencies simulate exploration and prepare for future missions.

Chenyu You, assistant professor in the Department of Applied Mathematics and Statistics and Department of Computer Science worked on the project, Martian World Models, tackling a problem that has long limited planetary research. Most AI models are trained on imagery from Earth, which limits their ability to interpret the distinct lighting, textures, and geometry of Mars. "Mars data is messy," said You. "The lighting is harsh, textures repeat, and rover images are often noisy. We had to clean and align every frame to make sure the geometry was accurate."

To overcome these limitations, the team built a specialized data engine, called M3arsSynth, that reconstructs physically accurate 3D models of Martian terrain using NASA rover images. The system processes pairs of photographs taken by the rovers to calculate precise depth and scale, building detailed digital landscapes that reflect the planet's actual structure. These reconstructions then serve as the foundation for MarsGen, an AI model trained to generate new, controllable videos of Mars from single frames, text prompts, or camera paths.

The result is a system that produces smooth, consistent video sequences that capture not only the appearance of Martian landscapes but also their depth and physical realism. "We're not just making something that looks like Mars," You said. "We're recreating a living Martian world on Earth - an environment that thinks, breathes, and behaves like the real thing."

Read the full story by Ankita Nagpal on the AI Innovation Institute website.