A field guide for the Wild West of AI-assisted science

Researchers at UC Santa Barbara's National Center for Ecological Analysis and Synthesis (NCEAS) are blazing a trail through the Wild West of AI-assisted science, with a field guide aimed at environmental data scientists.

"It started as something we needed for ourselves," said ecologist and data scientist Rachel King. "We kept repeating the same conversation, project after project: whether to trust a suggestion, how much access to give an AI agent, what to do when a long chat session forgets a decision it made an hour earlier. At some point it made more sense to work it out together, once, than have every team rediscover it independently."

The researchers' work is published in PLOS Computational Biology.

From internal guidance to community standards

When a six-person team set out to build the Wildfire Resilience Index, an open-access tool for measuring how prepared communities and landscapes are for wildfire, they knew the science would be hard: stitching together satellite imagery, land-cover data and socioeconomic variables with R and Python pipelines across two countries and 13 unique jurisdictions. What they didn't expect was how fast the ground would move beneath them. The project began in 2023, when AI was just emerging as a transformative tool, and by the time the project was released, an entirely new generation of AI coding tools had arrived - and lessons the team had learned six months earlier no longer applied.

That whiplash wasn't unique to the WRI team. At NCEAS, a kind of think tank for environmental science, dozens of research teams work side by side on everything from wildfire to biodiversity to climate change. Up and down the building, the same conversations kept surfacing: junior researchers leaning on AI they didn't fully understand, veteran scientists doubting it could be trusted at all, and everyone in between quietly inventing their own rules for what counted as responsible use. WRI wasn't an isolated case - it was a case study of a reckoning already underway across the center.

Rather than let every project work it out alone, NCEAS brought its community together, spanning researchers, software developers, data analysts and professors, to hash it out directly.

What began as internal guidance didn't stay internal for long. The result, refined through literature review and months of co-writing among 22 researchers, developers and data analysts, is "Ten simple rules for effective use of generative AI for code development in environmental science," - offered not just throughout NCEAS, but to the entire field.

"What started as guidance for our own community ended up filling a gap nobody else had addressed," said senior author Cat Fong, a researcher at NCEAS. "Existing advice was written for software engineers, or for science in the abstract. Almost none of it accounted for what our field actually looks like - messy, multi-source data, small teams, wildly different levels of coding experience in the same room. And because some of us were openly skeptical of GenAI going into this, what came out of it is honest about the tradeoffs, not just enthusiastic about the tool."

The 10 rules that came out of those conversations are organized around three phases of working with AI: before, during and after coding. The guidelines cover the preparation for the project, with AI selection; promote best practices during coding with AI; and verification of the AI-generated code and documentation of the process after coding.

The authors are equally direct about what could go wrong if the field doesn't get this right:

"GenAI's benefits are not landing evenly," King said. "Male researchers report larger productivity gains than their female counterparts, and a recent UN report found that while roughly two-thirds of people in some high-income countries use GenAI tools, usage in many low-income countries hovers near just 5%." Additionally, according to the researchers, as leading GenAI companies shift toward paid tiers, the most capable tools risk becoming inaccessible to researchers at underfunded institutions and in low-income countries, turning what was billed as an equalizer into a new barrier to entry.

The infrastructure behind GenAI also carries its own environmental costs, the researchers said, but there is a lot of uncertainty as to the magnitude because it's a rapidly changing landscape and much of the data needed to make these estimates are privately held. Data centers are projected to consume 4-12% of all U.S. electricity by 2030, and up to 32 billion gallons of water per year by 2028. Meanwhile, GenAI's rise has coincided with rising unemployment among recent computer science graduates and a steep decline in software development job postings, added Fong, "raising hard questions about who is responsible for a talent pipeline that public investment built and private industry has since disrupted."

The researchers stop short of prescribing whether GenAI should be used for a given task, arguing that question deserves its own dedicated ethical scrutiny. What they do argue is that once a researcher has decided to use it, doing so well is a skill - one their field hasn't yet been given the tools to build.

The Wildfire Resilience Index, used throughout the paper as a real-world example of the rules in action, is available at wildfireindex.org.

About UC Santa Barbara

The University of California, Santa Barbara is a leading research institution that also provides a comprehensive liberal arts learning experience. Our academic community of faculty, students, and staff is characterized by a culture of interdisciplinary collaboration that is responsive to the needs of our multicultural and global society. All of this takes place within a living and learning environment like no other, as we draw inspiration from the beauty and resources of our extraordinary location at the edge of the Pacific Ocean.