Posted on October 28, 2021
After almost two decades of working on coastal protection and restoration efforts in Louisiana, Carol Parsons Richards knows all too well the destruction hurricanes can cause.
A coastal resources scientist manager for the state’s Coastal Protection and Restoration Authority, Richards also realizes how critical storm modeling is to help prepare for a future made unpredictable by climate change.
“Storm modeling becomes more important every year with all the hurricanes that Louisiana has been exposed to,” she says. “Any time folks have to evacuate, it costs money, and Louisiana is not a rich state.
“We need to make the best decisions that we can make, considering the huge amount of scientific uncertainty about what might happen here in the future.”
Informing those decisions is the goal of a project underway between the Louisiana authority and the U.S. Army Corps of Engineers. The authority will use the Corps’ premier storm modeling system, CSTORM-MS, to study the possibility of harmful seawater surges up the Mississippi River, a major economic driver for the region, during a hurricane or major storm.
The initiative is an example of CSTORM’s growing importance for assessing storm risks to coastal communities. But using the system means the Army Corps must compete for limited time on U.S. Department of Defense supercomputers with a range of other projects, from COVID-19 modeling to military analyses.
Recognizing that reality, the U.S. Army Corps Engineer Research and Development Center (ERDC) is partnering with Microsoft to use cloud computing to improve its coastal ocean modeling and better enable data-sharing.
The initiative was started as a cloud computing pilot project under the Department of Defense’s High Performance Computing Modernization Program (HPCMP) and then received a grant from Microsoft’s AI for Earth program. The project will involve running CSTORM on Azure Government and exploring the possibility of using artificial intelligence and machine learning to speed data collection and gain new insights.
The ERDC’s growing need for computing power was underscored last year, when the center wrapped up the largest study of its type to date – an analysis of how future storms and natural disasters could impact communities along the U.S. South Atlantic coast.
The project, which used CSTORM to model some 2,300 synthetic storms over a huge geographic area including Puerto Rico and the U.S. Virgin Islands, required running several HPCMP supercomputers continuously for about three months and used a staggering 240 million hours of computer processing.
“It’s a massive amount of computational hours that go into these simulations,” says Chris Massey, an ERDC research mathematician who led the modeling project.
The hope, Massey says, is that running CSTORM on Azure Government can help alleviate the demand on DOD supercomputers, which can lead to lengthy wait times, missed deadlines or having to restrict studies with tight deadlines – for example, storm modeling during a hurricane.
“You’ve got lots of scientists and decision-makers running models and simulations on these supercomputers, and all of them have varying levels of importance,” says Massey, who works in the ERDC’s Coastal & Hydraulics Laboratory in Vicksburg, Mississippi.
“Just because one person or one group suddenly has a big need doesn’t mean everybody else’s need goes away.”
CSTORM, which Massey helped develop, is a key tool in helping the Army Corps plan for and build flood-risk management infrastructure such as seawalls, dikes and levees. The data it provides can help determine, for example, where to build a levee and whether putting it in one location could lead to flooding elsewhere. The system is also used to inform planning during hurricanes and other extreme weather events.
If storm modeling is restricted for lack of supercomputing resources, Massey says, that could result in costly infrastructure overbuilds to allow for the needed safety margins.
“If you’re building a levee that goes 10, 20 miles or so and you’ve got to have an error tolerance on how high that levee is, you’re talking millions of dollars if you’ve got to build it higher,” Massey says.
CSTORM is also used by other government agencies and academic researchers. Migrating it to the cloud, Massey says, will provide greater access and enable the Army Corps to more easily share data that is currently stored on highly secured supercomputers.
“These are massive datasets that are so rich in detail and ripe for so many different uses that we want to put them in the hands of as many people we can,” he says.
The initial phase of the ERDC-Microsoft partnership focused on testing CSTORM in the cloud; the next will involve using Azure resources to extend a North Atlantic CSTORM study to include a sea level rise not previously simulated. From there, the project team plans to explore applying AI and machine learning algorithms to the system.
Bruno Sanchez-Andrade Nuño, Microsoft’s planetary computer program director and the head of AI for Earth, says AI could help run the complex CSTORM system more quickly. Reducing processing time and costs, he says, allows researchers to explore climate change-related factors not reflected in historical weather data, like the impacts of higher temperatures and humidity levels on weather events.
Using AI, researchers could also investigate storm risks in U.S. locations that previously weren’t exposed to storms, or even in other countries, Sanchez-Andrade says.
“That’s the nature of this collaboration, to explore where and how AI can help speed things,” he says. “Many places in the world do not have any kind of (weather) modeling. It’s extremely interesting to explore if we could use the system in other places.”
Massey says machine learning – an application of AI that uses mathematical models to enable a computer to learn – could allow researchers to feed data on an approaching storm through an algorithm and quickly assess impacts to a particular area. Or they could run a series of storm scenarios with varying speeds and directions to help inform decisions such as whether to evacuate an area.
“That’s a game-changer, because you don’t want to just look at one realization of that storm that’s coming in,” he says. “You want to know, what happens if it moves left or moves right? What happens if it slows down or gets more intense?
“Having a suite of pre-computed storms and their responses, combined with machine learning-trained datasets, you can produce estimates for water levels and surge heights in a matter of seconds instead of hours,” Massey says. “That’s a very powerful tool for planners and decision-makers.”
The partnership, which represents an initial foray into cloud computing for the Army Corps, grew out of conversations between the Army Corps and Microsoft more than a year ago about the challenges of securing enough computing capacity to accommodate the ERDC’s storm modeling and other projects.
“We only have so much compute capacity, and right now we’re always having problems of projects sitting in queues,” says Carol Wortman, chief architect for the ERDC’s information technology lab, who oversees supercomputer allocations.
“If we could run some of those in the cloud, that’s of great interest to us.”
Wortman and Massey began talking with Tim Carroll, Microsoft’s director of HPC & AI for Research, about the possibility of using Azure Government to run CSTORM. Carroll immediately saw the potential for the cloud to help make the system accessible to more users.
“It’s about making the answers that (CSTORM) provides available to anybody,” Carroll says. “These models inform life-and-death decisions.”
Carroll likens Microsoft’s participation in the partnership to the company providing Azure computing time to a group that developed a ventilator splitter system for COVID-19 patients in the early days of the pandemic. Just as it stepped up to help during that crisis, Carroll says, Microsoft is providing Azure Government resources to address another global challenge.
“We at Microsoft need to be responding to the climate crisis in the same way that we stepped up for the COVID-19 work,” he says. “We’re using data and we’re using the tools that are at our disposal to accelerate solutions that are going to help people real time, right away.”
CSTORM and the data it provides, Carroll says, are “incredibly valuable to the world.”
While the partnership is still in the early days, Massey sees potential for AI and machine learning to help with updating CSTORM datasets that need to be refreshed about every decade as infrastructure, sea levels and topography change.
“We see a cycle of having to refresh and extend these datasets into the future as we go forward,” he says. “Maybe at some point, machine learning can show us a smarter way of doing it.”
