Performance issues while exporting Simulation to .CRF

[Fabian_Schroeer]

Hi, we are currently using ArcGIS Pro to simulate all areas affected by heavy rain and flooding within our jurisdiction.
We export the results to a .CRF file with a timestamp every 15 minutes, but we would also like to export the maximum values per cell throughout the entire simulation.
To achieve the second scenario, we need to set the export interval to every 2–5 seconds and then use a “Maximum” summary statistic to export the maximum values from each slice.

The problem we now face is that the export performance of the .CRF file is extremely poor. If we simulate for 5 hours and create an export every 5 seconds, we end up with 3,600 slices in the .CRF file. As soon as the flood simulation export is complete, up to 4 additional dimensions are automatically calculated for each slice, but ArcGIS Pro freezes throughout this process.

Especially when we set the export interval to 1–2 seconds, I have to wait several days before ArcGIS Pro stops freezing.

[CodyPatterson]

Hey @Fabian_Schroeer 

So just out of curiosity, why does the export interval have to be so quick, 2-5 seconds would create an immense amount of data, which more than likely directly contributes to the time that ArcGIS Pro is taking to do these calculations.

If possible, I'd move the interval to 60+ seconds or even 3-5 minutes, how are you calculating the maximum summary statistic? Using the Aggregate tool?

Cody

[Fabian_Schroeer]

Hi @CodyPatterson 

Yeah sounds weird, but when I simulate a heavy rain scenario, set the export interval to 10 seconds, and then calculate the maximum water depths from each export, it creates wavy-looking data like here:

The faster the water flows, the greater the distances between the waves. So if I set the interval to 1–2 seconds, it would at least reduce the likelihood of generating such “waves” in certain places.

For this, I use the “Summary Statistics” tool in the ‘Multidimensional’ section and then select “Maximum”.
(Or using the aggregate tool, seems to do the same)

Best regards
Fabian

[NathanShephard]

Hi Fabian,

If the waves are the primary issue, then there are two configuration options that help reduce that effect occurring in the math. 

Are you using the following options?

1. Set a Roughness value.  Even a hard-coded value for the whole scenario, such as 0.035, will reduce the way the core (St Venant) equation can result in a build-up-and-release of water as it starts its initial movement (eg: over small barriers, etc). If you're able to use a raster to define variable roughness over the AOI, that will also help with movement speed and accuracy.  (Note - the infiltration options also help a little).

2. Use the 'High' temporal resolution option. This does 20x more calculations per simulation-second than 'Low' temporal resolution, which allows water to move downhill in smaller levels (fewer large waves).

If you're already using both these options... then maybe the "heavy" rainfall rate you are using might be creating deep/fast flows that are not well-suited for the shallow water equation model.  ☹️  If the value is extremely high - as in, "not experienced by humans before" - then you should do a longer scenario with a smaller / feasible rainfall rate.

Otherwise, the general method of exporting multiple results to generate per-cell maximum water levels is solid - it's the volume and size of the export that is taking the time. 

You could also think about using the caching model to help with export time. That is, in the Layer Properties > Cache tab, set a small-ish interval (like one minute), run the scenario, and then use the "use layer's cache interval" export option to (effectively) convert the cache into a CRF without needing to rerun it all.

Hope this helps!

Thanks,

-Nathan Shephard.