I'm doing some climate data analysis using the ESRI multidimension tools. The data set is an Gridmet annual data for daily maximum temperature data. Key point - just one year of data.
dd5_gt5c_annual = arcpy.sa.AggregateMultidimensionalRaster(DD5_gt5c,"StdTime",'SUM',"air_temperature","INTERVAL_KEYWORD","YEARLY")
This command on a raster object takes about 25-30 minutes to run. Same result whether in a notebook or from the UI. Stats like mean don't take very long. I tried the same thing using xarray to sum the same data. Less than a minute.
The SUM function in the work I've done takes a inordinate amount of time.
Given your data are numeric, have you looked at
RasterToNumPyArray—ArcGIS Pro | Documentation
I don't know what arcpy.sa is using, but numpy and scipy may open up some possibilities. The results can be brought back in using NumPyArrayToRaster if you need to visualize the output of some process
I did look at the numpy option. The xarray option as well. The raster -> numpy worked fine, the calculations worked fine. The problem I ran into was that the NumPyArrayToRaster and XarrayToRaster functions failed for me. The multidimensional data resulting from the calcuations would not go back into Raster objects. I have a support ticket in with ESRI about that issue right now.