Low performances using 'Batch Calculate Statistics' in 'ArcGIS Pro 3.2'

Question

Hello,

I have just started to use 'ArcGIS Pro 3.2.2' (before I used 'ArcGIS Pro 3.1') and I'm working with the tool 'Batch Calculate Statistics':

I have used this tool with 4 geotif files:

3880

With the following settings:

Here below all the details and the time that the application needs to complete this task:

Batch Calculate Statistics
=====================
Parameters

Input Raster Datasets \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_30_GRID.tif;\gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_31_GRID.tif;\gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_32_GRID.tif;\gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_33_GRID.tif
X Skip Factor 1
Y Skip Factor 1
Ignore values 
Skip Existing 
Batch Calculate Statistics Succeeded true
=====================
Messages

Start Time: sabato 27 aprile 2024 11:33:12
Successfully calculated statistics : \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_30_GRID.tif
Successfully calculated statistics : \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_31_GRID.tif
Successfully calculated statistics : \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_32_GRID.tif
Successfully calculated statistics : \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_33_GRID.tif
Succeeded at sabato 27 aprile 2024 11:37:05 (Elapsed Time: 3 minutes 53 seconds)

------------------------

If I try to do the same using 'ArcGIS Pro 3.1' the performances are much much much BETTER:

Batch Calculate Statistics
=====================
Parameters

Input Raster Datasets \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_30_GRID.tif;\gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_31_GRID.tif;\gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_32_GRID.tif;\gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_33_GRID.tif
X Skip Factor 1
Y Skip Factor 1
Ignore values 
Skip Existing 
Batch Calculate Statistics Succeeded true
=====================
Messages

Start Time: Saturday, 27 April, 2024 11:44:21
Successfully calculated statistics : \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_30_GRID.tif
Successfully calculated statistics : \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_31_GRID.tif
Successfully calculated statistics : \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_32_GRID.tif
Successfully calculated statistics : \gisdata.eni.pri\GISDATA\WORK_AREA\SYSTEM_TEST\_DatiTest\EnGISPro\XwareLoader\Loader_Dem\dsm_33_GRID.tif
Succeeded at Saturday, 27 April, 2024 11:44:27 (Elapsed Time: 6.09 seconds)

SO THIS IS THE RESULT:

SoftwareGeoprocessing toolElapsed Time ArcGIS Pro 3.2.2Batch Calculate Statistics3 minutes 53 secondsArcGIS Pro 3.1Batch Calculate Statistics6.09 seconds

Spoiler (Highlight to read)THE TIME IN ArcGIS Pro 3.2.2 IS ABOUT THE 3880% IN MORE THAN IN ArcGIS Pro 3.1THE TIME IN ArcGIS Pro 3.2.2 IS ABOUT THE 3880% IN MORE THAN IN ArcGIS Pro 3.1This is a big issue for us, do you know if ESRI is working to improve these performances like ArcGIS Pro 3.1 ?

Thanks a lot for your support!

Regards,

Marco Zanieri

DavidColey · Answer

I moved over to Catalog and was able to batch calc raster statistics for 835 tif files in like 6 minutes.  Fortunately I still have an 10.8.2 install on a laptop...

RTPL_AU · Answer

Link to Idea:
when-releasing-a-new-version-of-pro-benchmark-gp

@SSWoodward @Bud

DavidColey · Answer

I can't even get past 2 tif files without the either the tool freezing up or my python script just hanging after taking 3 - 5 minutes per file, no matter if I am using a local workspace or across the network.  Typically, I run something like:

import sys, os, arcpy, time, smtplib
from datetime import date, datetime, timedelta
from arcpy import env

# Local variables...
message = ""
current_time = time.asctime()
todaysDate = date.today()
priorYear = str(todaysDate - timedelta(days=547)) #547
starting_time = time.time()

arcpy.SetLogHistory(False)
env.configKeyword= "DEFAULTS"
env.overwriteOutput = True

try:
env.workspace = "D:\Imagery"
#env.workspace = r"E:\Tiles\GeoTIFF\Property"
# Had to go into directory and give account full control - see if works for loop, after 2 files locks up
#env.workspace = r"\networkshare\imagery\2024\GeoTIFFs"
rasters = arcpy.ListRasters("*", "TIF")
for raster in rasters:
print(raster)
arcpy.CalculateStatistics_management(raster, "1", "1", "", "SKIP_EXISTING", "")
print (raster + " stats calculated")
message = message + "

" + raster + " stats calculated"

except Exception:
# If an error occurred, print line number and error message
import traceback, sys
tb = sys.exc_info()[2]
e = sys.exc_info()[1]
print(e.args[0])
print ("Line %i" % tb.tb_lineno)
message = message + "
" + "Line %i" % tb.tb_lineno
message = message + "
" + str(e)

And like I said, it won't get past 2 tif files before just hanging.  I should mention I have almost 900 of these - out entire county, 3 band tif images, 4" resolution.  I used the same script workflow above to BuildPyramids, that took about 8 hours.  Not great, but I can live with that.

@KenBuja  - would you happen to have any ideas?

Thanks,

David

Encuentra más publicaciones etiquetadas con

Accepted answers

All comments

Únete a la Conversación

Enlaces Rápidos