nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Python Execute Reviewer Batch Job script is slower in ArcGIS Pro

deleted-user-ZTub55yWbsya

Running arcpy stand-alone scripts with ArcGIS Pro (64-bit) is supposed to be faster than ArcGIS 10.5, but it seems to be the opposite with my first attempt at converting a stand-alone script to use ArcGIS Pro. The script is long, but among other things it uses Data Reviewer to run a few attribute checks. I've narrowed down where the difference is - specifically on ExecuteReviewerBatchJob_Reviewer:

arcpy<SPAN class="punctuation token">.</SPAN>ExecuteReviewerBatchJob_Reviewer<SPAN class="punctuation token">(</SPAN>reviewer_gdb<SPAN class="punctuation token">,</SPAN> currentSession<SPAN class="punctuation token">,</SPAN> AttributesBatchJob<SPAN class="punctuation token">,</SPAN> None<SPAN class="punctuation token">,</SPAN> None<SPAN class="punctuation token">,</SPAN> <SPAN class="string token">"ALL_FEATURES"</SPAN><SPAN class="punctuation token">,</SPAN> None<SPAN class="punctuation token">)</SPAN><SPAN class="line-numbers-rows"><SPAN>‍</SPAN></SPAN>

ArcGIS 10.5 takes 1 minute

ArcGIS Pro takes 5 minutes

Perhaps I'm doing it wrong, but my understanding is we should run a stand-alone script with Pro like so:

call "C:\Program Files\ArcGIS\Pro\bin\Python\scripts\propy.bat" C:\GISDataReview.py

Has anyone else seen negative impacts in running Python 3 vs 2.7 using the same script (and assuming syntax is corrected where needed for version 3)? I wanted to compare apples to apples, but if it requires somehow re-working it for Pro I'm open to suggestions. I made sure my Data Reviewer workspace is upgraded for Pro. When I run the execute batch job geoprocessing tool inside Pro it finishes in less than a minute. This script needs to run as a scheduled task.

Vind meer berichten getagd met

Accepted answers

All comments

Drew Flater

Hi Peter,

I work at Esri and have done some testing with the TableToTable tool - I just tried a dataset with ~70k records, exported to a dbf. This took a little over one second. I believe there might be something specific to your data or the settings you are using with the tool that is causing this problem. Maybe your dataset has a lot of fields? My dataset only had about 20. Would it be possible for you to share the data with me, and the Python command you are running? I can set up a box folder to upload the data, just send me a message at dflater@esri.com.

DanPatterson_Retired

Bizarre!!! you do certainly have a time problem there. Report back! It would be interesting to find out what caused this

PeterSchoenfield

I did some additional testing to record the times. Using ArcGIS Desktop python here are the results:

C:\Temp>python TimeTest.py
07:41:50 AM, Tuesday December 19, 2017
07:42:22 AM, Tuesday December 19, 2017
C:\Temp>

Then using the ArcGIS Pro python, same script:

C:\Temp>"C:\Program Files\ArcGIS\Pro\bin\Python\scripts\propy.bat" TimeTest.py
09:18:50 AM, Monday December 18, 2017
05:46:15 AM, Tuesday December 19, 2017
C:\Temp>

I think the results speak for themselves. I will be calling tech support about this.

DanPatterson_Retired

Neat... throw the decorator I posted around a def that contains your calls to TableToTable so you can time the same code in 2.7 and 3.5. If would allow you to check whether there is any changes in arcpy between the two implementations. It would be interesting to track these things.

PeterSchoenfield

I HAVE experienced the same kind of slowness in ArcGIS Pro and in stand-alone python when using TableToTable conversion to export to a DBF. I'm exporting 329518 address records. In ArcCatalog and Python 2.7 it takes about 5 seconds. After multiple attempts in ArcGIS Pro and its Python install I gave up after 10 minutes! Something obviously not right there.

deleted-user-ZTub55yWbsya

Thanks, Dan. I won't assume anything with ArcGIS Pro and stand-alone scripts.

DanPatterson_Retired

you could read for days on the 'which is faster...' debate. In short, it depends... but overall 3.x, except for the thing that you are using. This of course assumes pure python. As soon as you throw other stuff into the mix then all bets are off and that would include anything Arc* related. You could decorate your functions if your code is arranged that way. I found it useful for comparing function I used between 2/3 versions

<SPAN class="keyword token">def</SPAN> <SPAN class="token function">time_deco</SPAN><SPAN class="punctuation token">(</SPAN>func<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>&nbsp; <SPAN class="comment token"># timing originally</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="string token">"""timing decorator function
&nbsp;&nbsp;&nbsp; :print("\n&nbsp; print results inside wrapper or use <return> ... ")
&nbsp;&nbsp;&nbsp; """</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">import</SPAN> time
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">from</SPAN> functools <SPAN class="keyword token">import</SPAN> wraps

&nbsp;&nbsp;&nbsp; @wraps<SPAN class="punctuation token">(</SPAN>func<SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">def</SPAN> <SPAN class="token function">wrapper</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="operator token">*</SPAN>args<SPAN class="punctuation token">,</SPAN> <SPAN class="operator token">**</SPAN>kwargs<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; t0 <SPAN class="operator token">=</SPAN> time<SPAN class="punctuation token">.</SPAN>perf_counter<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token"># start time</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; result <SPAN class="operator token">=</SPAN> func<SPAN class="punctuation token">(</SPAN><SPAN class="operator token">*</SPAN>args<SPAN class="punctuation token">,</SPAN> <SPAN class="operator token">**</SPAN>kwargs<SPAN class="punctuation token">)</SPAN>&nbsp; <SPAN class="comment token"># ... run the function ...</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; t1 <SPAN class="operator token">=</SPAN> time<SPAN class="punctuation token">.</SPAN>perf_counter<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token"># end time</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; dt <SPAN class="operator token">=</SPAN> t1 <SPAN class="operator token">-</SPAN> t0
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">"\nTiming function for... {}"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>func<SPAN class="punctuation token">.</SPAN>__name__<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="string token">"&nbsp; Time: {: <8.2e}s for {:,} objects"</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>dt<SPAN class="punctuation token">,</SPAN> len<SPAN class="punctuation token">(</SPAN>result<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">)</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">return</SPAN> result&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token"># return the result of the function</SPAN>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">return</SPAN> dt&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <SPAN class="comment token"># return delta time</SPAN>
&nbsp;&nbsp;&nbsp; <SPAN class="keyword token">return</SPAN> wrapper<SPAN class="line-numbers-rows"><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN><SPAN>‍</SPAN></SPAN>

Doe mee aan het gesprek

Aangemelde leden kunnen berichten plaatsen, updates volgen en meer. Nieuw hier? Registreer een gratis account.

Snelle Links

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.