topic What is your "import arcpy" overhead? in Python Questions

What is your "import arcpy" overhead?

MattWilkie1 — Sat, 11 Dec 2021 22:23:48 GMT

I'm curious, how long does it take in your machine(s) for your scripts to get ready to start working?

Put something like this at the beginning of some scripts and share the results.

from timeit import default_timer as timer
start = timer()
from datetime import datetime, timedelta
print('=== {}'.format(datetime.now()))

start_arc = timer()
import arcpy
done_arc = timer() - start_arc

print('--- import arcpy overhead: {}'.format(timedelta(seconds=done_arc)))

#import os, math, ...
#carry on with your stuff

elapsed_time = timedelta(seconds=timer() - start)
    # get timer seconds and convert to hh:mm:ss
print('=== Total Elapsed Time: {}'.format(elapsed_time))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

After about a week and perhaps 30 trials in various circumstances the fastest time I've ever seen is 7.8 seconds, the slowest is 10.5, and the median is ~8.5.

(updated code to run in py 2.7 and 3.6+)

Re: What is your "import arcpy" overhead?

BlakeTerhune — Sat, 11 Dec 2021 22:23:51 GMT

On Windows 10 with ArcGIS 10.5.1, run from VS Code

--- import arcpy overhead: 0:00:02.766314
=== Total Elapsed Time: 0:00:02.768425‍‍‍‍

Re: What is your "import arcpy" overhead?

DanPatterson_Retired — Thu, 05 Mar 2020 22:23:03 GMT

is everyone in a rush today

or is esri's "call home" slow today?

https://community.esri.com/thread/249381-strip-arcpy-libraries

Re: What is your "import arcpy" overhead?

MattWilkie1 — Thu, 05 Mar 2020 22:46:40 GMT

Oh wow, I want times like that. It's so annoying to wait 10 secs for startup and then have the real work happen in half that time. I wonder why mine is so much slower. What's your machine and how are you licensed (AGOL Named User, Concurrent, Single Use, Portal, ...)?

My machine:

Dell Precision Tower 7810
2 physical processors: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz, 2401 Mhz, 6 Core(s), 12 Logical Processor(s)

32 GB Ram

Win10 Enterprise x64, version 10.0.18362 Build 18362

C: and 😧 are Samsung SSD 840 EVO 500GB (SSD).

Licensing:

Pro: AGOL Named User

ArcMap: Concurrent license manager (in same building).

Pro's python 3.6 via Pyzo IDE:

--- import arcpy overhead: 0:00:09.052498

ArcMap's python 2.7 via Pyzo IDE:

--- import arcpy overhead: 0:00:06.414285

(I updated the code in first post so it will run in python 2.7 also)

Re: What is your "import arcpy" overhead?

BenTurrell — Thu, 05 Mar 2020 22:56:22 GMT

Hey Matt Wilkie‌,

My results are:

=== 2020-03-06 09:54:51.555443
--- import arcpy overhead: 0:00:05.055231
=== Total Elapsed Time: 0:00:05.098868

Thanks,

Ben

If this answer has helpful please mark it as helpful. If this answer solved your question please mark it as the answer to help others who have the same question.

Re: What is your "import arcpy" overhead?

JoshuaBixby — Fri, 06 Mar 2020 00:49:53 GMT

On my local PC, ArcGIS Desktop 10.8 runs around 4 seconds while ArcGIS Pro 2.5 takes 14 seconds. The difference in times on my machine is easily explained by licensing mode/model. For ArcGIS Desktop 10.8 I am using Single-Use (which will be the fastest of any licensing model) while for Pro 2.5 I am using Named User licensing from AGOL. I am quite sure that Single Use licensing for Pro would put it down around 6 seconds or faster.

If you care about ArcPy startup times, it starts with sharing what your current licensing model is.

Re: What is your "import arcpy" overhead?

ShaneMiles — Sat, 11 Dec 2021 22:23:53 GMT

Hi Matt Wilkie‌,

My results were:

=== 2020-03-06 15:15:35.488000
--- import arcpy overhead: 0:00:03.081489
=== Total Elapsed Time: 0:00:03.094335

Shane

Re: What is your "import arcpy" overhead?

JoeBorgione — Fri, 06 Mar 2020 22:22:20 GMT

I surprised to see:

=== 2020-03-06 15:20:12.398747
--- import arcpy overhead: 0:00:02.838384
=== Total Elapsed Time: 0:00:02.838892

It always seems much longer that that.

(arcgis Pro 2.5, advanced license as named user in agol...)

Re: What is your "import arcpy" overhead?

JoshuaBixby — Fri, 06 Mar 2020 22:25:32 GMT

You either had Pro open or was working with Pro earlier in the day. If you reboot and do a cold test, i.e., without having open Pro, I am guessing it will be noticeably longer.

Re: What is your "import arcpy" overhead?

JoeBorgione — Fri, 06 Mar 2020 22:32:51 GMT

after a machine restart:

=== 2020-03-06 15:30:26.956646
--- import arcpy overhead: 0:00:03.617659
=== Total Elapsed Time: 0:00:03.618505

which is still faster than I would have bet on. It always seems longer than that. Just restarted spyder and got:

=== 2020-03-06 15:33:50.404610
--- import arcpy overhead: 0:00:03.998263
=== Total Elapsed Time: 0:00:03.998689

interesting.....

Re: What is your "import arcpy" overhead?

JoshuaBixby — Fri, 06 Mar 2020 23:05:16 GMT

That is interesting. Are you sure you aren't sitting in an Esri office somewhere...

Re: What is your "import arcpy" overhead?

DanPatterson_Retired — Fri, 06 Mar 2020 23:20:23 GMT

Cold start (both computer and operator)

Named User license

=== 2020-03-06 18:15:53.214205
--- import arcpy overhead: 0:00:07.948694
=== Total Elapsed Time: 0:00:07.949882

Spyder, kernel restart

=== 2020-03-06 18:17:48.751537
--- import arcpy overhead: 0:00:07.402486
=== Total Elapsed Time: 0:00:07.403328

Sadly, just about everything in the stream imports arcpy at some point

but if you just import what you need, you keep the namespace uncluttered.

Also, import python modules and other 3rd party modules "before" you import arcpy

My favorite... just run it again with no restart

=== 2020-03-06 18:19:29.711990
--- import arcpy overhead: 0:00:00.000067
=== Total Elapsed Time: 0:00:00.000738

Re: What is your "import arcpy" overhead?

MattWilkie1 — Sat, 07 Mar 2020 00:43:44 GMT

I updated the script so it outputs as a table, for easier comparison:

What is your "import arcpy" overhead? · GitHub

Does anyone know how to add what the license type and status is? (are we currently in mode "Single Use, Concurrent, Concurrent but Borrowed, AGOL, AGOL but Offline" etc.?)

And what about automatically appending to a public report Gist or similar?

Re: What is your "import arcpy" overhead?

DanPatterson_Retired — Sat, 07 Mar 2020 01:06:06 GMT

They may be in the arcgis module

Do you want the additions there? or here?

Re: What is your "import arcpy" overhead?

MattWilkie1 — Sat, 07 Mar 2020 21:15:48 GMT

There would be easier to track I think (and thanks in advance!)

Re: What is your "import arcpy" overhead?

MattAdams — Mon, 09 Mar 2020 20:24:35 GMT

Here are my results on Windows 10 running ArcGIS Pro 2.5:

=== 2020-03-09 16:22:25.363385
--- import arcpy overhead: 0:00:03.624114
=== Total Elapsed Time: 0:00:03.646515

Re: What is your "import arcpy" overhead?

CarlosSousaFerreira — Tue, 10 Mar 2020 12:18:26 GMT

Here are my results running from command line on Windows 10 Enterprise (64 bit); RAM: 8GB; Processor: Intel Core i#-6100 CPU @ 3.70GHz; ArcGIS Desktop 10.5.1 - License: Advanced (ArcInfo) Single Use

=== 2020-03-10 12:05:39.100000
--- import arcpy overhead: 0:00:02.663260
=== Total Elapsed Time: 0:00:02.666598

Surprised to see how it is a good result comparing with the other answers

Re: What is your "import arcpy" overhead?

Anonymous User — Tue, 10 Mar 2020 15:23:10 GMT

Desktop 10.5 - concurrent license; Win 10 Enterprise x64, 64GB RAM; test through PyCharm

First run: 9.35 seconds

Second run: 3.86 seconds

Pro 2.4 - AGOL license; test through PyCharm

First run: 5.8 seconds

Second run: 4.3 seconds