topic Re: arcpy import is really slow (2+ minutes!!!) in Python Questions

arcpy import is really slow (2+ minutes!!!)

DavidAskov1 — Sat, 11 Dec 2021 07:08:06 GMT

When I import arcpy under the ArcGIS 10.5 python interpreter, it is REALLY slow! I am seeing times of 149 - 197 seconds. Every once in a while, it will import in about 2.5 seconds, but the 2-3 minutes times are more typical. Further info:

I noticed it after borrowing a license for ArcGIS Desktop. I do need to take my laptop off the network, so going back to using the license server is not a good option.
ArcMap and ArcCatalog also can take a similarly long time to start. They are stuck on "Initializing License" on the splash screen for a long time.
This is in my IDE (PyDev/Eclipse), but I get the same loading python 2.7 from the DOS prompt.
Having ArcCatalog or ArcMap already open speeds up neither python nor opening another window of ArcCatalog/ArcMap.
Each of the imports below takes no less time individually than all together.

Here is what I am doing to test the timing (plus using my wristwatch with ArcGIS Desktop):

import timeit
start = timeit.default_timer()
import arcpy
from arcpy import env
from arcpy.sa import *
imports = timeit.default_timer()
print("completed arcpy imports: {} sec".format(int(imports - start)))‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Ideas? Thanks!

Re: arcpy import is really slow (2+ minutes!!!)

DanPatterson_Retired — Thu, 08 Feb 2018 02:09:54 GMT

That is a network issue then. Arcpy doesn't take anywhere near those times. The solution... get ArcGIS PRO or ArcMap installed locally. The nice thing about PRO is you can set it to work in disconnected mode... everything local, no network needed for those critical coding moments in remote areas.

Re: arcpy import is really slow (2+ minutes!!!)

RebeccaStrauch__GISP — Thu, 08 Feb 2018 02:16:11 GMT

Dan_Patterson , I think he does have it all local

I noticed it after borrowing a license for ArcGIS Desktop. I do need to take my laptop off the network, so going back to using the license server is not a good option.

I'm going to borrow and take offline in a few weeks, and will test this tomorrow, but had to upgrade my python IDE, Wing Pro. I know that importing arcpy in Wing Pro does take a little will, but my guess is only 10 secs (based on Desktop 10.3.1). I will report back with my finding in the morning, if no one has answered with another solution.

Re: arcpy import is really slow (2+ minutes!!!)

DavidAskov1 — Thu, 08 Feb 2018 02:17:43 GMT

Thanks, Rebecca - Please let me know the results of your test.

Re: arcpy import is really slow (2+ minutes!!!)

RebeccaStrauch__GISP — Thu, 08 Feb 2018 02:34:36 GMT

Thought I could do a quick test after getting Wing updated. I have Desktop 10.6 on this machine btw. But just importing arcpy is giving an error about a DLL not loading. I remember seeing other folks mention somehting and will have to go back and look at those threads.

error I'm getting

import arcpy
Traceback (most recent call last):
Python Shell, prompt 1, line 1
File "C:\Program Files (x86)\ArcGIS\Desktop10.6\arcpy\arcpy\__init__.py", line 20, in <module>
    import numpy
File "C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\__init__.py", line 187, in <module>
    from . import add_newdocs
File "C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
File "C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\lib\__init__.py", line 18, in <module>
    from .polynomial import *
File "C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\lib\polynomial.py", line 19, in <module>
    from numpy.linalg import eigvals, lstsq, inv
File "C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\linalg\__init__.py", line 51, in <module>
    from .linalg import *
File "C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\linalg\linalg.py", line 29, in <module>
    from numpy.linalg import lapack_lite, _umath_linalg
ImportError: DLL load failed: The network path was not found.
import arcpy
Traceback (most recent call last):
Python Shell, prompt 2, line 1
File "C:\Program Files (x86)\ArcGIS\Desktop10.6\arcpy\arcpy\__init__.py", line 20, in <module>
    import numpy
File "C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\__init__.py", line 187, in <module>
    from . import add_newdocs
File "C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
File "C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\lib\__init__.py", line 17, in <module>
    from . import scimath as emath
ImportError: cannot import name scimath

The path for the linalg.py is there, but not sure about the "from numpy.linalg import lapack_lite, _umath_linalg"

So, a bit of research ...or Dan might pipe in again, since he speaks numpy. Haven't checked for patches yet My guess (hope) is, it's a simple.

but beyond that...it loooked like it was going to be fairly fast (??)> Maybe see if your IDE is giving an error and continuing on...i.e. loading the rest, ignoring these errors? Worth a look.

BTW - I did not test your first two lines in your code, just starting with the arcpy.

Also, for kicks (although not needed), I opened up the Python window in ArcCatalog (which took 14 secs) and then imported arcpy...no error.

Re: arcpy import is really slow (2+ minutes!!!)

RebeccaStrauch__GISP — Thu, 08 Feb 2018 02:44:21 GMT

Actually, look like it might be an issues with the

C:\Python27\ArcGIS10.6\Lib\site-packages\numpy\lib\__init__.py

file? line 17? That pops up as a line of interest.

from . import scimath as emath

and yes, there is a space between the . and import

doesn't look right to me, but not sure what it should be. Any clue Dan_Patterson ? Remember, this happens to be 10.6.0.

ok, now really am stepping away from the machine.

Re: arcpy import is really slow (2+ minutes!!!)

DanPatterson_Retired — Thu, 08 Feb 2018 03:07:13 GMT

my machine... import arcpy test. Now remember, when importing it is only the first import that counts... importing the second third etc times are not relevant since the import is ignored since the namespace is taken...

So, reboot, Python 3.6, ArcGIS PRO 2.1 setup ... arcpy is largely the same, Spyder IDE.

%timeit -n1 -r1 import arcpy
6.32 s ± 0 ns per loop (mean ± std. dev. of 1 run, 1 loop each)

A whole 6 seconds.

Can't test in a 2.7 environment since I have used python 3 for all my code development since I can remember (even though my tools run from within arcmap use 2.7.x)

As for the import numpy in 2.7 for arcmap 10.6... hmm again, I would have to set up an IDE to work with python 2.7. I will try that tomorrow

Re: arcpy import is really slow (2+ minutes!!!)

JoshuaBixby — Thu, 08 Feb 2018 14:53:29 GMT

You have a borrowed license, I assume from your work. When you are at work on your work's local network, are the times faster than when you are off your work's local network? If you completely disconnect from all networks, i.e, including any wi-fi connections, does it import faster?

Although a borrowed license allows you to work off network, ArcGIS Desktop still tries to connect to the license manager and communicate with it. If you are connected to a network, but not your work network, the client will try to communicate with the license server but won't be able to resolve the server's name if the server is on an internal work network. ArcGIS Desktop will wait for a network timeout before giving up and moving on with the use of the borrowed license.

What I typically do is borrow a license and then clear out the license manager name in ArcGIS Administrator. Without a license manager defined, ArcGIS Desktop cannot try to communicate with it. Since borrowed licenses are automatically reclaimed by the server once they expire, you don't have to worry about actually returning the license for the server to get it back. This workflow has some disadvantages, but it can greatly speed up opening ArcGIS Desktop depending on the type of network you are connected to.

Re: arcpy import is really slow (2+ minutes!!!)

DavidAskov1 — Thu, 08 Feb 2018 18:08:06 GMT

Rebecca - The "from . import" is valid. I don't understand it well, but have followed another programmer's code in my org to get it working. I think it just means from the current module, much like dot-slash means from the current directory in Unix/DOS. I've used it to import objects from __init__.py. You can also use "from .module_name import ClassName" to specify another module/Class in the same space as where you're currently located. Anyhow, this is probably getting a bit off on a tangent from the thread, so in order to not annoy the forum moderators, I'd suggest you start another post with this topic.

Re: arcpy import is really slow (2+ minutes!!!)

DanPatterson_Retired — Thu, 08 Feb 2018 18:50:41 GMT

Annoy away... Rebecca and I are moderators and good tangents are fine

Re: arcpy import is really slow (2+ minutes!!!)

RebeccaStrauch__GISP — Thu, 08 Feb 2018 23:57:19 GMT

David Askov I have a call in with tech support because at least in my case I have it narrowed down to using a borrowed license, making sure I am not connected to our network which might grab a license from our concurrent license (even though I have a borrowed one).

With a borrowed license and disconnected, I can "import arcpy" in Catalog and Map, but I can not in my IDE withour getting the errors I mentioned above.

However, if I connect back to our network, it works. I am not certain right now if it works while connected because it sees our CU license manager or because it sees the network/internet. However, my CU license manager still showed it as borrowed (Activated) when I query active connections. So, my guess is, for some reason it just doesn't like the borrowed license with not network connection (for my external IDE only).

Anyway, tech support is testing that.

So, is there anyway for you to see if your IDE is ignoring the errors but continuing, and therefor maybe slowing down?

Again, my system is a bit different than yours, but I do think it is the same issues: Win7, ArcGIS 10.6.0, ArcPro 2.1.1

I went from Pro 2.1 to 2.1.1 earlier this week, and upgraded my MS Office from 2010 to 365(2016) earlier yesterday.

re: slow ArcMap/Catalog start up, I think bixb0012 has some good hints (which I am going to try), but also, make sure you are keeping your Results tab cleaned out. Otherwise is has to read that all in on startup, and then write is all to disk on shutdown. Clearing that up can speed up software startup time.

Re: arcpy import is really slow (2+ minutes!!!)

RebeccaStrauch__GISP — Wed, 14 Feb 2018 21:45:26 GMT

Although this may be unrelated, this is what finally fixed many of my issues, including the "import arcpy" issue on one of my machines. The real issues, I believe, is the external IDE was having an issue accessing the ArcGIS license.

For me, I have to uninstall the Flexara patch for the license manager that was a "highly recommended for all users" last fall. https://support.esri.com/en/Products/Desktop/arcgis-desktop/arcmap/10-6#downloads?id=7521 It says 10.6, but it is for all version back to 10.0. It looks like there was a January 2018 update, and I no longer see the "recommended that all users install" text, so I'm still trying to get clarification on that.

Anyway, if you have those Flexara patches showing in your Control Panel->Add/Remove, try uninstalling them and doing a reboot. if that does not help, try deleting the ArcGIS_* files in the c:\ProgramData\FLEXnet folder. I'm not sure which did the try for sure (did both before the reboot), but now things seem to be working better.

I'll come back with info re: the patch if/when I find out more.

Re: arcpy import is really slow (2+ minutes!!!)

DavidAskov1 — Thu, 15 Feb 2018 19:55:43 GMT

Joshua - This is exactly the issue. Here are the results of my tests to import arcpy:

On work network: 153 sec (can range up to 5 min, rarely faster than this)
Totally disconnected (no LAN cable/wifi): 3.5 sec
On home internet connection, noVPN: 3.3 sec
On home internet, with VPN: 280 sec

Opening ArcMap is about the same. It gets stuck on "Initializing License" on the splash screen.

Following your solution was exactly the issue. The reason it took me so long to reply was that I couldn't figure out your suggestion of removing the info for my license manager. The textbox where I would enter it is disabled and so is the "Change" button, so I couldn't edit them.

Then, I remembered that I am not an administrator on my PC. I located the ArcGIS Administrator in my Windows Start menu, right-clicked on it, and chose "Run as administrator". The textbox is still not editable, but the change button works now.

thanks! David

Re: arcpy import is really slow (2+ minutes!!!)

DavidAskov1 — Thu, 15 Feb 2018 23:45:20 GMT

Rebecca - I have not noticed any difference in my IDE or at the DOS prompt. I use the PyDev plugin to Eclipse. When you set up a new python project, you have to point it to your interpreter (I have a 2.7 from ArcGIS and a 3.6 I installed myself, plus I think Pro has one and QGIS might, too). Once you do that, it is pretty much the same thing as running it at the DOS prompt, except that you get debugging and integration into your project (right-click >> run), but the actual execution is pretty much the same.

Re: arcpy import is really slow (2+ minutes!!!)

JonSwoveland — Tue, 27 Mar 2018 23:59:56 GMT

Thank-you! This was driving me bonkers! I suspected the timeout trying to connect to license manager, but it hadn't occurred to me to simply remove the license manager hostname from ArcGIS Administrator.