error importing arcpy.da package into my Python IDE

[davedoesgis]

I am coding in a Python IDE (integrated development environment), not in Pro. When I do import arcpy, I can then work with most everything I need.  For example, as I write my code, I get code hints telling me about objects, functions, and properties, as you can see in this screenshot: 

Now, if I try to work with anything in the arcpy.da package, I don't get code hints, and also get errors in the IDE's editor. Hovering my mouse over the error, it says "Undefined variable from import: Describe".

I get basically the same error if I try this: 
from arcpy.da import Describe

Just to be clear, my code executes just fine. The issue is that my IDE doesn't give me any hints on coding this and it decorates that line with an error. Of course, looking for errors is an important part of coding, so this is super annoying. 

I see this trying to use external source code, for example. I resolve that by adding the packages to my PythonPath. But when I look at the source in arcpy.da, it is iterating over all the modules from arcgisscripting.da and adding them . As far as I can tell, all of arcpy.da is in a .pyd file here: 

C:\Users\<user>\AppData\Local\ESRI\conda\envs\<env>\Lib\site-packages\arcgisscripting\_arcgisscripting.pyd

My (very limited) understanding is .pyd files are DLLs that are loaded at run time. This makes sense that my IDE doesn't know what to do with it in advance, but it runs just fine. Any advice on how to add modules from a pyd file to the PythonPath is appreciated. 

FWIW: My IDE is PyDev in Eclipse. I mention this last, since it seems like a general problem and don't want to get dragged into the specifics of PyDev. 

[davedoesgis]

Note - Once I get the program running and stop it at a breakpoint, now I can work with everything just fine. This also fits with the idea of it not loading the modules until run-time. In this case, I imported arcpy and started to type the name of a function in the arcpy.da package and it came up with code hints. 

[DanPatterson]

works fine in Spyder, I just have my tips/help directed to a separate window, rather than cluttering up my visual space as I type... but the info is there

[davedoesgis]

Thanks. Unfortunately, Spyder is not approved at my work... There clearly is some way to follow the thread of run-time imports, and PyDev is not following it. It can import raw source, egg, wheel, dll, and .pyd files, but I have to tell it where to find that object, and so far have not located it...

[DanPatterson]

you could always do what the esri code does and import it directly.  I usually use

import arcgisscripting as ags

since it is in the site-packages folder of your install 

[davedoesgis]

That has the same result. I think that's the problem is the whole arcgisscripting package loads from a DLL (as a .pyd file) at runtime and not as source code. The import produces an error in my IDE, but will execute. 

My IDE can add DLL and PYD files to my PythonPath, but I have to locate it in order to do so. I have traced through the Esri source code about as far as I'm able to. It ends up calling function convertArcObjectToPythonObject in C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcobjects\arcobjectsconversion.py and that's where I lose the trail.

Clearly, there is something slightly less persistent in my IDE's imports, but looking at Esri's code, well, it's sharks with lasers...

[DanPatterson]

Things that are IDE specific are out of the realm of the things that can be addressed.  Perhaps bringing this up with them

Is this the github site?

 fabioz/Pydev: Python IDE for Eclipse (github.com)

Developers (pydev.org)

[davedoesgis]

Yes, that's it. I think this issue is at the intersection of Esri saying "your IDE could do this better" and my IDE's support asking "why is this package importing other packages that go 17 levels deep in the traceback with all of this redirection and over-complicated introspection and dynamic importing, plus reading run-time-only DLLs that import other run-time-only DLLs?"

[DanPatterson]

Seem like a case of proprietary software  trying to preserve their investments.  The rabbit hole is fraught with obstacles

[davedoesgis]

FWIW, I tried it in VS Code and got the same result as PyDev.

Idle (delivered with ArcGIS) doesn't give me any code hints for anything ever. Maybe I'm not using it right, but it seems like a very basic text editor that has some Python integration. 

I think VS Code, PyDev, and Idle are the only IDEs allowed in my environment.