topic Re: ImportError: DLL load failed: The specified module could not be found. in Python Questions

ImportError: DLL load failed: The specified module could not be found.

AustinMilt — Mon, 24 Jun 2013 14:40:38 GMT

My issue is similar to the one already discussed below:

http://forums.arcgis.com/threads/18047-quot-DLL-Load-Failed-The-specified-procedure-could-not-be-found.-quot?highlight=ImportError%3A+DLL+load+failed%3A+module+found

Specifically, I have written some code in Cython and compiled to a .pyd. In order to run on other machines with ArcGIS 10.1, I have included with my code the Microsoft Manifest (Microsoft.VC90.CRT.manifest) and other required DLLs (msvcm90.dll, msvcp90.dll, msvcr90.dll) that are included in the ArcGIS 10.1 installation. I do not think the code was compiled with these DLLs, but I have been able to get the .pyd to import successfully on other computers without Visual Studio using these DLLs.

When I import my code before arcpy, it imports fine. When I import after, it throws the error "ImportError: DLL load failed: The specified module could not be found."

Is there an option for me to make my .pyd work when I distribute this code as part of a toolbox?

Please let me know if more details or the files themselves would be helpful.

Re: ImportError: DLL load failed: The specified module could not be found.

JasonScheirer — Sat, 11 Dec 2021 07:48:05 GMT

ArcGIS uses a different version of the msvc libraries than what you're including, so once they try to load there's a version conflict when it tries to import your local ones. I'd recommend getting rid of the manifest altogether.

If you're using a setup.py to build, the thing I do is patch \PythonInstall\Lib\distutils\msvc9compiler.py and locate the following lines:

            # embed the manifest
            # XXX - this is somewhat fragile - if mt.exe fails, distutils
            # will still consider the DLL up-to-date, but it will not have a
            # manifest.  Maybe we should link to a temp file?  OTOH, that
            # implies a build environment error that shouldn't go undetected.
            mfinfo = self.manifest_get_embed_info(target_desc, ld_args)
            if mfinfo is not None:
                mffilename, mfid = mfinfo
                out_arg = '-outputresource:%s;%s' % (output_filename, mfid)
                try:
                    self.spawn(['mt.exe', '-nologo', '-manifest',
                                mffilename, out_arg])
                except DistutilsExecError, msg:
                    raise LinkError(msg)

And just comment them all out.

Re: ImportError: DLL load failed: The specified module could not be found.

AustinMilt — Mon, 24 Jun 2013 16:03:03 GMT

ArcGIS uses a different version of the msvc libraries than what you're including, so once they try to load there's a version conflict when it tries to import your local ones. I'd recommend getting rid of the manifest altogether.

If you're using a setup.py to build, the thing I do is patch \PythonInstall\Lib\distutils\msvc9compiler.py ...
And just comment them all out.

Two questions:
1) Is there a way for me to compile the code using the same libraries used by ArcGIS?
2) What's the danger or consequences of doing what you suggested?

Re: ImportError: DLL load failed: The specified module could not be found.

AustinMilt — Sat, 11 Dec 2021 07:48:07 GMT

I tried editing out the manifest embedding section of msvc90compiler.py as suggested and then re-compiling. I still get the same error.

Just to add details:

Here is my setup.py (very simple):

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy

regiongroup = Extension("RegionGroup", ["RegionGroup.pyx"])

setup(
    cmdclass = {'build_ext': build_ext},
    include_dirs = [numpy.get_include()],
    ext_modules = [regiongroup]

)

The compile command:

C:/Path/to/python.exe setup.py build_ext --inplace

And the compiling output:

running build_ext
cythoning RegionGroup.pyx to RegionGroup.c
building 'RegionGroup' extension
creating build
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBU
G -IC:\Users\milt\Desktop\ENV\lib\site-packages\numpy\core\include -IC:\Python27\ArcGIS10.1\include
-IC:\Users\user\Desktop\ENV\PC /TcRegionGroup.c /Fobuild\temp.win32-2.7\Release\RegionGroup.obj
RegionGroup.c
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\link.exe /DLL /nologo /INCREMENTAL:NO /LIB
PATH:C:\Python27\ArcGIS10.1\Libs /LIBPATH:C:\Users\user\Desktop\ENV\libs /LIBPATH:C:\Users\user\Desk
top\ENV\PCbuild /EXPORT:initRegionGroup build\temp.win32-2.7\Release\RegionGroup.obj "/OUT:C:\Users\
user\Dropbox\Graduate School\Dissertation\Marcellus Shale Gas Project\ArcGIS_Build\v0.3\compiled\Reg
ionGroup.pyd" /IMPLIB:build\temp.win32-2.7\Release\RegionGroup.lib /MANIFESTFILE:build\temp.win32-2.
7\Release\RegionGroup.pyd.manifest
Creating library build\temp.win32-2.7\Release\RegionGroup.lib and object build\temp.win32-2.7\Rel
ease\RegionGroup.exp

Note that despite having commented out the manifest embedding section in msvc90compiler.py, I still see the /MANIFESTFILE:build\temp.win32.7... which I am guessing is being embedded. That manifest file looks like this:

<?xml version='1.0' encoding='UTF-8' standalone='yes'?>
<assembly xmlns='urn:schemas-microsoft-com:asm.v1' manifestVersion='1.0'>
  <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
    <security>
      <requestedPrivileges>
        <requestedExecutionLevel level='asInvoker' uiAccess='false' />
      </requestedPrivileges>
    </security>
  </trustInfo>
  <dependency>
    <dependentAssembly>
      <assemblyIdentity type='win32' name='Microsoft.VC90.CRT' version='9.0.21022.8' processorArchitecture='x86' publicKeyToken='1fc8b3b9a1e18e3b' />
    </dependentAssembly>
  </dependency>
</assembly>

Re: ImportError: DLL load failed: The specified module could not be found.

JasonScheirer — Mon, 24 Jun 2013 19:24:11 GMT

Commenting out those specific lines will not prevent the manifest from being created, only bypass the part where it embeds it in the PYD.

Have you tried 1. NOT packaging those DLLs with your library, and then 2. making sure the <PYTHON>\Lib\site-packages\<yourlib> is completely cleared out of all files before rebuilding and testing?

Re: ImportError: DLL load failed: The specified module could not be found.

AustinMilt — Mon, 24 Jun 2013 19:41:28 GMT

Commenting out those specific lines will not prevent the manifest from being created, only bypass the part where it embeds it in the PYD.

Have you tried 1. NOT packaging those DLLs with your library, and then 2. making sure the <PYTHON>\Lib\site-packages\<yourlib> is completely cleared out of all files before rebuilding and testing?

Yes and yes. For the second suggestion, I may not be following proper protocol or may not understand your suggestion. I havent put any of my scripts in <PYTHON>\Lib\site-packages, including the msvc libraries. Nor have I specified the library locations in any of the files to-be-compiled. One of my previous questions was how I might specify specific msvc library paths for the compiler.

Re: ImportError: DLL load failed: The specified module could not be found.

JasonScheirer — Mon, 24 Jun 2013 20:15:59 GMT

You'll notice a bunch of Microsoft.VC* folders in <ArcGIS Install>\bin\. That has the various DLLs that ArcGIS uses. You can use one of many methods to determine their versions, set those versions as your dependencies, and then go back to embedding the manifest.

Though, based on how you're describing the symptoms, it may be something else at play here.

Re: ImportError: DLL load failed: The specified module could not be found.

AustinMilt — Tue, 25 Jun 2013 13:15:21 GMT

SOLUTION

I had thought upon posting this topic that I would not be able to import the compiled code before arcpy, or that if I found a work-around that let me do so, it would fail some other way.

It turns out I was wrong. For whatever reason, importing the compiled code before arcpy works just fine.

I believe the issue is still relevant and I would like to get to the bottom of it.

Re: ImportError: DLL load failed: The specified module could not be found.

AustinMilt — Tue, 25 Jun 2013 13:17:03 GMT

SOLUTION

I had thought upon posting this topic that I would not be able to import the compiled code before arcpy, or that if I found a work-around that let me do so, it would fail some other way.

It turns out I was wrong. For whatever reason, importing the compiled code before arcpy works just fine.

I believe the issue is still relevant and I would like to get to the bottom of it.

Whoopsy daisies. Apparently my other account that I thought was gone is still lurking.

Re: ImportError: DLL load failed: The specified module could not be found.

AustinMilt — Tue, 25 Jun 2013 13:17:58 GMT

You'll notice a bunch of Microsoft.VC* folders in <ArcGIS Install>\bin\. That has the various DLLs that ArcGIS uses. You can use one of many methods to determine their versions, set those versions as your dependencies, and then go back to embedding the manifest.

Though, based on how you're describing the symptoms, it may be something else at play here.

I am not sure how to set the dependencies for these particular files, and the python help on this is sparse.