AnsweredAssumed Answered

ETL tool error

Question asked by lemvig_kommune on Jul 16, 2019

Hi

I've just updated my Pro to 2.4 and now i'm running into this problem, this is just a little example, my big tools are doing the same.

 

I've created a FME workspace containing a creator and an emailer, nothing else.

When running from FME it runs smoothly, but when I'm making a ETL tool in a Toolbox in ArcGIS Pro, and try to run this tool from python, it fails.

 

This is the python script:

import sys, arcpy
try:
    default_path = os.path.dirname(os.path.realpath(__file__))
    print(default_path)
    # arcpy.ImportToolbox(os.path.join(default_path, 'WFStools.tbx'))
    arcpy.ImportToolbox(os.path.join(default_path, 'TestTools.tbx'))

    # tools = arcpy.ListTools('*_WFStools')
    # tools = arcpy.ListTools('*_TestTools')
    # for tool in tools:
    #     print(arcpy.Usage(tool))
    arcpy.emailer_TestTools()

except arcpy.ExecuteError:
    print(arcpy.GetMessages())
    # Handle the code somehow
except Exception:
    e = sys.exc_info()[1]
    print(e.args[0])
    arcpy.AddError(e.args[0])

And this is the error I'm getting



runfile('C:/ArcGIS Projects/wfs_project/test.py', wdir='C:/ArcGIS Projects/wfs_project')
Reloaded modules: arcpy, arcpy.geoprocessing, arcpy.geoprocessing._base, arcpy.toolbox, arcpy.ddd, arcpy.arcobjects, arcpy.arcobjects._base, arcpy.arcobjects.arcobjectconversion, arcpy.arcobjects.arcobjects, arcpy.arcobjects.mixins, arcpy.arcobjects.geometries, arcpy._mp, arcpy.pdfdocument, arcpy.utils, arcpy._symbology, arcpy._renderer, arcpy._colorizer, arcpy.reviewer, arcpy.agolservices, arcpy.analysis, arcpy.aviation, arcpy.ba, arcpy.cartography, arcpy.conversion, arcpy.edit, arcpy.ga, arcpy._ga, arcpy._base, arcpy.gapro, arcpy.geoanalytics, arcpy.geocoding, arcpy.locref, arcpy.lr, arcpy.management, arcpy.maritime, arcpy.md, arcpy.na, arcpy._na, arcpy._na._solvers, arcpy._na._sas, arcpy._na._rs, arcpy._na._odcms, arcpy._na._vrps, arcpy._na._cfs, arcpy._na._las, arcpy._na._nds, arcpy.parcel, arcpy.ra, arcpy.server, arcpy.sfa, arcpy.stats, arcpy.stpm, arcpy.td, arcpy.topographic, arcpy.un, arcpy.wmx, arcpy._wmx, arcpy.interop, arcpy.da, arcpy.mp, arcpy.sharing, arcpy._chart, arcpy.nax, arcpy.sa, arcpy.sa.Raster, arcpy.sa.Utils, arcpy.sa.Functions, arcpy.sa.ParameterClasses, arcpy.sa.CompoundParameter, arcpy.ia, arcpy.ia.Functions, arcpy.ia._ia, arcpy.time, arcpy.cmanagers, arcpy.cmanagers.EnvManager, arcpy.toolbox_code
C:\ArcGIS Projects\wfs_project
Start Time: 16. juli 2019 14:03:34
                FME 2019.0.0.0 (20190328 - Build 19238 - WIN64)
                    Data Interoperability (node locked-crc)
                               Permanent License.
                        Machine host name is: S01GIS181
          Operating System: Microsoft Windows 10 64-bit  (Build 16299)
                 Copyright (c) 1994 - 2019, Safe Software Inc.
                               Safe Software Inc.
FME 2019.0.0.0 (20190328 - Build 19238 - WIN64)
FME_HOME is 'C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\'
Data Interoperability (node locked-crc)
Permanent License.
Machine host name is: S01GIS181
START - ProcessID: 31608, peak process memory usage: 58704 kB, current process memory usage: 58704 kB
FME Configuration: Command line arguments are `C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\fme.exe' `C:\Users\agsSA\AppData\Roaming\ESRI\Interoperability\logs\custom-tool-20190716-140334-1563278614109_21088.fme'
FME Configuration: Connection Storage: 'C:\Users\agsSA\AppData\Roaming\Safe Software\FME\'
Shared folders for formats are : C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\datasources;C:\Users\agsSA\Documents\FME\Formats
Shared folders for transformers are : C:\Users\agsSA\AppData\Roaming\Safe Software\FME\Packages\19238-win64\transformers;C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\transformers;C:\Users\agsSA\AppData\Roaming\Safe Software\FME\FME Store\Transformers
Shared folders for coordinate systems are : C:\Users\agsSA\Documents\FME\CoordinateSystems
Shared folders for coordinate system exceptions are : C:\Users\agsSA\Documents\FME\CoordinateSystemExceptions
Shared folders for coordinate system grid overrides are : C:\Users\agsSA\Documents\FME\CoordinateSystemGridOverrides
Shared folders for CS-MAP transformation exceptions are : C:\Users\agsSA\Documents\FME\CsmapTransformationExceptions
Shared folders for transformer categories are : C:\Users\agsSA\Documents\FME\TransformerCategories
FME Configuration: Reader Keyword is `MULTI_READER'
FME Configuration: Writer Keyword is `NULL_DEST'
FME Configuration: Writer Group Definition Keyword is `NULL_DEST_DEF'
FME Configuration: Reader type is `MULTI_READER'
FME Configuration: Writer type is `NULL'
FME Configuration: No destination coordinate system set
FME Configuration: Current working folder is `C:\ArcGIS Projects\wfs_project\workspaces'
FME Configuration: Temporary folder is `C:\Users\agsSA\AppData\Local\Temp', set from environment variable `TEMP'
FME Configuration: FME_HOME is `C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\'
FME Configuration: FME_BASE is 'yes'
FME Configuration: FME_MF_DIR is 'C:\Users\agsSA\AppData\Roaming\ESRI\Interoperability\logs/'
FME Configuration: FME_MF_NAME is 'custom-tool-20190716-140334-1563278614109_21088.fme'
FME Configuration: FME_PRODUCT_NAME is 'FME(R) 2019.0.0.0'
System Status: 436.83 GB of disk space available in the FME temporary folder (C:\Users\agsSA\AppData\Local\Temp)
System Status: 128.00 TB of virtual memory available
Operating System: Microsoft Windows 10 64-bit  (Build 16299)
FME Platform: WIN64
Locale: en
Code Page: 1252  (ANSI - latin I)
FME Configuration: Process limit is 16.00 GB of physical memory
FME Configuration: Start freeing memory when process usage exceeds 48.00 GB of virtual memory
FME Configuration: Stop freeing memory when process usage is below 36.00 GB of virtual memory
Creating writer for format: NULL (Nothing)
Trying to find a DYNAMIC plugin for writer named `NULL'
Loaded module 'NULL' from file 'C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\plugins/NULL.dll'
FME API version of module 'NULL' matches current internal version (3.8 20181030)
Creating reader for format:
MULTI_READER(MULTI_READER): Will fail with first member reader failure
Using Multi Reader with keyword `MULTI_READER' to read multiple datasets
FME API version of module 'NULL' matches current internal version (3.8 20181030)
FME API version of module 'NULL' matches current internal version (3.8 20181030)
Loaded module 'Geometry_func' from file 'C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\plugins/Geometry_func.dll'
FME API version of module 'Geometry_func' matches current internal version (3.8 20181030)
Loaded module 'PythonFactory' from file 'C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\plugins/PythonFactory.dll'
FME API version of module 'PythonFactory' matches current internal version (3.8 20181030)
The workspace does not contain user Python. FME will attempt to use a Python 3.7+ interpreter
Using Python interpreter from `C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\fmepython37\python37.dll' with PYTHONHOME `C:\Program Files\ArcGIS\Data Interoperability for ArcGIS Pro\fmepython37'
Python version 3.7 loaded successfully
Python Exception <AssertionError>: SRE module mismatch
Failed to import `fmesite'
Python Exception <AssertionError>: SRE module mismatch
Failed to import `fme'
 (PythonFactory): The Python Framework could not be started. PythonFactory could not initialize
The Python Framework could not be started. PythonFactory could not initialize
A fatal error has occurred. Check the logfile above for details
Translation FAILED with 7 error(s) and 0 warning(s) (0 feature(s) output)
Translation FAILED with 7 error(s) and 0 warning(s) (0 feature(s) output)
FME Session Duration: 1.2 seconds. (CPU: 0.1s user, 0.2s system)
END - ProcessID: 31608, peak process memory usage: 76412 kB, current process memory usage: 76412 kB
A fatal error has occurred. Check the logfile above for details
Program Terminating
Translation FAILED.
Tool execution failed.
Failed to execute (emailer).
Failed at 16. juli 2019 14:03:40 (Elapsed Time: 6,79 seconds)

Why is Data Interoperability trying to using a python v. 3.7 and not v. 3.6 ?

I've tried to change the interpreter in FME to C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\python.exe so that i's using ArcGIS Pro's python, but that didn't help.

 

Can anyone help, my last solution is to downgrade to ArcGIS Pro 2.3.3 again

Outcomes