How does Data Interoperability relate to FME?

12065
6
07-26-2022 01:25 PM
BruceHarold
Esri Regular Contributor
4 6 12.1K

The Big Picture

 

The Data Interoperability team at Esri get this question a lot, especially at events like the Esri International User Conference.  This article will equip you with some background and details we find most helpful in purchasing and usage contexts.

ArcGIS Data Interoperability powering ArcGIS ProArcGIS Data Interoperability powering ArcGIS Pro

In the above screen capture you can see Data Interoperability in a few places; the Analysis ribbon provides access to the Workbench, Data Inspector and Quick Translator apps (also available in the ArcGIS program group start menu), the project has custom Spatial ETL tools defined in a project toolbox, and ModelBuilder is calling a Spatial ETL tool.  Not shown - the Quick Import and Quick Export system tools delivered with the extension in the Data Interoperability Tools system toolbox - these use default settings to translate between any non-raster format and file geodatabase (or other format already understood by ArcGIS) in either direction - very convenient for quick inspection or quick sharing of data.

Now, some details.

Firstly, ArcGIS Data interoperability is built using FME technology.  Originally, Data Interoperability was conceived as a way for ArcGIS users to take advantage of Safe Software's industry-leading format support, and FME's no-code approach to handling geospatial data at row level granularity, plus of course the transformational ability of FME technology when working with attributes and geometry.  The technology was originally exposed as browsable connections to data and as geoprocessing tools in ArcMap and geoprocessing services in ArcGIS Server.

None of that has gone away, but much new has been added, and how it is exposed has changed.  If the ancient Greeks had invented computers and studied ETL as a discipline, Heraclitus might have said something like "You cannot step twice into the same data flow".

Let's cover format support.  Data Interoperability is built from FME and therefore uses an FME 'engine' of a particular release, just like the FME product itself.  At writing (November 6th, 2024) the engine in Data Interoperability is 2024.1.2.1 and in the FME product is 2024.2.0.0.  We build Data Interoperability with the latest engine we can but  FME has a different release cycle and can get ahead.  If you're interested in what the differences between engines are you can go to this page and click the link for the 64bit Windows change log.  How all this relates to formats is that format support is bound to either an engine or an FME Hub package version, and FME packages themselves have minimum engine (and build) requirements.  In addition to engine dependencies Esri and Safe have independent license levels for format inclusion in our products.  Data Interoperability gets every format we can obtain, so is comparable to FME Form; here are all the formats that are in FME Form but not in Data Interoperability for ArcGIS Pro 3.4.

The Format Delta

 

FME Formats Not in Data Interoperability @ Pro 3.4

APT
Adobe Illustrator - Avenza MAPublisher
Aircom ENTERPRISE Map Data/ASSET Data
Autodesk AutoCAD Map 3D Object Data
Autodesk AutoCAD RealDWG DWG/DXF
Bentley Map XFM Design (V8)
Bentley Pointools POD
Bentley i-model Interchange Format
CARIS NTX
CARIS Spatial Archive (CSAR) Point Cloud
CARIS Spatial Archive (CSAR) Raster
DES
EDIGéO
ER Mapper ECW
FalconView File
GeoConcept Map
Graphic Technologies, Inc. (GTI) GTViewer
Intergraph FRAMME Standard Exchange Format (SEF)
JPEG 2000 (GeoJP2/GMLJP2)*
Marconi PlaNet
Metria AutoKa Transfer File (FF)
Microsoft MapPoint Web XML
Northrop Grumman C2PC Magic (Tech Preview)
Precisely MapInfo Extended TAB*
Precisely MapInfo TAB (EFAL- Tech Preview)*
Precisely MapInfo TAB (MAPINFO)*
Precisely Multi-Resolution Raster (MRR) (GDAL)
Smallworld 4/5
Tableau Hyper
Tele Atlas MultiNet Interchange format
VoxelGeo OpenInventor (VOIV)

 

* Note that JPEG2000 data is supported with the OpenJPEG2000 reader/writer and MapInfo TAB is supported with the MITAB reader/writer.

Let's cover transformer support.  Below is a list of all transformers in FME Form 2024.2 that are not in Data Interoperability at Pro 3.4.  They are 3rd party transformers available for purchase and use in FME - and hence out of scope for Data Interoperability.

The Transformer Delta

 

Transformers in FME 2024.2 not in Data Interoperability @ Pro 3.4

Curvefitter
GtransAttributeReprojector
GtransReprojector
ReframeReprojector

 

Other Considerations

 

Why would you buy FME or Data Interoperability over the other?  If the above formats or transformers are critical to your work then you will need to buy FME.  Otherwise there are other criteria you may balance.

ArcGIS Data Interoperability is sold and supported by Esri.  FME is sold through Safe Software's channel.

Data Interoperability extends ArcGIS Pro and ArcGIS Enterprise; in the case of Pro in the Analysis ribbon and as geoprocessing tools, in the case of Enterprise as web tools (although you can use the Workbench application on an Enterprise machine to do things like import credentials).

It is intended that FME Form and Data Interoperability can be used 'together' in terms of sharing Workbench workspace files but if the products are at different FME engine releases incompatibilities, while rare, can occur and are not bugs.  For example, transformers may fail to 'arrive' in Data Interoperability when opening a workspace authored with FME at a later release because it has a newer version of a transformer.  In this situation re-insert the missing transformer(s) using Data Interoperability with parameter values taken from the FME original workspace.  If this is a common occurrence at your office it can be useful to keep a workspace with any old releases of transformers on your FME machine to copy/paste into workspaces destined for Data Interoperability.

FME Form uses ArcGIS Pro software for Esri binary format support such as geodatabase (enterprise, file, mobile) and Knowledge Graph.  An exception is the API-based file geodatabase reader/writer that does not need Pro.  Pro can be licensed in any of the usual modes.  Enable offline use if you will be using standalone Python scripts that call an ETL tool.

FME Flow must use ArcGIS Enterprise software for any Esri binary format support such as file, mobile or enterprise geodatabase (with the above API-based format exception) or Knowledge Graph.  ArcGIS Server Basic is sufficient for enterprise geodatabase support.  ArcGIS Pro is licensed as a single-user app and may not be used on an FME Flow machine to enable Esri formats.  However, FME Flow can write to an enterprise geodatabase without a local ArcGIS Enterprise installation if the the target feature class or table is published by reference from a registered data store and FME Flow uses the Esri ArcGIS Portal Feature Service reader/writer with the resulting feature service.  This approach uses REST calls and not any Esri executable.  Please see Safe's article here, specifically the topic "Connecting to Esri Applications from FME Flow without a license."

Can you trade in Data Interoperability for a copy of FME or vice versa?  There is no formal program for any sort of cross grade.  Esri and Safe have tried to make the products interoperable and many sites have both products.

Does Data Interoperability have any FME Flow capability and does FME have any ArcGIS Enterprise capability?  Data Interoperability can be used to publish workspaces to FME Flow.  There is a dependency that a standalone workspace (fmw file) be used and that the workspace cannot have been used as the source for an ArcGIS Pro ETL tool.  Support for the workflow is from Safe Software.  FME has no ability to publish web tools to ArcGIS Enterprise but Data Interoperability can embed an FME workspace file (subject to fixing any incompatibility issues) into an ETL tool and publish it as a web tool.

We hope the above gives you confidence that Esri and Safe have anticipated customer use of our products in a cooperative manner and with the minimum of friction.  If you have further technical questions please post them in this forum or for business-related questions though your normal account representative.

To obtain a free trial of Data Interoperability extension go to this page.

6 Comments