How does Data Interoperability relate to FME?

7211
6
07-26-2022 01:25 PM
BruceHarold
Esri Regular Contributor
2 6 7,211

The Data Interoperability team at Esri get this question a lot, especially at events like the Esri International User Conference.  This article will equip you with background and details we find most helpful in purchasing and usage contexts.

Data Interoperability Powering ProData Interoperability Powering Pro

In the above screen capture you can see Data Interoperability in a few places; the Analysis ribbon provides access to the Workbench, Data Inspector and Quick Translator apps (also available in the ArcGIS program group start menu), the project has custom Spatial ETL tools defined in project toolboxes, and ModelBuilder includes a Spatial ETL tool.  Not shown - the Quick Import and Quick Export system tools delivered with Data Interoperability - but the Interoperability Connection toolbox in the project leverages Quick Import in a model to read any data source into a memory feature layer - you can obtain this toolbox here.

Firstly, ArcGIS Data interoperability is built using FME technology.  Originally, Data Interoperability was conceived as a way for ArcGIS users to take advantage of Safe Software's industry-leading format support, and FME's no-code approach to handling geospatial data at row level granularity, plus of course the transformational ability of FME technology when working with attributes and geometry.  The technology was originally exposed as browsable connections to data and as geoprocessing tools in ArcMap and geoprocessing services in ArcGIS Server.

None of that has gone away, but much new has been added, and how it is exposed has changed.  If the ancient Greeks had invented computers and studied ETL as a discipline, Heraclitus might have said something like "You cannot step twice into the same data flow".

Let's cover format support.  Data Interoperability is built from FME and therefore uses an FME 'engine' of a particular release, just like the FME product itself.  At writing (November 2023) the engine in Data Interoperability is 2023.1.1.1 and in the FME product is 2023.1.2.0.  We build Data Interoperability with the latest engine we can but  FME has a different release cycle and can get ahead.  If you're interested in what the differences between engines are you can go to this page and click the link for the 64bit Windows change log.  How all this relates to formats is that format support is bound to either an engine or an FME Hub package version, and FME packages themselves have minimum engine (and build) requirements.  In addition to engine dependencies Esri and Safe have independent license levels for format inclusion in our products.  Data Interoperability gets every format we can obtain, so is comparable to FME Form; here are all the formats that are in FME Form but not in Data Interoperability for ArcGIS Pro 3.2:

Formats Not in Data Interoperability @ Pro 3.2

APT
Adobe Illustrator - Avenza MAPublisher
Aircom ENTERPRISE Map Data/ASSET Data
Autodesk AutoCAD Map 3D Object Data
Autodesk AutoCAD RealDWG DWG/DXF
Bentley Map XFM Design (V8)
Bentley Pointools POD
Bentley i-model Interchange Format
CARIS NTX
CARIS Spatial Archive (CSAR) Point Cloud
CARIS Spatial Archive (CSAR) Raster
DES
EDIGéO
ER Mapper ECW
FalconView File
GeoConcept Map
Graphic Technologies, Inc. (GTI) GTViewer
Intergraph FRAMME Standard Exchange Format (SEF)
JPEG 2000 (GeoJP2/GMLJP2)*
Marconi PlaNet
Metria AutoKa Transfer File (FF)
Microsoft MapPoint Web XML
Northrop Grumman C2PC Magic (Tech Preview)
OGC CityGML 3 (Tech Preview)
Precisely MapInfo Extended TAB*
Precisely MapInfo TAB (EFAL- Tech Preview)*
Precisely MapInfo TAB (MAPINFO)*
Precisely Multi-Resolution Raster (MRR) (GDAL)
Smallworld 4/5
Tableau Hyper
Tele Atlas MultiNet Interchange format
VoxelGeo OpenInventor (VOIV)

 

* Note that JPEG2000 data is supported with the OpenJPEG2000 reader/writer and MapInfo TAB is supported with the MITAB reader/writer.

Let's cover transformer support.  Below is a list of all transformers in FME Form 2023.1 that are not in Data Interoperability at Pro 3.2.  They are all either 3rd party transformers available for purchase and use in FME - and hence out of scope for Data Interoperability - or specific to FME product architecture.

Transformers in FME not in Data Interoperability @ Pro 3.2

Curvefitter
GtransAttributeReprojector
GtransReprojector
ReframeReprojector
RasterBrainExtractor
FMEServerWorkspaceRunner

 

Why would you buy FME or Data Interoperability over the other?  If the above formats or transformers are relevant to your work then you will need to buy FME.  Otherwise there are other criteria you may balance.

ArcGIS Data Interoperability is sold and supported by Esri.  FME is sold through Safe Software's channel.

Data Interoperability extends ArcGIS Pro and ArcGIS Enterprise; in the case of Pro in the Analysis ribbon and as geoprocessing tools, in the case of Enterprise as web tools (although you can open the Workbench application on an Enterprise machine to do things like import credentials).

It is intended that FME Form and Data Interoperability can be used 'together' in terms of sharing Workbench workspace files but if the products are at different FME engine releases incompatibilities, while rare, can occur and are not bugs.  For example, transformers may fail to 'arrive' in Data Interoperability when opening a workspace authored with FME at a later release because it has a newer version of a transformer.  In this situation re-insert the missing transformer(s) using Data Interoperability with parameter values taken from the FME original workspace.  If this is a common occurrence at your office it can be useful to keep a workspace with any old releases of transformers on your FME machine to copy/paste into workspaces destined for Data Interoperability.

FME Form uses ArcGIS Pro software for Esri binary format support such as geodatabase (enterprise, file, mobile) and Knowledge graph.  An exception is the API-based file geodatabase reader/writer that does not need Pro.  Pro can be licensed in any of the usual modes.  Enable offline use if you will be using standalone Python scripts that call an ETL tool.

FME Flow must use ArcGIS Enterprise software for any Esri binary format support such as enterprise geodatabase (with the same exception as above).  While technically FME Flow can use ArcGIS Pro, the Pro product is licensed as a single user app and cannot be used in a server context.  It is sufficient for ArcGIS Server Basic to be installed on an FME Flow machine in order to support read/write access to enterprise geodatabases.

Can you trade in Data Interoperability for a copy of FME or vice versa?  There is no formal program for any sort of cross grade.  Esri and Safe have tried to make the products interoperable and many sites have both products.

Does Data Interoperability have any FME Flow capability and does FME have any ArcGIS Enterprise capability?  Data Interoperability can be used to publish workspaces to FME Flow.  There is a dependency that a standalone workspace (fmw file) be used and that the workspace cannot have been used as the source for an ArcGIS Pro ETL tool.  Support for the workflow is from Safe Software.  FME has no ability to publish web tools to ArcGIS Enterprise but Data Interoperability can embed an FME workspace file (subject to fixing any incompatibility issues) into an ETL tool and publish it as a web tool.

At writing we can share that FME Form and Flow (along with Data Interoperability) have enhanced support for the EsriJSON format, with a view to using this as a communication format between FME Flow and ArcGIS Enterprise.

We hope the above gives you confidence that Esri and Safe have anticipated customer use of our products in a cooperative manner and with the minimum of friction.  If you have further technical questions please post them in this forum or for business-related questions though your normal account representative.

To obtain a free trial of Data Interoperability extension go to this page.

6 Comments