Data Exchange Between ArcPy and ArcGIS API for Python

[JoshuaBixby]

As much as one-API-to-rule-them-all just doesn't work, introducing multiple Python-based APIs for ArcGIS raises some questions.  One question that has been nagging me is, "what is the preferred, or better yet optimal, way of transferring geometry objects and record/feature sets between ArcPy and ArcGIS API for Python?"  For me, dumping the data to intermediate file formats isn't efficient, so I am not looking for that to be the answer.

[simoxu]

This is exactly the same question I came here to post --- luckily I did a search first and found someone has already asked this question. Surely lots of developers will face this problem.

I use "ArcPy + ArcGIS API for Python" to connect my Web GIS and local enterprise GIS database and GIS tools, to automate my workflow. I will need to do some data exchange between two python data models (feature, featureset, table, geometry, etc.).

I can't help thinking the following questions:

Why ESRI doesn't provide some helper modules to in either or both API to facilitate the data exchange?

Why a separate Python API from ArcPy? will it be more logical to expand existing ArcPy to support the  online GIS (ArcgIS Online and Portal)?

[JoshuaBixby]

It is sad, in a pitiful kind of way, that no one from the ArcGIS API for Python team ever responded to this question, but I can't say I am surprised either.

When I asked this question 8+ months ago, the API was back on version 1.0.1.  Although there has been quite a bit of change from version 1.0.1 to 1.4, I would argue there hasn't been a whole lot of improvement on this front.  In fact, I think the ArcGIS API for Python has made matters worse by introducing objects with the same name as ArcPy objects that are not compatible with ArcPy.  The most egregious example is FeatureSet.

So, what options are available?  Here are some ideas (not necessarily exhaustive) for interchanging data between ArcGIS API for Python and ArcPy.

• SpatialDataFrame
  • from_featureclass (not quite directly from ArcPy but closer than from a web service)
  • to_featureclass (not quite directly from ArcPy but closer than from a web service)
  • as_arcpy (sounds promising, but only exports geometries and not records)

(I know there are a couple others, but I am drawing a blank at the moment).

[simoxu]

As we all know:

ArcGIS API for Python is a Pythonic wrapper for ArcGIS REST API which is specifically designed for Web GIS.

ArcPy, I guess, is more a wrapper around ArcObjects which is underpinning ArcGIS Desktop GIS functions.

These two sets of Python APIs give us different level of access to our GIS assets using python, I view them supplementary to each other.

It makes sense to use common data model. or at least provide some utilities to facilitate data model exchange.

Let's see what the others think.

[RohitSingh2]

You can load feature data (from Web GIS or local file system) in SpatialDataFrame and use it for feature and geometry manipulation. It can then be saved to feature classes (shapefiles or fgdb...) for use with ArcPy. 

If you wish to do fine grained geometry manipulation you can do it using methods on the SpatialDataFrame as well. Additionally, you can use as_arcpy to get ArcPy Geometries from a SpatialDataFrame. Geometry manipulation using the Geometry utility service (arcgis.geometry) is another option. The other option is to get the json representation and use it to construct arcpy geometries and do those operations locally.

If these options do not cover the range of data exchange operations you want, please post what additional types of data exchange you need.

[Arne_Gelfert]

Great thread... but I'm surprised it's been quiet here for almost a year. I'm a happy camper using the arcgis module (ArcGIS API for Python) for Portal administration - yes, it's a web API - and arcpy for more data-centric automation and processing. arcgis helps with managing users and content, arcpy lets me transform feature, feature classes, fields, etc.

Where I'm struggling right now is in the space in between (or is it overlap?) when trying to publish data to server. As a heavy Jupyter Notebook user, I have both arcgis and arcpy installed in Python 3.6.5 (Anaconda) but the arcpy is different from what it's available in the laughable Python 2.7.13 that still gets installed under C:\Python27 in Windows when you install Desktop. The two flavors of arcpy are very different and most of the documentation is for the older version, some documentation doesn't seem to support either version. Where do I find some of the functionality from old arcpy in new arcpy? I asked a related question here when I ran into issues publishing to AGS from Python.

Short version for the TLDR crowd:

Any recipes for success for leveraging the full power of both APIs in Jupyter?