Using arcpy to copy a portal feature service to a fgdb feature class

5101
14
Jump to solution
01-10-2020 10:07 AM
JoeBorgione
MVP Esteemed Contributor

ArcGIS Pro 2.4.3

In pro, you can add a feature service and manually export the features to a file geodatabase feature class.

How do you mimic that functionality in python?  I have tried:  

import arcpy
source = r'https://path/toOur/rest/services/MapServer/0'

arcpy.FeatureClassToFeatureClass_conversion(source,r'C:\pathTo\Test.gdb','newFC')

and get :

ExecuteError: ERROR 000210: Cannot create output C:\pathTo\Test.gdb\newFC
Failed to execute (FeatureClassToFeatureClass).

I googled the error, and then tried to use the same python approach with an Egdb feature class as the source, and it worked splendidly.  Is this a case for the python api?

That should just about do it....
0 Kudos
1 Solution

Accepted Solutions
JoshuaBixby
MVP Esteemed Contributor

Wow, that is an Esri Support #fail for sure.  That technical article is ancient from a web GIS perspective.  It was "last published" right around the time of the first release of the ArcGIS API for Python, which likely explains why it is not using that API and instead having people make HTTP calls themselves.

Here are some basic steps to download a feature service layer as a feature class in a file geodatabase:

from arcgis.gis import GIS
from arcgis.features import FeatureLayer

url_gis = # URL to AGOL or Portal
url_fl =  # URL for feature layer to download as feature class
user =  # AGOL or Portal username
pwd = # user password
fgdb = # path to file geodatabase
fc = # name of feature class

gis = GIS(url_gis, user, pwd)
fl = FeatureLayer(url_fl)
fs = fl.query()
fs.save(fgdb, fc)

View solution in original post

14 Replies
JoshuaBixby
MVP Esteemed Contributor

Joe, ArcPy isn't really built to directly deal with "web GIS," which is why the ArcGIS API for Python | ArcGIS for Developers was developed.  Use arcgis.features module — arcgis 1.7.0 documentation to retrieve a FeatureLayer of the service layer, then query it to get a FeatureSet that has a save option to file geodatabase.

JoeBorgione
MVP Esteemed Contributor

Mondays project...  Thanks.

That should just about do it....
0 Kudos
JoeBorgione
MVP Esteemed Contributor

Joshua-  esri tech support provided this link to me : How To: Create and download a file geodatabase replica from a hosted feature service using Python .  This particular approach creates a replica and uses userLib2(); as I told them, I'm not interested in a replica and userlib2() is not available in 3.x.  Maybe ESRI can get on board to provide us a more modern approach....

That should just about do it....
0 Kudos
JoshuaBixby
MVP Esteemed Contributor

Wow, that is an Esri Support #fail for sure.  That technical article is ancient from a web GIS perspective.  It was "last published" right around the time of the first release of the ArcGIS API for Python, which likely explains why it is not using that API and instead having people make HTTP calls themselves.

Here are some basic steps to download a feature service layer as a feature class in a file geodatabase:

from arcgis.gis import GIS
from arcgis.features import FeatureLayer

url_gis = # URL to AGOL or Portal
url_fl =  # URL for feature layer to download as feature class
user =  # AGOL or Portal username
pwd = # user password
fgdb = # path to file geodatabase
fc = # name of feature class

gis = GIS(url_gis, user, pwd)
fl = FeatureLayer(url_fl)
fs = fl.query()
fs.save(fgdb, fc)
JoeBorgione
MVP Esteemed Contributor

Thank you so much Joshua. I've been floundering about with Jupyter notebooks for the pas hour or so, but this looks just like what I need.  Locally, we do have a little glitch  with the GIS() method, but I'm working with the S.A to get past that.

That should just about do it....
0 Kudos
dcafdg
by
Occasional Contributor II

Joshua Bixby‌ - Back when I used to work with MapService type services, you had to set a maximum number of features that could be retrieved. The default was 1000 and you could set it higher, but a very large data set could always be more than whatever number you put in there. So a query like this always ran the risk of not getting all the features.

Is that an issue with your code querying FeatureService type services? If so, any suggestions for a work-around?

JoshuaBixby
MVP Esteemed Contributor

David, the ArcGIS API for Python is REST API-aware, so the underlying code addresses the feature limit of services and does the appropriate batching of records to retrieve the entire data set.

SoratoSouza_e_Silva
Occasional Contributor II

This code is very good, but my results is:

Can you help me?

0 Kudos
JoeBorgione
MVP Esteemed Contributor

Joshua- following the instructions given here: Working with different authentication schemes | ArcGIS for Developers  I'm taking a slightly different approach in passing the user, and pwd variables; I need to pass a 'client_id' as well:

gis = GIS(url_gis,user,pwd,client_id = 'Ox05rrnllB1biFgW')

But I get an error:

  File "C:\EnvClones\arcgispro-py3-clone243\lib\site-packages\arcgis\_impl\connection.py", line 449, in oauth_authenticate
    from bs4 import BeautifulSoup

ModuleNotFoundError: No module named 'bs4'‍‍‍‍

Not having a whole lot of luck with dr.google trying to find a remedy for this: it seems as though with the python 3.6 install from Pro, bs4 is no where to be found...

edited to add: actually found a couple threads that suggest using pip3 to install bs4 into python 3.6: here is one from quoraAnaconda offers one as well.  I guess I need to use one of these while in my cloned environment?

That should just about do it....
0 Kudos