topic Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python in Python Questions

Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

WillemVan_Riet1 — Wed, 06 Dec 2023 22:37:16 GMT

I am trying to download feature service items from ArcGIS online to then clip to my project area, add specific fields using ArcPY and then upload to AGOL overwriting specified layers.

I'm following the instructions as per this tutorial:

https://support.esri.com/en-us/knowledge-base/how-to-download-feature-service-items-from-arcgis-onlin-000018909

# define the download format and the specified output path

# path = my_test_path ## changed for forum post 

def downloadUserItems(owner, downloadFormat):

    try:

        # Search items by username

#         items = gis.content.search('owner:{0}'.format(owner))

#         print(items)

        # search items by group

        group = gis.groups.search("title:ArcGIS API Test", max_groups=15)[0]

        items = group.content()

        print(items)

        # Loop through each item and if equal to Feature service then download it

        for item in items:

            print(item.type)

           # if item.type in ['Feature Service', 'Vector Tile Service', 'Scene Service']:                

            result = item.export('sample {}'.format(item.title), downloadFormat, wait =True)

            result.download(path)

            # Delete the item after it downloads to save space (OPTIONAL)

           # result.delete()

            print("Exported item: {}".format(result))

    except Exception as e:

        print(e)

# Function takes in two parameters. Username and the type of download format

downloadUserItems('dsfgis', downloadFormat='Shapefile')

And getting this error.


[<Item title:"Storm_Feature" type:Feature Layer Collection owner:dsfgis>] 

Feature Service 

Could not export item.

I've gone through the documentation on the Item class. Any ideas/help appreciated.

Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

AntEsk — Thu, 07 Dec 2023 01:55:54 GMT

i think you need to include extra parameters to tell it whihc layer from the feature service you want to put into the shapefile,

/export: Export Item—ArcGIS REST APIs | ArcGIS Developers

i had a lot of trouble trying to download data from a Portal using export(), download()) .

i found it was better to get the rest end point of the layer and use it in Feature Class to Feature class conversion or Feature Class to Geodatabase conversion (to keep domains, attachments etc )

Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

DavidPike — Thu, 07 Dec 2023 20:10:11 GMT

As a feature layer collection I'd guess you have to iterate through the feature layers individually for export and download as shapefiles. I think "Feature Layer" in the search also grabs collections.

#check if FLC - not tested this so not sure if right if item.type == 'Feature Layer Collection': #access layers by .layers property and loop thru for flayer in item.layers: result = item.export(.......

Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

WillemVan_Riet1 — Thu, 14 Dec 2023 19:10:08 GMT

I'm not too familiar with this approach - do you need to have ArcGIS Enterprise?

Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

WillemVan_Riet1 — Thu, 14 Dec 2023 19:16:10 GMT

not getting an error message anymore but the feature layer collection isn't downloading/exporting to my path...

I have a print statement that spits out the item type in line 11 and its reading as a "feature service" even though line 7 ("group.content()" says its type is a "Feature Layer Collection". Your insights are appreciated!

def downloadUserItems(owner, downloadFormat): # try: # search items by group group = gis.groups.search("title:ArcGIS API Test", max_groups=15)[0] items = group.content() print(items) # Loop through each item and if equal to Feature service then download it for item in items: print(item.type) if item.type == 'Feature Layer Collection': for flayer in item.layers: result = item.export('{}'.format(item.title), downloadFormat, wait =True) #esult.download(path) # Delete the item after it downloads to save space (OPTIONAL) # result.delete() print("Exported item: {}".format(result)) # except Exception as e: # print(e) # Function takes in two parameters. Username and the type of download format downloadUserItems('dsfgis', downloadFormat='Shapefile')

Output of downloadUserItems:

Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

AntEsk — Thu, 14 Dec 2023 22:49:36 GMT

Feature Layer, Feature Layer collection and even table are all Feature services, as i mentioned above i had no joy using .download() or . export(),

this is general principle,

gis = GIS( portal connection details)

# make dict of item titles and items,  limit this by only getting the ones that you want
portal_items = {x.title:x for x in gis.content.search(query="", item_type="Feature Layer", max_items=-1)}

# need to define dest_fc
for title, url in portal_items.items():
    arcpy.CopyFeatures_management(url , dest_fc)

# or if you want to keep domain, attachment etc you can set the environmental vairables as you export all
# this will downloas all faeture service, but agina will need tweaking if there is more than one layerin thefeture service
# need to define the out gdb,
fs = "\n".join([name for name, _ in portal_items.items()])
print(f' - Downloading Feature Services \n{fs}')
items_urls = ";".join([f"{item.url}/0" for _, item in portal_items.items()])
with arcpy.EnvManager(transferGDBAttributeProperties="TRANSFER_GDB_ATTRIBUTE_PROPERTIES",
                      transferDomains="TRANSFER_DOMAINS"):
    arcpy.conversion.FeatureClassToGeodatabase(
        Input_Features=items_urls,
        Output_Geodatabase=out_gdb)

Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

AntEsk — Thu, 14 Dec 2023 22:47:06 GMT

no dont need Enterprise can be any portal, once you have the rest end point for the layer you can treat it like a normal feature class, i have given an example below

Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

AntEsk — Thu, 14 Dec 2023 22:56:44 GMT

this is my method for downloading all feature services and tables

tool_fs = [list of feature service names]

tool_tables = [list of table names]

agol_project_layers_dct = get_tool_lyrs(gis, tool_fs)

agol_project_tables_dct = get_tool_lyrs(gis, tool_tables)

def get_tool_lyrs(gis, tool_fs):
    agol_project_layer_info = {x.title: x for x in gis.content.search("", item_type="Feature Layer", max_items=-1)
                               if x.title in tool_fs}

    return agol_project_layer_info

def download_tool_data(dest_fldr, dest_gdb_name, agol_project_layers_dct, agol_project_tables_dct):
    # make output geodatabase
    out_gdb = os.path.join(dest_fldr, dest_gdb_name)
    if not arcpy.Exists(out_gdb):
        print(f'Making output gdb {out_gdb}')
        arcpy.CreateFileGDB_management(dest_fldr, dest_gdb_name)

    print(f"Downloaded Feature Services will be saved to {dest_gdb_name}")

    ## downlaod feature services
    fs = "\n".join([name for name, _ in agol_project_layers_dct.items()])
    print(f' - Downloading Feature Services \n{fs}')
    items_urls = ";".join([f"{item.url}/0" for _, item in agol_project_layers_dct.items()])
    with arcpy.EnvManager(transferGDBAttributeProperties="TRANSFER_GDB_ATTRIBUTE_PROPERTIES",
                          transferDomains="TRANSFER_DOMAINS"):
        arcpy.conversion.FeatureClassToGeodatabase(
            Input_Features=items_urls,
            Output_Geodatabase=out_gdb)

    ## downlaod tables
    tbls = "\n".join([name for name, _ in agol_project_tables_dct.items()])
    print(f' - Downloading Tables \n {tbls}')
    items_urls = ";".join([f"{item.url}/0" for _, item in agol_project_tables_dct.items()])
    arcpy.conversion.TableToGeodatabase(
        Input_Table=items_urls,
        Output_Geodatabase=out_gdb)

Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

WillemVan_Riet1 — Mon, 18 Dec 2023 23:48:09 GMT

appreciate you laying out the code for me, but its breaking on line 14

items_urls = [f"{item.url}/0" for _, item in agol_project_layers_dct.items()]

and reads:

ERROR 000732: Input Features: Dataset https://services.arcgis.com/YCM10gnCAvCoAhpj/arcgis/rest/services/Point_of_Interest/FeatureServer/0/0 does not exist or is not supported

according to that error message, it looks like the url is incorrect - when I click it it leads to a broken link (screenshot attached). However, when I added this ESRI REST layer to my AGOL account and associated group - I use this URL. I am confused why the .url function is taking the URL and adding additional characters to it..

https://services.arcgis.com/YCM10gnCAvCoAhpj/arcgis/rest/services/Point_of_Interest/FeatureServer/0

Re: Error when exporting feature service items from ArcGIS Online using ArcGIS API for Python

AntEsk — Mon, 18 Dec 2023 23:53:05 GMT

Hi,

yes, sorry, i grabbed bits of code and cobbled them together for my answer, you dont need to append the extra '/0'

should be

items_urls = [item.url for _, item in agol_project_layers_dct.items()]