topic Re: Automate overwrite web layer, feature class in Python Questions

Automate overwrite web layer, feature class

EinarDørheim1 — Thu, 14 Jun 2018 07:53:26 GMT

I'm trying to automate an update process for one of my hosted features in portal. So far I have:

1. Solved the automatic download and adjustment of the file in question. (using powershell/task scheduler)

2. exportet the python code that runs the "Geocode Addresses" tool on my file, saves it in a file gdb.

3. manually published the feature class with the "overwrite web layer" function.

My challange is to automate step 3, and connect it to step 2.

Please keep in mind that I'm completely new to scripting/python.

I'm currently using ArcGIS Pro 2.1.0, and have admin rights on the portal.

I have looked at thise sites, and think they describe partly how to solve my issue, I'm just not able to pick out the relevant parts and build a script that works for my particular problem..

community.esri.com "using python to overwrite a feature layer"

developers.arcgis.com python, overwriting feature layers

esri updating-your-hosted-feature-services-with-arcgis-pro-and-the-arcgis-api-for-python

Re: Automate overwrite web layer, feature class

JakeSkinner — Fri, 10 Dec 2021 20:48:05 GMT

Hi Einar,

I recently implemented the script in your 3rd option above. I found this easiest. Save your layer in a Pro project and then you can run the below:

import arcpy
import os, sys
from arcgis.gis import GIS

### Start setting variables
# Set the path to the project
prjPath = r"C:\PROJECTS\NightlyUpdates\NightlyUpdates.aprx"

# Update the following variables to match:
# Feature service/SD name in arcgis.com, user/password of the owner account
sd_fs_name = "MyPublicMap"
portal = "http://www.arcgis.com" # Can also reference a local portal
user = "UserName"
password = "p@sswOrd"

# Set sharing options
shrOrg = True
shrEveryone = False
shrGroups = ""

### End setting variables

# Local paths to create temporary content
relPath = os.path.dirname(prjPath)
sddraft = os.path.join(relPath, "WebUpdate.sddraft")
sd = os.path.join(relPath, "WebUpdate.sd")

# Create a new SDDraft and stage to SD
print("Creating SD file")
arcpy.env.overwriteOutput = True
prj = arcpy.mp.ArcGISProject(prjPath)
mp = prj.listMaps()[0]
arcpy.mp.CreateWebLayerSDDraft(mp, sddraft, sd_fs_name, ‘MY_HOSTED_SERVICES’, ‘FEATURE_ACCESS’,", True, True)
arcpy.StageService_server(sddraft, sd)

print("Connecting to {}".format(portal))
gis = GIS(portal, user, password)

# Find the SD, update it, publish /w overwrite and set sharing and metadata
print("Search for original SD on portal…")
sdItem = gis.content.search("{} AND owner:{}".format(sd_fs_name, user), item_type="Service Definition")[0]
print("Found SD: {}, ID: {} n Uploading and overwriting…".format(sdItem.title, sdItem.id))
sdItem.update(data=sd)
print("Overwriting existing feature service…")
fs = sdItem.publish(overwrite=True)

if shrOrg or shrEveryone or shrGroups:
  print("Setting sharing options…")
  fs.share(org=shrOrg, everyone=shrEveryone, groups=shrGroups)

print("Finished updating: {} – ID: {}".format(fs.title, fs.id))‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Automate overwrite web layer, feature class

EinarDørheim1 — Fri, 10 Dec 2021 20:48:08 GMT

Hi, firstly thansk for the quick and explainatory answer. However I'm getting an error when i try to run the script, not understanding how to solve it:

Any tips?

Creating SD file




--------------------------------------------------------------------------

ExecuteError                              Traceback (most recent call last)
<ipython-input-3-540a66d6dc34> in <module>()
     32 mp = prj.listMaps()[0]
     33 arcpy.mp.CreateWebLayerSDDraft(mp, sddraft, sd_fs_name, 'MY_HOSTED_SERVICES', 'FEATURE_ACCESS','', True, True)
---> 34 arcpy.StageService_server(sddraft, sd)
     35 
     36 print("Connecting to {}".format(portal))

C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\server.py in StageService(in_service_definition_draft, out_service_definition, staging_version)
    846         return retval
    847     except Exception as e:
--> 848         raise e
    849 
    850 @gptooldoc('UploadServiceDefinition_server', None)

C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\server.py in StageService(in_service_definition_draft, out_service_definition, staging_version)
    843     from arcpy.arcobjects.arcobjectconversion import convertArcObjectToPythonObject
    844     try:
--> 845         retval = convertArcObjectToPythonObject(gp.StageService_server(*gp_fixargs((in_service_definition_draft, out_service_definition, staging_version), True)))
    846         return retval
    847     except Exception as e:

C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\geoprocessing\_base.py in <lambda>(*args)
    504         val = getattr(self._gp, attr)
    505         if callable(val):
--> 506             return lambda *args: val(*gp_fixargs(args, True))
    507         else:
    508             return convertArcObjectToPythonObject(val)

ExecuteError: ERROR 999999: Error executing function.
Failed to execute (StageService).

‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Automate overwrite web layer, feature class

JakeSkinner — Fri, 15 Jun 2018 16:56:32 GMT

Looks like it's erroring when trying to create the SD file. Double-check to make sure you have the correct path to your aprx file.

Re: Automate overwrite web layer, feature class

EinarDørheim1 — Wed, 20 Jun 2018 06:15:51 GMT

figured it out, the problem was with:

relPath = sys.path[0]

Changed sys.path[0] to an absolute path and it worked like a charm!

Re: Automate overwrite web layer, feature class

BenBaker1 — Tue, 23 Oct 2018 17:50:33 GMT

Hello, I am getting a different error when I run it in the Python Window in ArcGIS Pro:

### End setting variables

# Local paths to create temporary content
relPath = sys.path[0]
sddraft = os.path.join(relPath, "CitizenConcerns_all.sddraft")
sd = os.path.join(relPath, "CitizenConcerns_all.sd")

# Create a new SDDraft and stage to SD
print("Creating SD file")
arcpy.env.overwriteOutput = True
prj = arcpy.mp.ArcGISProject(prjPath)
mp = prj.listMaps()[0]
arcpy.mp.CreateWebLayerSDDraft(mp, sddraft, sd_fs_name, 'MY_HOSTED_SERVICES', 'FEATURE_ACCESS','', True, True)
arcpy.StageService_server(sddraft, sd)

print("Connecting to {}".format(portal))
gis = GIS(portal, user, password)

# Find the SD, update it, publish /w overwrite and set sharing and metadata
print("Search for original SD on portal…")
sdItem = gis.content.search("{} AND owner:{}".format(sd_fs_name, user), item_type="Service Definition")[0]
print("Found SD: {}, ID: {} n Uploading and overwriting…".format(sdItem.title, sdItem.id))
sdItem.update(data=sd)
print("Overwriting existing feature service…")
fs = sdItem.publish(overwrite=True)

if shrOrg or shrEveryone or shrGroups:
print("Setting sharing options…")
fs.share(org=shrOrg, everyone=shrEveryone, groups=shrGroups)

print("Finished updating: {} – ID: {}".format(fs.title, fs.id))
Creating SD file
Traceback (most recent call last):
File "<string>", line 33, in <module>
File "c:\program files\arcgis pro\Resources\arcpy\arcpy\mp.py", line 105, in CreateWebLayerSDDraft
return _convertArcObjectToPythonObject(arcgisscripting._mapping.CreateWebLayerSDDraft(*_gp_fixargs([map_or_layers, out_sddraft, service_name, server_type, service_type, folder_name, overwrite_existing_service, copy_data_to_server, enable_editing, allow_exporting, enable_sync, summary, tags, description, credits, use_limitations], True)))
RuntimeError

Re: Automate overwrite web layer, feature class

WaiChan — Wed, 05 Jun 2019 19:09:52 GMT

I am using Windows Authentication for the Portal. And it gives the "IndexError: list index out of range" error message when it reaches the line sdItem = gis.content.search("{} AND owner:{}".format(sd_fs_name, user), item_type="Service Definition")[0].

Here are my 4 lines for connecting to the Portal in the beginning of the script.

sd_fs_name = "<feature layer name>"
portal = "<Portal URL>"
user = ""
password = ""

Any help will be greatly appreciated!

Re: Automate overwrite web layer, feature class

WaiChan — Thu, 06 Jun 2019 15:17:46 GMT

Never mind. User error. It would help to have the feature layer already hosted in the Portal. Once the feature layer exists in Portal, the script overwrote the hosted feature layer successfully.

Re: Automate overwrite web layer, feature class

Tweed_ShireAdmin — Mon, 11 Nov 2019 23:05:22 GMT

I am having the same error appear for me when running the script in Python from ArcGIS Pro. Did you get a solution as to why the error was coming and if so what was it.

Any help would be appreciated

Re: Automate overwrite web layer, feature class

JakeSkinner — Thu, 14 Nov 2019 13:55:04 GMT

Tweed Shire Admin‌ can you post the code you are executing?

Re: Automate overwrite web layer, feature class

GlenShepherd — Thu, 05 Dec 2019 02:49:54 GMT

Hi Jake, I'm currently working with Tweed Shire Admin to try and resolve this issue.

Code is as follows:

- User is unable to run a python script that overwrites a feature layer in ArcGIS Online
- Script is fine, and I can run her script (even with her credentials included) from my ArcGIS Pro
- Issue is not specific to her machine
- Complete uninstall and reinstall did not resolve the issue.
- Error message cites Runtime error
- Script tested same problems in Python window of ArcGIS Pro and IDLE
- Replicated on another machine within the network
- Replicated on a laptop outside of the network
- Comprehensive uninstall and reinstallation had no effect
- Client cannot manually overwrite the feature service from Pro either
- Systems check revealed machine environment is fine for Pro
- There are no other feature services published from the same data

Same error received as bbaker_mhgis

Re: Automate overwrite web layer, feature class

JakeSkinner — Thu, 05 Dec 2019 21:07:22 GMT

Hi Glen

Client cannot manually overwrite the feature service from Pro either

If ArcGIS Pro is not working correctly, the code will most likely not work. I would focus on getting Pro working, then circle back to the code.

Re: Automate overwrite web layer, feature class

PatFelser — Mon, 06 Jan 2020 16:45:18 GMT

Hello, I am also receiving the same error. I am not having any issues with Pro however, and can manually update from there just fine. I am currently working out of jupyter notebook if that helps at all.

Re: Automate overwrite web layer, feature class

AmandaBishop2018 — Thu, 06 Feb 2020 15:43:39 GMT

Hello Jake,

Your script is awesome and I have successfully used it to updated a feature layer on my AGOL organization account. The only issue I am facing now is to be able to preserve the "allow export data" option.

Please help!

Amanda

Re: Automate overwrite web layer, feature class

JakeSkinner — Thu, 06 Feb 2020 16:01:57 GMT

Hi Amanda,

Try updating the following line to include the allow_exporting parameter:

arcpy.mp.CreateWebLayerSDDraft(mp, sddraft, sd_fs_name, ‘MY_HOSTED_SERVICES’, ‘FEATURE_ACCESS’,", True, True, allow_exporting=True)‍

Re: Automate overwrite web layer, feature class

AmandaBishop2018 — Thu, 06 Feb 2020 16:10:54 GMT

Worked perfectly! Another questions... since I'm fairly new to automating this task... How would I preserve the thumbnail pic?

Re: Automate overwrite web layer, feature class

AmandaBishop2018 — Thu, 06 Feb 2020 16:13:20 GMT

And also... with the allow export option.. it seems to allow the export but not to a fgdb format?

Re: Automate overwrite web layer, feature class

AmandaBishop2018 — Fri, 07 Feb 2020 17:17:50 GMT

I tried this update script to another copy of our parcel feature service called "TaxParcels". I recieved an error this time.

Re: Automate overwrite web layer, feature class

BenYeager — Tue, 11 Feb 2020 19:10:33 GMT

Hello Glen,

I am receiving the same runtime error above. I have tried all of the items you mentioned. Were you able to resolve this issue?

Thank you!

Ben

Re: Automate overwrite web layer, feature class

GlenShepherd — Tue, 11 Feb 2020 20:57:49 GMT

Hi Ben, yes I was able to figure out the issue.

sys.path[0] is a Python executable that refers to the directory of the script being run. Seeing as we were copying the script into the Python window, it doesn't have a saved directory, so we needed to give it one. i.e. r"D:\BigFolder\LittleFolder"

This is the script I used, modified from the original one in this thread:

Here's the text if you'd like to copy it. Obviously you'll need to update all of your own parameters:

--------------------------------------------------------------------------------------------------------------------------------

import arcpy
import os, sys
from arcgis.gis import GIS

# Start setting variables
# Set the path to the project
prjPath = r"D:\Sample Data\SampleWorking\SampleWorking.aprx"

# Update the following variables to match
sd_fs_name = "ShereeTest"
portal = "https://www.arcgis.com/"
user = "enter_username"
password = "enter_password"

# Set sharing options
shrOrg = True
shrEveryone = True
shrGroups = ""

# End setting variables

# Local paths to create temporary content
relPath = r"C:\GIVE_A_FILE_DIRECTORY"
sddraft = os.path.join(relPath, "WebUpdate.sddraft")
sd = os.path.join(relPath, "WebUpdate.sd")

#Create a new sddraft and stage to sd
print("Creating SD file...")
arcpy.env.overwriteOutput = True
prj = arcpy.mp.ArcGISProject(prjPath)
mp = prj.listMaps()[0]

arcpy.mp.CreateWebLayerSDDraft(map_or_layers=mp,
out_sddraft=sddraft,
service_name=sd_fs_name,
server_type="HOSTING_SERVER",
service_type="FEATURE_ACCESS",
folder_name=None,
overwrite_existing_service=True,
copy_data_to_server=True)

arcpy.StageService_server(in_service_definition_draft=sddraft,
out_service_definition=sd)

print("Connecting to {}".format(portal))
gis = GIS(portal, user, password)

# Find the sd, update it, publish it with overwrite and set sharing and metadata
print("Search for original SD on portal...")
sdItem = gis.content.search("{} AND owner:{}".format(sd_fs_name, user), item_type="Service Definition")[0]
print("Found SD: {}, ID: {} n Uploading and Overwriting...".format(sdItem.title, sdItem.id))
sdItem.update(data=sd)
print("Overwriting existing feature service...")
fs = sdItem.publish(overwrite=True)

if shrOrg or sheEveryone or sheGroups:
print("Setting sharing options...")
fs.share(org=shrOrg, everyone=shrEveryone, groups=shrGroups)

print("Finished updateing: {} - ID: {}".format(fs.title, fs.id))