I am trying to append features to an AGOL HFL from a Geojson file with ArcGIS API for Python, but it is very slow. I do not know if there is some sort of size limit for this procedure, as I am attempting to append 14K point features with about 20 columns to the AGOL HFL.
Has anyone attempted such an operation with success?
Any help or hints are greatly appreciated. Thanks in advance.
Solved! Go to Solution.
Hello @AustinAverill !! I have a very large point file that ive been trying to test around with appending some of those features into a different layer as some what of a backup/storage, we're trying to automate and append of those features nightly into an agol notebook, i was wondering if you were accomplishing something similar and if so, how many of those features are you able to append? mine seems to cap at 28800 and i get a json decode error if i try to do any more than that. are you able to accomplish something similar in your environment
Yes. I typically handle these workflows using a truncate and append approach, handling the geojson file to AGOL as an individual upload then using the hosted geojson file as my append data source. That code typically looks something like the following (Please note I am quickly adapting this from some classes I have written to handle this, so some of the syntax is going to look a bit odd):
from arcgis.gis import GIS
from arcgis.gis import ItemProperties, ItemTypeEnum
def upload_geojson(gis, filepath: str, itemname: str) -> str:
if ".geojson" not in filepath:
raise TypeError("Filepath does not indicate a .geojson format")
folder = gis.content.folders.get()
upProps = ItemProperties(title=f"{itemname}",
description=f"Temp Upload for {itemname} append",
item_type=ItemTypeEnum.GEOJSON)
item_upload = folder.add(item_properties=upProps,
file=filepath).result()
return item_upload.id
def clear_layer(hosted_fl: FeatureLayer) -> dict:
try:
hosted_fl.manager.truncate()
output = {
"status" : "success"
}
return output
except Exception as e:
output = {
"status" : "error",
"error" : str(e)
}
return output
def append_data(hosted_fl: FeatureLayer, upload_itemid: str) -> dict:
try:
hosted_fl.manager.append(upload_itemid,
"geojson")
output = {
"status" : "success"
}
return output
except Exception as e:
output = {
"status" : "error",
"error" : str(e)
}
return output
gis = GIS()
geojson = "path/to/your/file.geojson"
hosted_fl = gis.content.get("itemid").layers[0] #or whatever ordinal position the layer is
upload_itemid = upload_geojson(gis, geojson, "temp_upload")
truncate_result = clear_layer(hosted_fl)
if truncate_result["status"] == "success":
append_result = append_data(hosted_fl, upload_itemid)
print(append_result)
if append_result["status"] == "success":
#clear the geojson item to reduce storage costs
temporary_item = gis.content.get(upload_itemid)
temporary_item.delete()
@MichaelVolz yes, you can use the Delete Features tool, however you should note that the OBJECTIDs will not reset with this tool. This tool is pretty fast, but not as fast as truncate.