Overwrite Feature Service Error

Question

I am having a reoccurring error with the Spark DataFrameWriter on overwrite mode. I am attempting to overwrite a data layer with ~50,000 records. When I first run the cell, the Feature Service is truncated but no new records are loaded. If I re-run the same cell, the records are loaded to the Feature Service. Has anyone else experienced this error?

NoahSlocum · Answer

Hi Claire, 
We are still investigating the behavior you're seeing, but as a workaround for now could you try the append mode with the truncate option set to True? This would be a good workaround if you don't have any schema changes in your layer. For example:

featureServiceDf.write.format("feature-service")\
     .option("gis", "myGIS")\
     .mode("append")\
     .option("serviceUrl", myURL)\
     .option("layerId", 0)\
     .option("truncate", True)\
     .save()

I'm hoping this will work as expected and not require running the same cell twice to update the layer.

sbattersby · Answer

Hi Claire - we are working on reproducing this internally to see if we can figure out what might be happening.  Just for a few more details for our testing - is your ~50,000 record dataset points, lines, polygons?  If lines or polygons, are they really complex geometries (i.e., lots of vertices) or relatively simple?

Thank you, and I'll let you know if we can recreate this and get some ideas on the issue / how to address it.

-Sarah.

ClaireSullivan · Answer

So far, I have only been using one dataset. There is no error, the run ends successfully but my dataset in AGOL is truncated instead of the new dataset being uploaded. When I run the cell again, the records are updated in AGOL. I am using this method to update my feature service:

## upload feature servicefeatureServiceDf.write.format("feature-service")\    .option("gis", "myGIS")\    .mode("overwrite")\    .option("serviceUrl", myURL)\    .option("layerId", 0)\    .save()