topic Re: Upload a csv file with poly data and generate points (centroids) in ArcGIS API for Python Questions

Upload a csv file with poly data and generate points (centroids)

DonaldPike — Fri, 18 Aug 2023 17:21:48 GMT

I have a python script that performs the following:
1) Exports csv data from a feature layer (that contains polygons and centroids that were generated from the polygons)
2) Performs some external processing on the data (this performs some logic joining original and related data that the internal feature for joining tables in arcgis can't handle, hence the need to do it externally)
3) Imports the processed csv data back into arcgis, creates a feature layer from the data, and associates the feature layer with a webmap

I'm running into an issue with step #3, where I need to recreate the centroids from the uploaded csv data, and I'm hoping I can do this in an automated fashion within the same python script.

Here's the pertinent section of the code (stripped down for readability):

def generate_points_2(params): gis = GIS("https://fotnf.maps.arcgis.com/", "XXXX", "YYYYY") print(f"Connected to {gis.properties.portalHostname} as {gis.users.me.username}") arcpy.AddMessage(f"Connected to {gis.properties.portalHostname} as {gis.users.me.username}") snake_case_random = uuid.uuid1() csv_file_orig = args.data_path / f"Joined.csv" csv_file_new = args.data_path / f"Joined_{snake_case_random}.csv" shutil.copy(csv_file_orig.resolve(), csv_file_new.resolve()) csv_agol = gis.content.add(item_properties={ 'title': f"Test CSV polys {snake_case_random}", }, data=csv_file_new.as_posix()) csv_agol.move(f'Oliver') feature_layer = csv_agol.publish() sr = 4326 # or whatever your spatial reference is feature_layer = FeatureLayer(feature_layer['url'] + '/0') # query features to dataframe df = feature_layer.query( as_df=True, return_centroid=True, return_geometry=False, out_sr=sr ) arcpy.AddMessage("Here 61") # convert centroid to dict # ordinarily I don't like long single lines, but there's no point in assigning all the intermediate outputs to their own variables sdf = GeoAccessor.from_xy(df.join(pd.DataFrame(df.loc[:, 'centroid'].to_dict()).T), 'x', 'y', sr=sr)

But this line:

df = feature_layer.query( as_df=True, return_centroid=True, return_geometry=False, out_sr=sr )

Generates the following error:
'returnCentroid' parameter is invalid
Return geometry centroid is only supported on layer with polygon geometry type.
(Error Code: 400)

I found the following suggesting for what appears to be a similar issue:
https://community.esri.com/t5/data-management-questions/import-csv-data-and-convert-to-polygon-shapefile/m-p/234948/highlight/true#M13336

But that thread is quite old (2015) so I'm wondering if that would still be the right direction to take or if there would be another, possibly simpler, approach?

Re: Upload a csv file with poly data and generate points (centroids)

David_McRitchie — Mon, 21 Aug 2023 16:15:50 GMT

Hey Donald,

Reading this I think the thread you linked will be the course to take. From your code you are loading the csv file holding your polygons into the query and so this expects a Feature Layer to which it can query the geometry.

As a result you will need to have your CSV converted into a Feature Class first before querying it.

Hope that helps,

David

Re: Upload a csv file with poly data and generate points (centroids)

Clubdebambos — Tue, 22 Aug 2023 13:15:14 GMT

Hi @DonaldPike

Check out this link

When running....

feature_layer = csv_agol.publish()

.... use publish parameters as per below for example (see link above for params, scroll down to CSV publish parameters JSON properties)...

publish_parameters = { "type" : "csv", "name" : "TABLE NAME", "locationType" : "coordinates", "latitudeFieldName" : "LAT", "longitudeFieldName" : "LONG" } feature_layer = csv_agol.publish(publish_parameters=publish_parameters)

This will create a point feature layer in a feature service.

Re: Upload a csv file with poly data and generate points (centroids)

DonaldPike — Thu, 07 Sep 2023 22:10:15 GMT

Here's the code that we ended up using to generate the centroids:

(inspired by this post: https://community.esri.com/t5/data-management-questions/import-csv-data-and-convert-to-polygon-shape... )

def generate_points(_args): gis = _args.gis poly_feature_layer = gis.content.get("7da90f1a777c4d6e9d59e3314e70fbe5") arcpy.AddMessage(poly_feature_layer["url"]) poly_feature_layer2 = FeatureLayer(poly_feature_layer["url"] + "/0") # EPSG 4326 is a coordinate system for lat/long that makes used of the WGS84 # spheroid sr = 4326 df = poly_feature_layer2.query( as_df=True, return_centroid=True, return_geometry=False, out_sr=sr ) arcpy.AddMessage("Here 61") arcpy.AddMessage(df.iloc[0]) # Convert centroid to dict sdf = GeoAccessor.from_xy( df.join(pd.DataFrame(df.loc[:, "centroid"].to_dict()).T), "x", "y", sr=sr, ) arcpy.AddMessage("Here 62") arcpy.AddMessage(sdf.iloc[0]) sdf = sdf.drop("Correct_Area_SqM", axis=1) arcpy.AddMessage("Here 63") arcpy.AddMessage(sdf.iloc[0]) # Create a shortened uuid to avoid name clashes. # We made want to add cleanup for all temporary layers. tmp_uuid = "".join( random.choice(string.ascii_letters + string.digits) for _ in range(6) ) arcpy.AddMessage(tmp_uuid) new_feature_layer = sdf.spatial.to_featurelayer( title=f"Polys Centroids {tmp_uuid}", gis=gis, tags=["centroids", "outputs"], folder="Oliver", sanitize_columns=False, service_name=f"polys_centroids_{tmp_uuid}", ) return new_feature_layer