Publish csv to feature using python - Append missing location

RichardSt_John — Sun, 12 Dec 2021 09:34:58 GMT

I am attempting to write a python script to append csv data to a feature service. I manually perform this daily with the online update/append. Using the same csv I wrote a script by following an example from a presentation by Paul Baker on youtube. ArcGIS Online: Managing Data.

I am able to append the data to the Feature Service and all of the fields populate with the proper attributes but it must lack location because I can not zoom to the new features on the map.

I use the analyze function then pass the results to perform the append.

analyze_results = gis.content.analyze(item=append_item.id,
                                     file_type='csv',
                                     #location_type='none')
                                     source_locale='en')

I print the fields names and types in my source file

for csv_field in analyze_results['publishParameters']['layerInfo']['fields']:
print(f"{csv_field['name']:30} {csv_field['type']}")

Blink                          esriFieldTypeInteger 
Element_Name                   esriFieldTypeString 
ACK                            esriFieldTypeInteger 
Upline_Feeder                  esriFieldTypeString 
Upline_Device                  esriFieldTypeString 
UplineRecloser                 esriFieldTypeString 
Phasing                        esriFieldTypeString 
Consumer_Type                  esriFieldTypeString 
Consumer_Bldng_Num             esriFieldTypeString 
Name                           esriFieldTypeString 
Tadd                           esriFieldTypeString 
Y                              esriFieldTypeDouble 
X                              esriFieldTypeDouble 
Date_                          esriFieldTypeDate

The location is in the x and y fields

I can see this by printing the publishParameters

{'type': 'csv', 'name': 'data', 'useBulkInserts': True, 'sourceUrl': '', 'locationType': 'coordinates', 'maxRecordCount': 1000, 'latitudeFieldName': 'Y', 'longitudeFieldName': 'X', 'columnDelimiter': ','

The append code..

try:
    fs_layer.append(item_id=append_item.id,
                   upload_format='csv',
                   source_table_name='',
                   #field_mappings=None,
                   field_mappings=[{"source":"ACK","name":"ACK"},
                                   {"source":"Blink","name":"Blink"},
                                   {"source":"Consumer_Bldng","name":"Consumer_Bldng"},
                                   {"source":"Consumer_Type","name":"Consumer_Type"},
                                   {"source":"Date_","name":"Date_"},
                                   {"source":"Element_Name","name":"Element_Name"},
                                   {"source":"y","name":"y"},
                                   {"source":"x","name":"y"},
                                   {"source":"Meter_Number","name":"Meter_Number"},
                                   {"source":"Name","name":"Name"},
                                   {"source":"Phasing","name":"Phasing"},
                                   {"source":"Tadd","name":"Tadd"},
                                   {"source":"Upline_Device","name":"Upline_Device"},
                                   {"source":"Upline_Feeder","name":"Upline_Feeder"},
                                   {"source":"Upline_Recloser","name":"Upline_Recloser"}],
                                   
                    edits=None,
                    source_info=analyze_results,
                    upsert=False,
                    skip_updates=False,
                    use_globalids=False,
                    update_geometry=False,
                    append_fields=["Blink",
                                   "Element_Name",
                                   "ACK",
                                   "Upline_Feeder",
                                   "Upline_Device",
                                   "UplineRecloser",
                                   "Phasing",
                                   "Consumer_Type",
                                   "Consumer_Bldng_Num",
                                   "Name",
                                   "Tadd",
                                   "Latitude",
                                   "Longitude",
                                   "X",
                                   "Y",
                                   "Date_"],
                    rollback=True)
    print("Append completed successfully")
except:
    print("Something went wrong")

Any help would be appreciated.

Re: Publish csv to feature using python - Append missing location

Arne_Gelfert — Wed, 29 Jul 2020 13:49:01 GMT

Are you mixing lower case "x" and "y" with uppercase "X" and "Y" throughout the code, including in the mapping?

Re: Publish csv to feature using python - Append missing location

RichardSt_John — Wed, 29 Jul 2020 14:19:50 GMT

I believe I tried to standardize on Upper in one of my trials. I will try again. Thanks

Re: Publish csv to feature using python - Append missing location

NicolasMarecos — Sun, 12 Dec 2021 09:35:01 GMT

Hi Richard,

I believe that the problem is that this code is updating attribute fields, a feature in ArcGIS Online is stored in the following format:

{"geometry": {"x": , "y": }, "attributes": {"OBJECTID": , "GlobalID": "", "Field1":, "Field2":}}

Field mapping is going to match the attribute dictionary, but won't add the geometry. This sample notebook goes into adding and updating features from a CSV as well as adding/updating geometry (Cells 18 and 23). The important parts are:

    input_geometry = {'y':float(matching_row['latitude']),                       'x':float(matching_row['longitude'])}

    output_geometry = geometry.project(geometries = [input_geometry],                                       in_sr = 4326,                                         out_sr = cities_fset.spatial_reference['latestWkid'],                                      gis = gis)

    # assign the updated values    new_feature.geometry = output_geometry[0]

I would look at the notebook mentioned above, as it looks like it closely matches your needs.

I hope this helps,

Nico

topic Re: Publish csv to feature using python - Append missing location in ArcGIS API for Python Questions

Publish csv to feature using python - Append missing location

Re: Publish csv to feature using python - Append missing location

Re: Publish csv to feature using python - Append missing location

Re: Publish csv to feature using python - Append missing location