I need to automate uploading a ~35,000 record json file to portal. The method I'm using shows the points and record count on the portal, but the attribute table is empty. I'm using the
fl.edit_features(adds="")
method. I started trying to chunk the json into smaller parts b/c I thought that would be useful, but it's the same result, an empty table. In the code below I'm using a chunk size of 100 and limiting to 500 rows for quicker testing. Does anyone know what I can try to get the table to display its data?
localJSON = "E:/batch/AddressRematch/data.json"
with open(localJSON, "r") as jsonfile:
all_data = jsonfile.read()
# load json data to a dict
entire_featureset_dict = json.loads(all_data)
# convert to a list
features_list = entire_featureset_dict["features"]
base_featureset_dict = entire_featureset_dict.copy()
# create a chunk
start = 0
chunk_size = 100
end = start + chunk_size
#breakpoint()
while start < 500: #len(features_list):
# grab a chunk
# copy a slice of the feature set
base_featureset_dict["features"] = features_list[start:end]
chunk_fs = arcgis.features.FeatureSet.from_dict(base_featureset_dict)
#print(chunk_fs)
# upload it to the layer
fl.edit_features(adds=chunk_fs)
#print(arcpy.GetMessages(0))
start += chunk_size
end += chunk_size
Can you print off a subset of the features list? There really shouldn't be a problem with doing this this way, but the FeatureSet.from_dict may be expecting a specific format. Or try printing chunk_fs, does that register as a valid FeatureSet object?
Another approach might be to read the file into a pandas dataframe, then use df.spatial.to_featureset() to create your FeatureSet object. But I don't think it's necessary, we have scripts that build and apply edits using JSON and the requests module.
Ok, I'll try and work with a df. Printing the chunk_fs returns valid records, but that might not be what edit_features() is looking for, you're right.
The fact that the attribute table is empty suggests the issue is how the fields are being given. It should follow the REST API spec:
[
{
"attributes": {
"field1": "value",
"field2": 12
},
"geometry": {
…
}
},
{
// and so on
}
]
Make sure your non-spatial attributes end up in their own dict, rather than being at the same level as the geometry key.