Trouble using addFeatures endpoint using Python (requests)

GFlonk · ‎03-08-2019

Encountered trouble today trying to query a hosted feature service and sending the results as new features to an identical (same fields/configuration) hosted feature service using Python (requests). For the sake of this example, I reduced my code that produces the error to the example below.

import json

url = r"https://services9.arcgis.com/ty0YDnowzprP81CR/ArcGIS/rest/services/testlayer/FeatureServer/0"
url_query = r"https://services9.arcgis.com/ty0YDnowzprP81CR/ArcGIS/rest/services/testlayer/FeatureServer/0/query"
url_token = r"https://www.arcgis.com/sharing/generateToken"
referer = r"https://www.arcgis.com/"

# query from featureservice 1
token_endpoint = r"https://www.arcgis.com/sharing/generateToken"
querystring = {"f": "pjson",
               "client": "referer",
               "referer": referer,
               "expiration": "60",
               "username": "<username>",
               "password": "<password>"}

response = requests.post(token_endpoint, data=querystring).json()
token = response["token"]

feature_querystring = {"where": "1=1",
                        "objectIds": "",
                        "outFields": "*",
                        "returnGeometry": "true",
                        "f": "json",
                        "token": str(token)}

response_featuredata = requests.request("GET", url_query, params=feature_querystring)
response_featuredata_dict = response_featuredata.json()
features = response_featuredata_dict["features"]

# copy to featureservice 2
add_features_url = r"https://services9.arcgis.com/ty0YDnowzprP81CR/arcgis/rest/services/testlayer2/FeatureServer/0/addFeatures"

params = {
    "features": json.dumps(features),
    "token": token,
    "f": "pjson"
}

response = requests.post(url=add_features_url, params=params)

print(response)
print(response.text)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The above code results in the following repsonse:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>404 - File or directory not found.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;} 
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;} 
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;} 
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
-->
</style>
</head>
<body>
<div id="header"><h1>Server Error</h1></div>
<div id="content">
 <div class="content-container"><fieldset>
  <h2>404 - File or directory not found.</h2>
  <h3>The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.</h3>
 </fieldset></div>
</div>
</body>
</html>

‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Which is weird, because when I try to manually insert the features into the addFeatures-endpoint via the browser, everything works just fine.

Something else I noticed; when I change:

params = {
    "features": json.dumps(features),
    "token": token,
    "f": "pjson"
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

into:

params = {
    "features": json.dumps(features[0]),
    "token": token,
    "f": "pjson"
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The code works fine (but only sends the first of the list of features to the endpoint). Does anyone have an idea about what i'm doing wrong? I'm guessing it has something to do with the way the dict/json is encoded before sending it.

Thanks in advance

EarlMedina · ‎03-08-2019

Hi there. It would be helpful to know what the value is for:

features = response_featuredata_dict["features"]

This will work:

import requests, json

payload = {'features': '[{"geometry":{"x":-9969835.864784643,"y":6707017.275694676,"spatialReference":{"wkid":102100}},"attributes":{"phone":null,"email":null,"note":null,"link":null,"name":null}}]','f':'json'}
resp = requests.post('https://sampleserver6.arcgisonline.com/arcgis/rest/services/Notes/FeatureServer/0/addFeatures', payload)

data=resp.json()
print(data)‍‍‍‍‍‍‍‍‍‍‍‍‍‍

But seems like your query would return something like this:

[   {    "attributes": {     "placename": "Tom's Toys"    },    "geometry": {     "x": 1036256.7442350946,     "y": 1860503.3023153618    }   }
]


You would need to be supplying this for addFeatures to work in my example:

[{"attributes":{"phone":null,"email":null,"note":null,"link":null,"name":null}},{"geometry":{"x":-9969835.864784643,"y":6707017.275694676,"spatialReference":{"wkid":102100}}}]

So, maybe you're missing some braces around the geometry.

GercoFlonk · ‎03-12-2019

Hi Earl, thanks for your reply.

I pasted the value for features = response_featuredata_dict["features"] below. As you can see it is a list with dictionaries describing features which have the two components that are needed to add a feature; "attributes" and "geometry". As far as I can see this structure follows the same structure that is required for the addFeatures-endpoint to work (Add Features—ArcGIS REST API: Services Directory | ArcGIS for Developers ). I think the problem lies somehwere in the way the url-request is encoded, but I don't understand exactly how/why. To clarify: if I write the variable features = response_featuredata_dict["features"] to a json-file > open it > paste the json manually on the addFeatures-endpoint, everything works just fine (I also tried removing the esri-reserved fields objectid and shape__length from the dict, but same result).

[
    {
      "attributes" : {
        "OBJECTID" : 1, 
        "testveld1" : null, 
        "testveld2" : null, 
        "testveld3" : null, 
        "testveld4" : null, 
        "testveld5" : null, 
        "Shape__Length" : 0.233080015281243
      }, 
      "geometry" : 
      {
        "paths" : 
        [
          [
            [5.75271606445174, 53.1577808372716], 
            [5.98068237304519, 53.2063365338456]
          ]
        ]
      }
    }, 
    {
      "attributes" : {
        "OBJECTID" : 2, 
        "testveld1" : "pioh", 
        "testveld2" : 3, 
        "testveld3" : null, 
        "testveld4" : 1551774600000, 
        "testveld5" : "domein1", 
        "Shape__Length" : 0.0814480403011684
      }, 
      "geometry" : 
      {
        "paths" : 
        [
          [
            [6.26358032226356, 53.2642844121363], 
            [6.2779998779276, 53.1841229520489]
          ]
        ]
      }
    }, 
    {
      "attributes" : {
        "OBJECTID" : 3, 
        "testveld1" : null, 
        "testveld2" : null, 
        "testveld3" : null, 
        "testveld4" : null, 
        "testveld5" : null, 
        "Shape__Length" : 0.221553255888095
      }, 
      "geometry" : 
      {
        "paths" : 
        [
          [
            [5.73348999023302, 53.0216975033831], 
            [5.94703674316242, 53.0807197831419]
          ]
        ]
      }
    }, 
    {
      "attributes" : {
        "OBJECTID" : 4, 
        "testveld1" : null, 
        "testveld2" : null, 
        "testveld3" : null, 
        "testveld4" : null, 
        "testveld5" : null, 
        "Shape__Length" : 1.0799569282187
      }, 
      "geometry" : 
      {
        "paths" : 
        [
          [
            [5.75340270995956, 53.2601772176945], 
            [5.63873291015502, 53.2679805498839], 
            [5.73692321777208, 53.2273054601658], 
            [5.55976867675669, 53.2342928214055], 
            [5.60028076171757, 53.1437800104326], 
            [5.67375183105341, 53.1717770993397], 
            [5.51582336425675, 53.1989332854727], 
            [5.61882019042849, 53.2207280784468], 
            [5.62156677245973, 53.2437444951522], 
            [5.71769714355335, 53.2782459228982], 
            [5.82550048827977, 53.2449771682925]
          ]
        ]
      }
    }
  ]‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I shared the featureservices in my example code with everyone. If it helps in your reply/tests; feel free to use them.

EarlMedina · ‎03-12-2019

Hi Gerco,

I don't run into any issues running the code - it grabs 4 features from the one layer and adds them to the second. For reference, I ran this code with the token part removed as you shared the services publicly:

import json, requests

url = r"https://services9.arcgis.com/ty0YDnowzprP81CR/ArcGIS/rest/services/testlayer/FeatureServer/0"
url_query = r"https://services9.arcgis.com/ty0YDnowzprP81CR/ArcGIS/rest/services/testlayer/FeatureServer/0/query"


feature_querystring = {"where": "1=1",
                        "objectIds": "",
                        "outFields": "*",
                        "returnGeometry": "true",
                        "f": "json"}

response_featuredata = requests.request("GET", url_query, params=feature_querystring)
response_featuredata_dict = response_featuredata.json()
features = response_featuredata_dict["features"]

# copy to featureservice 2
add_features_url = r"https://services9.arcgis.com/ty0YDnowzprP81CR/arcgis/rest/services/testlayer2/FeatureServer/0/addFeatures"

params = {
    "features": json.dumps(features),
    "f": "pjson"
}

response = requests.post(url=add_features_url, params=params)

print(response)
print(response.text)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

GFlonk · ‎03-26-2019

I tried your script. It worked on the four features that where in the feature layer. I tested it a little bit extra by adding a few features and trying it again. Now when the service has ~10 featueres I get the same error again (I changed nothing in the code. It is the exact same script, but different amount of features). I run into this problem outside this example aswell in different projects.

vpapali · ‎11-16-2021

I just wanted to say thanks for sharing your code. It gave me the direction that I need to go in adding a point to a Feature Layer. Do you know how to do the same using the ArcGIS python api? I have combed through multiple help pages but can't seem to get a simple how-to. I am just trying to add a point on a map with a hand full of attributes.

Freygeospatial

Also having this issue. Can’t load more than 5 features to an AGOL feature service layer at a time. Works with the browser, but not with Python code. Posting more than five features in one request yields a 404 error, which is a weird status code to get back in response. I definitely shouldn’t be hitting any size limit as the features are quite small and don’t contain geometries. 5 records or less posts just fine.

EDIT: I was able to resolve this by sending the features as a payload instead of request/uri parameters. The documentation needs to be made more clear as it insinuates request parameters must be used, which is wrong.