- Home
- :
- All Communities
- :
- Developers
- :
- ArcGIS API for Python
- :
- ArcGIS API for Python Questions
- :
- JSON to Hosted Feature Service
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
JSON to Hosted Feature Service
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm trying to get an JSON API response with latitude and longitude keys into a Spatial DataFrame so that I can create a hosted feature service from it. Frankly, I'm quite out of practice with Python and I feel like I'm just learning it all over again. I don't want to use ArcPy to accomplish this even though I know it has the JSONToFeatures_conversion tool or in an advanced notebook.
The SpatialDataFrame route is not a requirement. It just seemed the most obvious to me. The goal is to be able to create a new Hosted Feature Service from the JSON Response which looks like:
[{
"Points": [{
"SiteId": 521,
"Latitude": 41.496978,
"Longitude": -95.451919,
"Address1": "10 East Street",
"Address2": null,
"AddressId": 0,
"AddressTypeId": 0,
"City": "Shelby",
"Comments": null,
"County": null,
"Name": "Site 426",
"StoreUrl": "http://myloves426.com",
"SpecialInstructions": null,
"State": "IA",
"Zip": "51570",
"ExitNumber": "34",
"Highway": "I-80",
"PhoneNumber": "(712) 207-2441",
"MapPinId": "e1ff88a9cdba42009a68a6bdbe86b71a",
"MapPinZIndex": 99999999,
"FacilityId": 5712
}, {
"SiteId": 978,
"Latitude": 42.652564,
"Longitude": -88.542025,
"Address1": "100 E Commerce Ct",
"Address2": null,
"AddressId": 0,
"AddressTypeId": 0,
"City": "Elkhorn",
"Comments": null,
"County": null,
"Name": "Site 752",
"StoreUrl": "http://myloves752.com",
"SpecialInstructions": null,
"State": "WI",
"Zip": "53121",
"ExitNumber": "25",
"Highway": "I-43",
"PhoneNumber": "(262) 723-8888",
"MapPinId": "e1ff88a9cdba42009a68a6bdbe86b71a",
"MapPinZIndex": 99999999,
"FacilityId": 6485
}],
"FuelPrices": null,
"StateAbbreviation": null,
"CustomFields": null
}]
So far, I've been able to get it into a standard pandas df using:
import pandas as pd
import requests
import json
points = json.loads(requests.get("https://www.loves.com/api/sitecore/StoreSearch/SearchStores").text)
df = pd.DataFrame(points)
#print(df)However, I don't know where to go from here in terms of turning that response into a hosted feature service based on the latitude and longitude keys given in the response - I do not want to geocode here as I already have the coordinates.
I've also tried using FeatureSet.from_dict method in the Python API, which I presume I should be able to turn into a Hosted Feature Service with relative ease, but I can't get the features loaded into a Featureset from the dictionary for some reason. Here's the code I'm working with currently for that approach:
import requests
import json
import arcgis
from arcgis.gis import GIS
from getpass import getpass
portal = GIS("https://johns.maps.arcgis.com", "john.m.dye", getpass())
portal.users.me
lovesLocator = "https://www.loves.com/api/sitecore/StoreSearch/SearchStores"
response = json.loads(requests.get(lovesLocator).text)[0]
type(response)
# dict
fs = arcgis.features.FeatureSet.from_dict(response)
print(fs)
# {"features": [], "fields": []}[update 2/8/2020]
I was able to solve my initial problem this morning thank's to PerAkeMattias comment on a separate but similar question.
It basically, I needed to just use the SpatialDataFrame.from_xy method to create a Spatial Dataframe, passing in my Latitude and Longitude fields from my pandas DataFrame to get my locations plotted. From there, I expected to be able to use the SpatialDataFrame.to_featurelayer method to publish it, but I'm running into a TypeError and I'm not sure why.
import requests
import json
import arcgis
from arcgis.gis import GIS
from getpass import getpass
import pandas as pd
portal = GIS("https://johns.maps.arcgis.com", "john.m.dye", getpass())
portal.users.me
response = json.loads(requests.get("https://www.loves.com/api/sitecore/StoreSearch/SearchStores").text)[0]["Points"]
df = pd.DataFrame(response)
print(df)
sdf = arcgis.features.SpatialDataFrame.from_xy(df, "Longitude", "Latitude", sr=4326)
print(sdf)
m1 = GIS().map('United States')
m1.zoom = 4
m1.center = [39,-98]
sdf.plot(kind='map', map_widget= m1)
m1
sdf.to_featurelayer(title="lovesTest", gis=portal, tags="test, loves")Unfortunately, I'm seeing a TypeError returned and I'm not sure why:
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-10-bbe2612cd7ab> in <module> ----> 1 sdf.to_featurelayer(title="lovesTest", gis=portal, tags="test, loves") /anaconda3/lib/python3.7/site-packages/arcgis/features/_data/geodataset/geodataframe.py in to_featurelayer(self, title, gis, tags) 1727 raise ValueError("GIS object must be provided") 1728 content = gis.content -> 1729 return content.import_data(self, title=title, tags=tags) 1730 #---------------------------------------------------------------------- 1731 def set_geometry(self, col, drop=False, inplace=False, sr=None): /anaconda3/lib/python3.7/site-packages/arcgis/gis/__init__.py in import_data(self, df, address_fields, **kwargs) 3666 if isinstance(df, SpatialDataFrame) : 3667 ds = df.to_featureclass(out_location=temp_dir, -> 3668 out_name=name) 3669 else: 3670 ds = df.spatial.to_featureclass( /anaconda3/lib/python3.7/site-packages/arcgis/features/_data/geodataset/geodataframe.py in to_featureclass(self, out_location, out_name, overwrite, skip_invalid) 1407 out_location=out_location, 1408 out_name=out_name, -> 1409 overwrite=overwrite, skip_invalid=skip_invalid) 1410 #---------------------------------------------------------------------- 1411 def to_hdf(self, path_or_buf, key, **kwargs): /anaconda3/lib/python3.7/site-packages/arcgis/features/_data/geodataset/io/fileops.py in to_featureclass(df, out_name, out_location, overwrite, out_sr, skip_invalid) 599 return _pyshp_to_shapefile(df=df, 600 out_path=out_location, --> 601 out_name=out_name) 602 else: 603 raise Exception("Cannot Export the data without ArcPy or PyShp modules. "+ \ /anaconda3/lib/python3.7/site-packages/arcgis/features/_data/geodataset/io/fileops.py in _pyshp_to_shapefile(df, out_path, out_name) 93 if idx > -1: 94 geom_type = df.loc[idx][geom_field].type ---> 95 shpfile = shapefile.Writer(GEOMTYPELOOKUP[geom_type]) 96 shpfile.autoBalance = 1 97 row_cols = [] /anaconda3/lib/python3.7/site-packages/shapefile.py in __init__(self, target, shapeType, autoBalance, **kwargs) 1015 self.shp = self.shx = self.dbf = None 1016 if target: -> 1017 self.shp = self.__getFileObj(os.path.splitext(target)[0] + '.shp') 1018 self.shx = self.__getFileObj(os.path.splitext(target)[0] + '.shx') 1019 self.dbf = self.__getFileObj(os.path.splitext(target)[0] + '.dbf') /anaconda3/lib/python3.7/posixpath.py in splitext(p) 120 121 def splitext(p): --> 122 p = os.fspath(p) 123 if isinstance(p, bytes): 124 sep = b'/' TypeError: expected str, bytes or os.PathLike object, not int
Any help is appreciated.
