That's good to know, especially the link you shared!
I wasn't talking about ethics so much as basic CYA. I've known a few projects that got tanked due to a Data Owner taking issue with how they obtained / used their data, so my default response to getting data in a way that isn't specifically and explicitly permitted is perhaps overly cautious.
Knowing now that this is for academic research, and that you're clearly well aware of your legal "footing", so to speak, I certainly feel more comfortable elaborating.
So, we've got the web map definition JSON. Looking at it, it's clear the data we want is in a series of separate CSV files added to the web map in AGOL, but not published as a standalone service. That's unfortunate, as we can't directly query a service URL for those. But we can still work from the map.
Using the arcgis module in Python, we can pull a list of layers for the map, then interact with each to convert the contents to friendlier formats. The JSON says that these layers support using query, but I can't seem to get it to work. Anyhow, it's less elegant, but this still works:
from arcgis import GIS
from arcgis.features import FeatureSet
from arcgis.mapping import WebMap
gis = GIS()
the_map = WebMap(gis.content.get('df5f3c6ca21a44c5ba7c39d9355ff9dd')
for l in the_map.layers[:-1]:
fs = FeatureSet.from_dict(l['featureCollection']['layers'][0]['featureSet'])
We're still one line short of this code giving you any kind of output, but that depends where you're running your code.
Running in ArcGIS
with access to arcpy, you can call save on a FeatureSet. Simply append "fs.save('your-file-path', l['title'])" inside the for-loop.
Outside of ArcGIS - to JSON / GeoJSON
To JSON / GeoJSON: you can call the property "to_geojson" or "to_json" on the FeatureSet. Writing the output of that to a file could then be used with JSON to Feature Class. Seems redundant, though.
Using a Spatially Enabled DataFrame
I like using DataFrames when I can, but it can be irritating if the data's not just so. "fs.sdf" gets you the FeatureSet as a DataFrame, and from there, you can use the spatial.to_featureclass method. You might need to explicitly tell it what the data types are, though, and that could be tricky inside of a loop if the CSVs don't have identical schemas.
- Josh Carlson
Kendall County GIS