How to convert a spatially enabled dataframe(sdf) to a gdf

ZacharyKasson · ‎11-18-2024

python: 3.9

arcgis: 2.3.0.1

I am accessing a Feature Layer from the Living Atlas. I convert it to a sdf with the column 'SHAPE' as the geometry. How do I add the sdf to a leafmap map. I have added other layers to the map via add_gdf(). Can I convert the sdf to a gdf? or a geojson which i could then create a gdf from. I found this but need to create the geojson as a variable instead of a local file.

Here is some of the code I am using:

from arcgis.gis import GIS
from arcgis import gis, GeoAccessor, geometry
gis = GIS()

def read_fl(item_id):
    living_atlas_item = gis.content.get(item_id)
    feature_layer = living_atlas_item.layers[0]
    sdf = feature_layer.query(where="1=1", out_sr=4326).sdf
    return sdf

item_id = "21638fcd54d14a25b6f1affdef812146"
sdf = read_fl(item_id)

# Transform sdf to gdf somehow

# Create map
map = leafmap.Map(
    layers_control=True,
    draw_control=False,
    measure_control=False,
    fullscreen_control=False)

map.add_basemap(basemap_selection)
map.add_gdf(
    gdf=sdf,
    zoom_to_layer=False,
    layer_name='Fires',
    info_mode='on_click',
    )

ZacharyKasson · ‎11-19-2024

I found it to be simple than I thought.

gdf = gpd.GeoDataFrame(sdf, geometry='SHAPE')

View solution in original post

jcarlson · ‎11-19-2024

That method assumes you have GeoPandas. While you *can* get your sdf into a GeoDataFrame, there's not a built-in method for it.

It is possible to use requests to query a Feature Service, and for GeoPandas to parse the response.

import geopandas as gp
import requests

layer_params = {
  "where" : "1=1",
  "f" : "geojson"
}

layer_req = requests.post(
  'URL to the feature service',
  layer_params
)

gdf = gp.read_file(layer_req.text)

# then do the rest of it

- Josh Carlson
Kendall County GIS

ZacharyKasson · ‎11-19-2024

Going straight from the Feature Service to the gdf using requests would work. I get a data source error when trying it this way though. Do I need to consider authentication?

DataSourceError                           Traceback (most recent call last)
Cell In[3], line 13
      4 layer_params = {
      5   "where" : "1=1",
      6   "f" : "geojson"
      7 }
      9 layer_req = requests.post(
     10   'https://zklxc2usnyaidrpe.maps.arcgis.com/home/item.html?id=21638fcd54d14a25b6f1affdef812146',
     11   layer_params
     12 )
---> 13 gdf = gpd.read_file(layer_req.text)
     14 gdf

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\geopandas\io\file.py:294, in _read_file(filename, bbox, mask, columns, rows, engine, **kwargs)
    291             from_bytes = True
    293 if engine == "pyogrio":
--> 294     return _read_file_pyogrio(
    295         filename, bbox=bbox, mask=mask, columns=columns, rows=rows, **kwargs
    296     )
    298 elif engine == "fiona":
    299     if pd.api.types.is_file_like(filename):

File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\geopandas\io\file.py:547, in _read_file_pyogrio(path_or_bytes, bbox, mask, rows, **kwargs)
    538     warnings.warn(
...
File pyogrio\\_io.pyx:1240, in pyogrio._io.ogr_read()

File pyogrio\\_io.pyx:220, in pyogrio._io.ogr_open()

DataSourceError: 'DOCTYPE html><html>  <head>    <meta http-equiv="content-type" content="text/html; charset=utf-8" />    <meta name="viewport" content="width=device-width" />    <link rel="stylesheet" href="https://cdn-a.arcgis.com/cdn/1B87370/js/calcite-web/dist/css/calcite-web-no-fonts.min.css" />    <link rel="stylesheet" type="text/css" href="https://cdn-a.arcgis.com/cdn/1B87370/js/arcgis-components/dist/ItemBrowser/index.css" />    <link rel="stylesheet" type="text/css" href="https://cdn-a.arcgis.com/cdn/1B87370/js/arcgis-components/dist/Share/index.css" />    <link rel="stylesheet" type="text/css" href="https://cdn-a.arcgis.com/cdn/1B87370/js/arcgis-components/dist/GroupBrowser/index.css" />    <link rel="stylesheet" href="https://cdn-a.arcgis.com/cdn/1B87370/js/arcgisonline/css/app.css" />    <link rel="stylesheet" type="text/css" href="https://cdn-a.arcgis.com/cdn/1B87370/js/calcite-components/calcite/calcite.css">    <link rel="stylesheet" type="text/css" href="https://cdn-a.arcgis.com/cdn/1B87370/js/arcgis-app-components/arcgis-app/arcgis-app.css">    <link rel="stylesheet" href="https://cdn-a.arcgis.com/cdn/1B87370/js/jsapi/dojox/highlight/resources/highlight.css">    <link rel="stylesheet" href="https://cdn-a.arcgis.com/cdn/1B87370/js/jsapi/dojox/highlight/resources/pygments/friendly.css" />  </head>  <body spellcheck="false">    <div class="loader is-active padding-leader-5 padding-trailer-5 js-page-loader">      <div class="loader-bars"></div>      <div class="loader-text"></div>    </div>    <script type="text/javascript" src="./js/arcgisonline/config.js"></script>    <script src="https://cdn-a.arcgis.com/cdn/1B87370/js/jsapi/dojo/dojo.js"></script>    <script type="module" src="https://cdn-a.arcgis.com/cdn/1B87370/js/calcite-components/calcite/calcite.esm.js"></script>    <script nomodule="" src="https://cdn-a.arcgis.com/cdn/1B87370/js/calcite-components/calcite/calcite.js"></script>    <script type="module" src="https://cdn-a.arcgis.com/cdn/1B87370/js/arcgis-app-components/arcgis-app/arcgis-app.esm.js"></script>    <script>      require(window.dojoConfig, ["arcgisonline/main"]);    </script>  </body></html>' does not exist in the file system, and is not recognized as a supported dataset name.

EarlMedina · ‎11-19-2024

I think the quickest way to get what you need is to just use geojson. I'm not sure if leafmap takes a geojson string, or dict but this should give you geojson string:

def read_fl(item_id):
    living_atlas_item = gis.content.get(item_id)
    feature_layer = living_atlas_item.layers[0]
    fs = feature_layer.query(where="1=1", out_sr=4326)
    return fs.to_geojson

If it requires a dict, then modify to this:

import json
def read_fl(item_id):
    living_atlas_item = gis.content.get(item_id)
    feature_layer = living_atlas_item.layers[0]
    fs = feature_layer.query(where="1=1", out_sr=4326)
    geojson_string = fs.to_geojson
    return json.loads(geojson_string)

Per leafmap documentation, you should not have to create a gdf for this to work:

m = leafmap.Map(center=[0, 0], zoom=2)

in_geojson = "https://raw.githubusercontent.com/opengeos/leafmap/master/examples/data/cable_geo.geojson"
m.add_geojson(in_geojson, layer_name="Cable lines")

ZacharyKasson · ‎11-19-2024

I appreciate the responses. It seems there is a direct route from sdf to gdf. I accepted it as a solution.

ZacharyKasson · ‎11-19-2024

I found it to be simple than I thought.

gdf = gpd.GeoDataFrame(sdf, geometry='SHAPE')