topic Add geopandas package to AGOL Notebook in ArcGIS Online Questions

Add geopandas package to AGOL Notebook

KimOllivier — Tue, 17 Sep 2024 22:10:49 GMT

I cannot get WFS to work correctly in AGOL by adding a feed as a FeatureLayer. It sort of works but ignores the layer selection and simply gets the first layer.

If I use a WebMap and add a layer with a source as WFS then the same dialogs appear and the layer selection does work to produce a map layer. Fine, but it is not very useful because I cannot use Arcade on it. It is read-only. I can add a cql_filter with custom parameters. Finally I cannot export it to a FeatureLayer from the WebMap. It is slightly promising because it indicates that it might work if the bug when defining a FeatureLayer source is fixed.

I really need to copy the WFS layers into a proper FeatureLayer Collection. You CAN do this easily using Pro! Set up a new WFS server in ArcCatalog, including custom parameters as required, add the layers to the table of contents.

Turn off some fields, add more filters, rename layers, load metadata and edit, set up symbology. Change the download limits from 3000 to 100,000. Now just Share to AGOL! Any schema change or new data requires manually replacing the layer, watch out for caches.

Now I want to automate this with Python. Particularly in AGOL using a Notebook and a Task. Seems to be possible with the new Pandas SeDF (spatially enabled dataframe) in arcgis package. Not so fast!

Loading WFS features into Pandas sdf.

This should be easy to do with python requests and geopandas. There are new functions in arcgis to translate from geopandas to sdf and from sdf to a FeatureLayer. You need to import geopandas so that you can read the WFS feed directly into a sdf data frame first. There is a posting noting that you cannot read a WFS feed directly into an Esri sedf. Catch 22 - you cannot import geopandas in a AGOL notebook because it is a Docker instance started up each time. It can work from a Desktop because you can set up a venv that does have geopandas.

So how can I install geopandas in AGOL Notebook?

Re: Add geopandas package to AGOL Notebook

KimOllivier — Thu, 19 Sep 2024 11:10:15 GMT

prefix = '\"%s\"' % sys.prefix !conda install --yes --prefix {prefix} geopandas

Re: Add geopandas package to AGOL Notebook

ILoveMap — Wed, 11 Mar 2026 17:02:02 GMT

Are you able to share your notebook that copies a WFS toa hosted feature layer? Also need this functionality

Re: Add geopandas package to AGOL Notebook

KimOllivier — Thu, 12 Mar 2026 07:49:27 GMT

I never completed the script because Esri fixed the bug to allow adding a WFS layer directly in AGOL. Since then Esri have made a lot of changes to WFS connections. They do not like them because it is not the preferred REST interface. For a limited number of records it works well, but over a few thousand it might have a performance hit. You can filter the records in the WFS URL but that becomes fiddly and specific. I prefer the 'dynamic' interface that avoids having to run a python Docker container to refresh a featurelayer at intervals. I can see a lot of problems trying to maintain a multiuser live dataset behind the scenes. The result is a fragile interface where sometimes the update is not successful. I also had to solve importing more modules into Docker that were not in the default. This is feeling very hacky! WFS is a batch process so it is not a live update, the full table is cached somewhere. It only gets a snapshot, the on demand button does nothing. Since my source is only updated nightly it did not need a live feed.

The disadvantage of a direct WFS layer is that Avenue is disabled and everything is read-only. This results in a WISIWIG display that depends on how well the source server structures and presents the data. But you can still do most filtering, labelling and symbology suitable to use in a Dashboard which is my sole purpose of using the WFS layer.

I had thought that creating a WFS feature layer and then using it in a Web Map would fool AGOL but it doesn't, everything is read-only, not even a virtual field is permitted.

My original workflow use ArcGISPro to do the WFS feed and then overwrite the AGOL featurelayer. This menu item has now been disabled in the latest update 3.5 update. Copilot suggests this is because keeping the connection open causes problems closing Pro so it had to go. I can still open a WFS feed and export it to a geodatabase. The obvious next step was to automate this with arcpy and arcgis. That is hard to make work because there is no easy way to export a dataframe to a feature layer. The extension for geoframes does not support WFS so I have to use pandas and that is not compatible. Also it required Pro to run. My thoughts turned to using an AGOL notebook. There was a conference technical presentation on how to do it, but I got the impression that the speaker did not recommend it.

I have managed to make a semi-dynamic Dashboard that does not require any update process to get the data as entered into a separate app called Trap.nz that provides a WFS interface with some non-standard format.

Pest Activities

In conclusion I have given up any customization in the interests of simplicity. I managed to do most filtering and dynamic displays of the activity in three Dashboards. Any deep analysis is best done on the desktop.

The result is a completely no-code solution, although it still requires a lot of expertise and a manual to set up.

But here is my script anyway. Notebooks are not a valid file type, take off the .txt suffix after downloading.

A second script I have written that does not use arcpy and Pro, only arcgis module and python with a geopackage. You have to upload a geopackage and then go into AGOL and create a published feature layer from the online geopackage item. Then the next time you run the script fill in the item_ids and do an update so that the item_ids are retained for the webmap and dashboard. All good except: geopackages do not have a date field. The dates are not recognised by the dashboard. How can this be fixed?