error loading WFS url

[HermanGroeneveld]

Hello,

I'm still struggling with WFS by using ArcPy. Beneath I have a link to some GIS data. When I click the link, the links shows me the coorect result in the browser, but if I use this link in Arcpy it stumbles on the fs.load(url) and raises an error saying:

RuntimeError: RecordSetObject: Cannot open table for Load

I can't find what's wrong. is it the url? Is it the outputformat? The request? Or is it something else?

Here is the link:

http://geodata.nationaalgeoregister.nl/bag/wfs?service=wfs&version=2.0.0&request=GetFeature&typeName...

Here is the code in Python I use: 

import arcpy

url = 'http://geodata.nationaalgeoregister.nl/bag/wfs?service=wfs&version=2.0.0&request=GetFeature&typeName=bag:verblijfsobject&count=5&outputFormat=json'
fs = arcpy.FeatureSet()
fs.load(url)
fs.save(r'D:\GIS data\FGDB_datatest.gdb\test_001')‍‍‍‍‍‍

Please, any help is welcome!

[Egge-Jan_Pollé]

Hi Herman Groeneveld

I think it is the output format that is wrong; I suppose this FeatureSet expects input in Esri JSON format, not in the GeoJSON format from the BAG WFS service from nationaalgeoregister.

But did you know that Esri Nederland also serves the official Dutch BAG data via ArcGIS Online? See this page: BAG - Basisregistratie Adressen en Gebouwen 

You can query the Adres layer from this service via this page: Query: Adres (ID: 0) 

So following the same pattern as the answer you got from Esri Nederland in another thread (Make Arcpy use Esri Feature Service) you can retrieve address points for a Dutch postal code (e.g. 3524WL) like this:

import arcpy

# Query URL opbouwen op basis van bepaalde postcode
ppc = '3524WL'
url = 'https://basisregistraties.arcgisonline.nl/arcgis/rest/services/BAG/BAGv2/MapServer/0/query?where=postcode+%3D+%27{}%27&outFields=*&returnGeometry=true&f=pjson'.format(ppc)

# Omzetten van query naar feature class
fs = arcpy.FeatureSet()
fs.load(url)
fs.save(r'C:\Data\output.gdb\PPC6_{}'.format(ppc))
print ("Klaar")‍‍‍‍‍‍‍‍‍‍‍

Hope this helps,

Egge-Jan

[HermanGroeneveld]

Thanks for your reaction Egge-Jan! I'm totally aware of the  ontent Esri Nederland offers and I recently received the same part of code by a colleague of yours. In that way my question was a bit unlucky, I guess: in this case Esri Nederland serves the same content as a non ESRI-WFS.

But I was testing WFS in generally, tested the BAG WFS and I thought the problem which occured was a common issue. But if I understand correct, this is a issue due to mismatch in outputFormat? So in this case I won't be able to get the WFS in ArcGIS by any other offered outputformat (gml, json, application/json, gml+xml, etc.)?

[Egge-Jan_Pollé]

Hi Herman Groeneveld,

Did you have a look at this one: WFS To Feature Class—Conversion toolbox | ArcGIS Desktop ?

(I am not working for Esri Nederland though 🙂

If we translate the Code Sample from that page to a Dutch example - let's take the Dutch National Parks, my favorite dataset from PDOK - we get something like this:

# Name: WFSToFeatureClass_example1.py
# Description: Create a feature class from a WFS service

# Import arcpy module
import arcpy

# Set local variables
WFS_Service = "https://geodata.nationaalgeoregister.nl/nationaleparken/wfs?request=GetCapabilities&service=wfs"
WFS_FeatureType = "nationaleparken"
Out_Location = "C:/Data/output.gdb"
Out_Name = "Nationale_Parken"

# Execute the WFSToFeatureClass tool
arcpy.WFSToFeatureClass_conversion(WFS_Service, WFS_FeatureType, Out_Location, Out_Name)‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This will get you all the National Parks in the Netherlands, including the newly created park Nieuw Land - see screen capture below.

HTH,

Egge-Jan

[HermanGroeneveld]

Egge-Jan, sorry for mistaken you worked for ESRI, I don't know why I had that idea. Maybe because your detailed answer. Anyhow, thanks again for your reply.

I copied your code and it works like a charm. However, some other datasets provided by PDOK I can't get to work, like this one:

https://geodata.nationaalgeoregister.nl/ecotopen/wfs?request=GetCapabilities&service=wfs

https://geodata.nationaalgeoregister.nl/cbspostcode4/wfs?&request=GetCapabilities&service=wfs

It raises an error saying 'ERROR 000366: Invalid geometry type'. Any idea why this one doesn't seem to work? 

[Egge-Jan_Pollé]

Hi Herman Groeneveld,

Please have a look at the local variables to set. If you want to retrieve - say - the postal code polygons for the year 2017, you should not only change the variable WFS_Service, but also WFS_FeatureType (and of course Out_Name, to avoid overwriting previous output...)

When you have a look at the XML returned by the GetCapabilities request for the postal code polygons, you will see a FeatureTypeList with 3 FeatureTypes: polygons for the years 2015 (postcode42015), 2016 (postcode42016) and 2017 (postcode42017).

Below you will find the Python script to retrieve all postal code polygons for the year 2017 (4,066 of them) - and yes, like you say, it works like a charm 🙂

BTW - if you consider one or more of the sub answers useful ("Nuttig"), please do not hesitate to mark them as such. GeoNet likes this kind of feedback.

And GeoNet would also like to know whether your initial question has been answered - in this way we can help the Esri Community to find useful and correct answers.

And of course, you are always welcome to post new questions on this great learning resource. Or, equally useful: to post answers to questions posted by others...!

HTH,

Egge-Jan

...
<FeatureTypeList>
...
<FeatureType>
  <Name>cbspostcode4:postcode42017</Name>
  <Title>Postcode 2017, numeriek deel</Title>
  <Abstract>
    Gegevens per numeriek deel van de postcode voor het jaar 2017
  </Abstract>
...‍‍‍‍‍‍‍‍‍‍

# Name: WFSToFeatureClass_example1.py
# Description: Create a feature class from a WFS service

# Import arcpy module
import arcpy

# Set local variables
WFS_Service = "https://geodata.nationaalgeoregister.nl/cbspostcode4/wfs?&request=GetCapabilities&service=wfs"
WFS_FeatureType = "postcode42017"
Out_Location = "C:/Data/output.gdb"
Out_Name = "postcode42017"

# Execute the WFSToFeatureClass tool
arcpy.WFSToFeatureClass_conversion(WFS_Service, WFS_FeatureType, Out_Location, Out_Name)‍‍‍‍‍‍‍‍‍‍‍‍‍‍

[HermanGroeneveld]

I tested your code with the postcode42017, which indeed works like a charm. But I'm not able to use the environment Extent setting. The documentation tells me the WFS To Feature Class conversion should work with the extent of a feature class, display or whatever:

By default all features from the WFS source are added to the feature class. The extent environment setting can be used to limit the features to just those that intersect a user-defined extent...

So I copied your code, added lines 3,4 en 5. (Those lines worked well on a regelar featureclass in combimnation with Select_analysis)

import arcpy

fc_ext = r"C:\Users\Herman\OneDrive\Algemeen\ArcGIS\GIS data\FGDB_datatest.gdb\DS_datatest\test_01"
desc = arcpy.Describe(fc_ext)
arcpy.env.extent = desc.extent

WFS_Service = "https://geodata.nationaalgeoregister.nl/cbspostcode4/wfs?&request=GetCapabilities&service=wfs"
WFS_FeatureType = "postcode42017"
Out_Location = r"C:\Users\Herman\OneDrive\Algemeen\ArcGIS\GIS data\FGDB_datatest.gdb"
Out_Name = "tmp2"

# Execute the WFSToFeatureClass tool
arcpy.WFSToFeatureClass_conversion(WFS_Service, WFS_FeatureType, Out_Location, Out_Name)‍‍‍‍‍‍‍‍‍‍‍‍‍

The code above raises an error saying:

ERROR 999999: Error executing function

How can I make the extent setting get to work for me???

[Egge-Jan_Pollé]

Hi Herman Groeneveld,

Yeah, you cannot query a WFS service when you use the GetCapabilities request. By default all features from the WFS source are added to the feature class. It is cool that you can use the extent environment setting to set a bounding box. I didn't know that....

But no, I do not get it running either. I tried to set the Extent (Environment setting) in both RD_New and WGS84, but in both cases I run into the same ERROR 999999 like you...

Below you will find the code I used to test this.

Tip: maybe you should ask this in a new question on GeoNet, to see whether someone from the wider community can help.

BR,

Egge-Jan

# Name: WFSToFeatureClass_example1.py
# Description: Create a feature class from a WFS service

# Import arcpy module
import arcpy

# Set the extent environment using the Extent class.
# arcpy.env.extent = arcpy.Extent(140000, 448000, 170000, 478000) # Coordinates in RD_New (28992)
arcpy.env.extent = arcpy.Extent(5.1686749, 52.0201494, 5.6070564, 52.2897848) # Coordinates in WGS84 (4326)

# Set local variables
WFS_Service = "https://geodata.nationaalgeoregister.nl/cbspostcode4/wfs?&request=GetCapabilities&service=wfs"
WFS_FeatureType = "postcode42017"
Out_Location = "C:/Data/output.gdb"
Out_Name = "postcode42017_new"

# Execute the WFSToFeatureClass tool
arcpy.WFSToFeatureClass_conversion(WFS_Service, WFS_FeatureType, Out_Location, Out_Name)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

. 

[diaconori]

Same issue 3 years later. Doesnt work. 

Owslib doesnt work. Regular python libs wont work. Qgis wont work. Seems wfs services are dead...