Latest Contributions by KimOllivier

‎09-17-2024

I cannot get WFS to work correctly in AGOL by adding a feed as a FeatureLayer. It sort of works but ignores the layer selection and simply gets the first layer. If I use a WebMap and add a layer with a source as WFS then the same dialogs appear and the layer selection does work to produce a map layer. Fine, but it is not very useful because I cannot use Arcade on it. It is read-only. I can add a cql_filter with custom parameters. Finally I cannot export it to a FeatureLayer from the WebMap. It is slightly promising because it indicates that it might work if the bug when defining a FeatureLayer source is fixed. I really need to copy the WFS layers into a proper FeatureLayer Collection. You CAN do this easily using Pro! Set up a new WFS server in ArcCatalog, including custom parameters as required, add the layers to the table of contents. Turn off some fields, add more filters, rename layers, load metadata and edit, set up symbology. Change the download limits from 3000 to 100,000. Now just Share to AGOL! Any schema change or new data requires manually replacing the layer, watch out for caches. Now I want to automate this with Python. Particularly in AGOL using a Notebook and a Task. Seems to be possible with the new Pandas SeDF (spatially enabled dataframe) in arcgis package. Not so fast! Loading WFS features into Pandas sdf. This should be easy to do with python requests and geopandas. There are new functions in arcgis to translate from geopandas to sdf and from sdf to a FeatureLayer. You need to import geopandas so that you can read the WFS feed directly into a sdf data frame first. There is a posting noting that you cannot read a WFS feed directly into an Esri sedf. Catch 22 - you cannot import geopandas in a AGOL notebook because it is a Docker instance started up each time. It can work from a Desktop because you can set up a venv that does have geopandas. So how can I install geopandas in AGOL Notebook?

‎09-17-2024

I cannot get WFS to work correctly in AGOL by adding a feed as a FeatureLayer. It sort of works but ignores the layer selection and simply gets the first layer. If I use a WebMap and add a layer with a source as WFS then the same dialogs appear and the layer selection does work to produce a map layer. Fine, but it is not very useful because I cannot use Arcade on it. It is read-only. I can add a cql_filter with custom parameters. Finally I cannot export it to a FeatureLayer from the WebMap. It is slightly promising because it indicates that it might work if the bug when defining a FeatureLayer source is fixed. I really need to copy the WFS layers into a proper FeatureLayer Collection. You CAN do this easily using Pro! Set up a new WFS server in ArcCatalog, including custom parameters as required, add the layers to the table of contents. Turn off some fields, add more filters, rename layers, load metadata and edit, set up symbology. Change the download limits from 3000 to 100,000. Now just Share to AGOL! Any schema change or new data requires manually replacing the layer, watch out for caches. Now I want to automate this with Python. Particularly in AGOL using a Notebook and a Task. Seems to be possible with the new Pandas SeDF (spatially enabled dataframe) in arcgis package. Not so fast! Loading WFS features into Pandas sdf. This should be easy to do with python requests and geopandas. There are new functions in arcgis to translate from geopandas to sdf and from sdf to a FeatureLayer. You need to import geopandas so that you can read the WFS feed directly into a sdf data frame first. There is a posting noting that you cannot read a WFS feed directly into an Esri sedf. Catch 22 - you cannot import geopandas in a AGOL notebook because it is a Docker instance started up each time. It can work from a Desktop because you can set up a venv that does have geopandas. So how can I install geopandas in AGOL Notebook?

‎09-15-2024

Well it is possible in ArcGISPro without any python programming. All you do is add a WFS server, tweak the URL, add any cql_filter expressions and get the layers listed. Add these to a map and tweak symbology, visible items and definition queries, layer names etc. Then select all the layers and Share>featurelayer>publish. This will create a better FeatureLayerCollection in AGOL. Why better? Because you can edit, make WebMaps and anything else. The only drawback? It is not dynamic. But wait, neither is WFS usually. The one I have just extracted only updates nightly anyway. So how can you replicate this in Python. In principle duplicate the steps: 1. Use the requests module to set up a request on the WFS endpoint just like the WFS server 2. Download each sublayer as a json extract and move into a Pandas SpatiallyEnabledDataFrame. Not quite so easy in AGOL because the geopandas module is missing. In principle it should work but is not supported directly. But you can copy a geopandas dataframe to an esri spatialdataframe. 3. upload the sdf to AGOL with the GeoAccessor esri extension to arcgis. If the WFS data can be queried for changes you could use the gis.content.edit() functions to just add new records. Don't know about delete records.

‎09-15-2024

The arcgis python API is a rest interface, not a relational database. Everything is atomic and restful. This means you cannot edit a single attribute in a record. You have to retrieve the whole record, edit the attributes you want and post the whole record back into the online feature layer. Because the REST interface is Stateless. It does not remember anything you have done, which record you edited etc. Its like a whole new phone call to a new support caller without a log. You have to explain the issue all over again. I liken it to having string puppets digging a hole with a shovel instead of you just doing it with a shovel yourself. The puppet is clumsy and very poor at digging holes. You cannot use the arcpy package to edit online data. You have to use the arcgis package.

‎09-05-2024

Thanks for that explanation. It does show why it is happening. But the option switch says "to use field types that are compatible with ArcGISPro 3.1", which are long integer. So even if it is correct to move 64 bit fields in Sqlite to BigInteger they do not work in Pro for many functions if they are cast that way. This must be a problem with other databases in Enterprise too. I suspect they had a very narrow view of this problem when implementing the option. All I am asking for is for the switch to work everywhere. [Edit. It only works inside ArcGISPro, you can use the switch in arcpy.env.useCompatibleFieldTypes = True when running scripts outside Pro. But a cast to Double is not useful for me.]

‎06-07-2024

I wouldn't be using float values for relates. That will be very unreliable with rounding errors. The only solution if you have very large integers is to convert them to text strings. This has happened to me in the past with very large prefixes added in Oracle to merge databases creating 13 digit integers with lots of zeros. If you use text then it will still preserve the IDs even in Shape (😂) files. And now BigIntegers are appearing that do not work in most applications, relates or whatever. If Long Integer is not enough, then Text is the best option there too. Census tracts are very long numbers and sometimes they have leading zeros that are significant. Text fields will preserve these.

‎05-23-2024

Ok, post up the table schema and attribute schema and your relate expression. there are two types of relate, a Map relate (in memory) and a database relate (stored in the database). Are both primary and foreign key fields the same? Are they indexed? If they are characters lots of things go wrong with case, leading or trailing blanks, unicode. It is safest to use integers but now esri has wrecked things with Big Integers that don't always work. What are you expecting with a relate? It is not a join. You can see related records with a popup, or a selection can be transferred with the hamburger options on the table. Otherwise fairly useless for mapping.

‎05-23-2024

Relates are only possible with internal feature classes, not external data. You will have to either copy the data into a file geodatabase (always a good first step) or you could use python and dictionaries with cursors to get the related data if you are familiar with scripting.

‎05-23-2024

Not fixed in 3.3. Ah well, back to 3.1 so that I can attempt to port workflows from ArcMap! There is a setting in Options/Map and Scene - but it doesn't work for me. [Edit: it casts BigInteger to Double, not Long so cannot be used as a key]

‎04-16-2024

Good question, I dropped the issue for myself so I never finished debugging for Joshua. Maybe I will have another look to develop more tips with an example.

‎03-15-2024

line 19. You are changing Connections, not featureclass paths. (I do not know why a shapefile exists in your system!!) I could not find one to test out your script! line 19. The updateConnectionProperties is a search and replace function, so you need to have a string replace option, not a full path option line 9 . If you are comparing paths it is safer to normalize the paths for string comparisons using old = os.path.normpath(old_path). Maybe updateFolderConnections would work better for shapefiles Also you are changing the wrong object source. The project has a separate connections. You should be changing the Layer connection properties.

‎02-22-2024

And now we have BigInteger (64bit) which have appeared since 3.2 that are not interchangeable with Long Integer or Short Integer in many tools. eg RelationshipClasses do not allow them as a valid type for a key.

‎02-22-2024

I agree with @JohannesBierer that you must not divide up the river, leave it as a line. The Near tool find the perpendicular distance and the xy point at the intersection of the river as attributes which you can then use.

‎02-22-2024

The limit for a shapefile field is 10 characters. Are you really still using shapefiles? Not really very sympathetic if you are! Maybe you have a path problem so you are pointing at a folder and not a geodatabase. Some example code might help others explain the probably "obvious" error.

‎02-22-2024

I have found other problems with 64 bit Integers now called BigInteger. Integer fields defined in geopackages and spatialite are now cast by default to BigInteger instead of Integer. RelationshipClasses cannot use them which was a surprise because I suddenly had BigIntegers for every primary and foreign key in my database. I have had to go back and explicitly cast to Integer (32 bit) using FieldMapping on every copy operation so I am not surprised that Pandas needs editing. Just about to do some datetime manipulation, thanks for the heads-up.

Online Status	Offline
Date Last Visited	‎10-20-2025 06:39 PM

My Ideas

Latest Contributions by KimOllivier

Add geopandas package to AGOL Notebook

Re: Problems with WFS on ArcGIS Online

Re: Adding WFS to webmap using Python API

Re: API for python spatial.to_featureclass - maintain attachments?

Re: Environment setting to default Integer to Long not BigInteger

Re: Model builder calculate field, from iterate fields to new field

Re: Relate Tool not Working

Re: Relate Tool not Working

Re: Environment setting to default Integer to Long not BigInteger

Re: using arcpy to change data sources in aprx files

Re: using arcpy to change data sources in aprx files

Re: Integer field types (short, long) - put the word "integer" back

Re: How transpose a point on a line

Re: Error 000013 - Length of Field Name - Python

Re: Pandas and DateTimes in Pro 3.2

Re: Trying to configure ArcGIS Pro/ArcPy + VS Code...

Re: Building Database Relates in Pro is ten times ...

Building Database Relates in Pro is ten times slow...

Backup and Restore a Dashboard (and related items)

Re: Convert .mdb to .gdb without the use of deskto...