Latest Contributions by KimOllivier

OK, here is my broken public dashboard. Cannot make table selector see the service layer.  Here's hoping you can identify it with this link https://www.arcgis.com/apps/dashboards/e7c0e881e4e84f00b03be7e3abc52f36 ... View more

It appears that there are many bugs introduced in the June 2026 update. I have just found my dashboard is broken with a missing link that cannot be fixed. ... View more

Pro Tip: Schema definitions Use schema.ini to define a schema on CSV files before adding to the contents. This enables you to rename fields, force the type, define text field lengths and create a table that is ready to save into a filegeodatabase.  If you have an existing table with the desired schema an easy way to create the schema.ini is to export one line to a TXT file and ArcGIS will write out a definition for you. Just edit the header to point to the new file, make any changes such as deleting the objectid field and you are away! Esri supports schema.ini. If you don't have a schema.ini entry all text fields will default to 255 characters.  ... View more

You still have unlimited access to the python geometry engine. So you can re-program every tool that you need using arcpy. You will have more low level work, processes work on pairs of geometry objects so you will need to interate over the featureclass. Often you can program in a shortcut that makes it faster than the general tool. If you only need a few restricted tools this may be worth it to keep using the Basic licence. I agree that not being able to create more complex featureclasses is a serious drawback. https://pro.arcgis.com/en/pro-app/latest/arcpy/classes/geometry.htm  This often happens to me when a customer wants an app that works only on Basic. If there is a tool not available I create my own. eg I wanted to Densify a polygon from four straight lines to many vertices so that when it was projected the lines were curves. Oops, Standard or Advanced only. But I only have one polygon to edit and that can be done easily by adding vertices in a script. You can move selections easily between a featureclass and an object list. Much can be done in memory or with python lists. There are some tools that disappeared in ArcMap that I have rebuilt. eg Frequency with Case. I often want to use a composite key but relates only work with a single field. Just run my tool that creates a new case# that is a single number for the composite key. Many others that I used to have are now in Pro such as Convex Hull. http://www.ollivier.co.nz/support/python_tips/index.htm  Look at the sample Furthest Town task for an example of how to replace Near().     ... View more

I have a solution where the field is redefined in the geopackage before uploading. A second version allows for non UTC datetime. import pandas as pd def normalise_date_fields(gdf, use_utc: bool = True, local_tz=None, verbose: bool = True): """ Convert epoch-based date fields to timezone-aware datetimes. Targets columns with 'date' or 'time' in the name. Parameters ---------- gdf : GeoDataFrame Input GeoDataFrame. use_utc : bool, default True If True, output datetimes are in UTC (tz-aware, +00:00). If False, output datetimes are converted to local timezone (tz-aware with offset). local_tz : str | tzinfo | None Timezone to use when use_utc=False. - None => use the machine's local timezone. - str => e.g. "Pacific/Auckland" verbose : bool Print conversion messages. Returns ------- GeoDataFrame GeoDataFrame with converted datetime columns where applicable. """ # Resolve local timezone (only used when use_utc=False) if not use_utc: if local_tz is None: # Use the runtime environment's local timezone local_tz = pd.Timestamp.now().tz_localize("UTC").tz_convert(None).tzinfo # Note: tz_convert(None) returns naive; tzinfo can still be recovered in many envs, # but if your environment behaves oddly, pass local_tz="Pacific/Auckland". elif isinstance(local_tz, str): # Pandas can use IANA tz database strings local_tz = local_tz # else assume tzinfo-like object is passed through for col in gdf.columns: col_lc = col.lower() if "date" in col_lc or "time" in col_lc: series = gdf[col] # Only attempt conversion on numeric columns if pd.api.types.is_numeric_dtype(series): max_val = series.dropna().max() # Heuristic: epoch milliseconds are usually > 1e12 if pd.notna(max_val) and max_val > 1e12: try: # Step 1: interpret epoch(ms) as UTC dt_utc = pd.to_datetime(series, unit="ms", utc=True, errors="coerce") # Step 2: optionally convert to local timezone (keeping offset) if use_utc: gdf[col] = dt_utc if verbose: print(f" ✓ Converted epoch(ms) → datetime (UTC): {col}") else: gdf[col] = dt_utc.dt.tz_convert(local_tz) if verbose: print(f" ✓ Converted epoch(ms) → datetime (local offset): {col} ({local_tz})") except Exception as e: if verbose: print(f" ⚠ Failed to convert {col}: {e}") return gdf # -------- main ----------- # other processing... gdf = gpd.GeoDataFrame.from_features(features) gdf = gdf.set_crs("EPSG:4326") gdf = normalise_date_fields(gdf) gdf.to_file(GPKG_PATH, layer=layer, driver="GPKG") print(f" Saved: {layer} ({len(gdf)} features)")    ... View more

Reinstalling is not the solution. It is to do with configuration in your <user>\appdata\local|roaming\Esri\ArcGISPro folders. Try renaming them to force Pro to rebuild them. Other tools may hang Pro such as partly failed web connections, graphics card incompatibility and a host of other settings not cleared with a reinstall. ... View more

I am having trouble with date fields in a geopackage. The best format is an ISO string because it can include the timezone. But it can be an epoch number. Sqlite handles all dates as functions, not a data type. This means that AGOL cannot read a date in a geopackage. How does Survey123 handle dates in a geopackage?   ... View more

YES, but with Pro 3.5.3. I have tried a lot of fixes but to no avail. Copilot blamed the GPU, and all sorts of things. The easy solution is to rename the folders  <user>Appdata\Local\Esri\ArcGISPro and <user>Appdata\roaming\Esri\ArcGISPro and reboot. You lose all your recent list etc but its a small price to pay. There are a lot of other things that may be hanging and remain open that freezes everything including Exit. If you have to use TaskManager its Esri's fault, not you or your data. Don't wait for things to close. If anything takes longer than a cup of coffee, interrupt the process and find a fix or find a better way. ... View more

You might try my script. It extracts the data from WFS, loads it into a geopackage, uploads the geopackage, zipped up. Then turn the online geopackage into a published feature layer collection. Get out the item_ids and put them in your script to update instead of add. It is fast and reliable, only uses open source tools arcgis and a AGOL licence. Not a notebook. Now all I need to solve is how to format dates so that AGOL can recognise them as dates. ... View more

I never completed the script because Esri fixed the bug to allow adding a WFS layer directly in AGOL. Since then Esri have made a lot of changes to WFS connections. They do not like them because it is not the preferred REST interface. For a limited number of records it works well, but over a few thousand it might have a performance hit. You can filter the records in the WFS URL but that becomes fiddly and specific. I prefer the 'dynamic' interface that avoids having to run a python Docker container to refresh a featurelayer at intervals. I can see a lot of problems trying to maintain a multiuser live dataset behind the scenes. The result is a fragile interface where sometimes the update is not successful. I also had to solve importing more modules into Docker that were not in the default. This is feeling very hacky! WFS is a batch process so it is not a live update, the full table is cached somewhere. It only gets a snapshot, the on demand button does nothing. Since my source is only updated nightly it did not need a live feed. The disadvantage of a direct WFS layer is that Avenue is disabled and everything is read-only. This results in a WISIWIG display that depends on how well the source server structures and presents the data. But you can still do most filtering, labelling and symbology suitable to use in a Dashboard which is my sole purpose of using the WFS layer. I had thought that creating a WFS feature layer and then using it in a Web Map would fool AGOL but it doesn't, everything is read-only, not even a virtual field is permitted.  My original workflow use ArcGISPro to do the WFS feed and then overwrite the AGOL featurelayer. This menu item has now been disabled in the latest update 3.5 update. Copilot suggests this is because keeping the connection open causes problems closing Pro so it had to go. I can still open a WFS feed and export it to a geodatabase. The obvious next step was to automate this with arcpy and arcgis. That is hard to make work because there is no easy way to export a dataframe to a feature layer. The extension for geoframes does not support WFS so I have to use pandas and that is not compatible. Also it required Pro to run. My thoughts turned to using an AGOL notebook. There was a conference technical presentation on how to do it, but I got the impression that the speaker did not recommend it. I have managed to make a semi-dynamic Dashboard that does not require any update process to get the data as entered into a separate app called Trap.nz that provides a WFS interface with some non-standard format. Pest Activities In conclusion I have given up any customization in the interests of simplicity. I managed to do most filtering and dynamic displays of the activity in three Dashboards. Any deep analysis is best done on the desktop. The result is a completely no-code solution, although it still requires a lot of expertise and a manual to set up. But here is my script anyway. Notebooks are not a valid file type, take off the .txt suffix after downloading. A second script I have written that does not use arcpy and Pro, only arcgis module and python with a geopackage. You have to upload a geopackage and then go into AGOL and create a published feature layer from the online geopackage item. Then the next time you run the script fill in the item_ids and do an update so that the item_ids are retained for the webmap and dashboard. All good except: geopackages do not have a date field. The dates are not recognised by the dashboard. How can this be fixed? ... View more

I have spend a week off and on trying to get VSCode to work as it used to. I found this gem from Copilot. Presumably other people know about this. I don't like the solutions proposed much. ArcGIS Pro’s integration with VSCode is well-intentioned - but a bit heavy-handed. When you launch VSCode from within ArcGIS Pro, it tries to “help” by injecting its environment into your global settings.json , but it sets: the path to the env folder without python.exe on the end —which is a folder, not an executable. VSCode then tries to run that as if it were python.exe, and boom: you get the conda.exe error because it’s misinterpreting the path. ... View more

Date arithmetic is awful. The best thing is to convert all date fields to ISO strings. Then comparisons and ranges are easy and if you need to do date arithmetic you can convert the ISO to a date object easily. ISO format is set up so that dates are in string sort order, the timezone can be appended (UTC = Z) or the offset added. It is easy to get a subset to extract part of the date because they are in fixed widths. Date formats often do not go before 1970 (Microsoft), 1900 (Unix) so don't use them. ... View more

Date arithmetic is awful. The best thing is to convert all date fields to ISO strings. Then comparisons and ranges are easy and if you need to do date arithmetic you can convert the ISO to a date object easily. ISO format is set up so that dates are in string sort order, the timezone can be appended (UTC = Z) or the offset added. It is easy to get a subset to extract part of the date because they are in fixed widths. Date formats often do not go before 1970 (Microsoft), 1900 (Unix) so don't use them. ... View more

If you just try to edit the dashboard and replace the source all settings will be lost so you have to rebuild the dashboard from scratch. ... View more