Data Generation At Scale
Here's 350K places of interest points in the Czech Republic I generated on demand, with descriptive base table fields and related alternate category descriptions, all as features in ArcGIS Pro's memory workspace. You can make your own data for any area of interest worldwide using the data generation script tool in the blog download. You'll need ArcGIS Pro 3.5+ and a few seconds of your time - in my case 35 seconds for the Czech data (that's 761422 features total between the place points and lookup table).
I'll let you inspect the tool source yourself, but the key takeaway is that it's easy to wrangle global scale data in Pro without worrying about scale limitations or performance - when you are using the right approach - namely with DuckDB.
Inspiration for this post came when I was reviewing a presentation from PyData 2025, where I was reminded that larger than memory workflows can be handled by DuckDB, including queries with a spatial component.
In the case of the post's tool, the spatial component is an input parameter of polygon data (any coordinate system, any number of polygons, the pale blue feature in the map) that encloses an area of interest, and for a non-spatial component, an optional where clause to be applied in addition to the area of interest. In the case of the data shown, the clause was country = 'CZ'. You should make some test data in a small area of interest to refine any where clause you need - the clause must use identifiers from the output data schema and DuckDB SQL syntax.
Note in the tool I do not bounce the data through a dataframe, I use a DuckDB relation and ArcPy insert cursor.
That's it, performant data generation into ArcGIS on demand. The tool accesses the latest release of Overture Maps Foundation places theme data, refreshed monthly.
Please do comment here with any issues.
