Parquet is widely used in mainstream data processing, and GeoParquet (https://geoparquet.org/) is supported by many GIS tools. Overture Maps data is distributed in the format, and BigQuery also recently started supporting it as well. It's possible to use it in the ArcGIS ecosystem, through the data interoperability extension and through notebooks (like in https://community.esri.com/t5/etl-patterns-blog/go-cloud-native-overture-geoparquet-from-object/ba-p/1371965) , but it'd be really great if anyone with ArcGIS Pro could easily drag and drop a (geo)Parquet file just like a shapefile or geopackage.
Well, it is partly implemented. Apache Arrow as the in-memory representation? Web publishing AGOL/Enterprise?
Thank you so much! Really awesome to see this come to fruition.
Thank you everyone for your kudos and comments on this Idea! ArcGIS Pro 3.5 introduces support for the parquet format. For more information on this support and what the roadmap for parquet in ArcGIS Pro looks like, please check out the Enhancing Support for Parquet Files blog linked below.
Enhancing Support for Parquet Files
It's now supported in ArcGIS Pro v3.5 I believe .. I hope soon it can be supported in ArcGIS Online / Enterprise as well as more Docs on how to load/Export from and to ArcGIS Pro
EDIT: Also, natively supporting Arrow in memory could be more efficient than relying on caching or feature classes. The core idea behind Parquet is to support Apache Arrow by storing data efficiently on disk or object storage.
By doing so, you contribute to the open data ecosystem, allowing Arrow data to be accessed in memory from anywhere. Any language or library can easily interact with it .. but currently, it's locked down .. correct me if I am wrong
That's great news to see that it's been added to the product plan. Thank you.
GeoParquet is an essential format for working with large geospatial datasets in the cloud.
Essential capability for modern GIS data integration.
"Under consideration" this is great news!
Yes, our office would very much like to see this.
Yes please! As format in hub /ability to publish feature layer from one would be nice too.
It's overdue
Completely agree - sooner the better
It's time.
Sign in to post, follow content, and more. New here? Register for free.
