Unable to Read Parquet Files in ArcGIS Pro 3.2

369
5
08-20-2024 12:53 PM
Labels (2)
RyanUthoff
Frequent Contributor

For some reason, I am unable to read any parquet files in ArcGIS Pro 3.2 when creating a multifile feature connection. Per the documentation here (https://pro.arcgis.com/en/pro-app/3.2/help/data/big-data-connections/big-data-connections.htm), multifile feature connections support parquet files. Yet, when I try to create a connection file with parquets, it says "No datasets discovered in subfolders. Choose a different source folder."

I know there is a specific folder structure I need to follow for to create a connection file, but I believe I'm doing it correctly because it works just fine if I use shapefiles. But if I use the same structure for parquet files, it doesn't read them.

This is the first time I've done anything with parquet files, so I'm probably doing something wrong. But does anyone know why the tool isn't detecting the parquet files? 

Tags (3)
0 Kudos
5 Replies
DanPatterson
MVP Esteemed Contributor

You saw the documentation in here then?

What is a multifile feature connection?—ArcGIS Pro | Documentation

and its subsections.


... sort of retired...
0 Kudos
RyanUthoff
Frequent Contributor

Correct. I structured it exactly like it said and it doesn't detect the parquet files. However, if I add a subfolder containing shapefiles into the source folder, it picks up the shapefiles, but not the parquet files. If I use the actual Create Multifile Feature Connection tool, it says it succeeded with the shapefile, but it failed with the two parquet subfolders. Unfortunately, the tool provides no additional error messages or reasons on why it failed. This is literally all the information it gives me.

RyanUthoff_0-1724210666134.png

 

0 Kudos
NanaDei
Esri Contributor

There are plans to support GeoParquet in ArcGIS Pro. As you noted, GeoParquet isn't supported with the Multifile Feature Connection. I'll send you a message for us to coordinate and obtain the two GeoParquet files for review.

0 Kudos
RyanUthoff
Frequent Contributor

Update: I've partially figured out the issue. In my testing, apparently ArcGIS Pro can only read "regular" parquet files and not GeoParquet files, even though they have the same file extension.

I find it very strange that Esri does not support GeoParquet files considering ArcGIS Pro is geospatial software, yet they only support non-spatial parquet files?

Furthermore, nowhere in Esri's documentation (that I could find, feel free to prove me wrong) does it state that it only supports non-spatial parquet files. Per Esri's documentation (What is a multifile feature connection?—ArcGIS Pro | Documentation), it states that .parquet is a supported data format, yet gives no limitations that it doesn't support GeoParquet.

If anyone has any more input about this, please feel free to comment!

Edit: QGIS natively supports GeoParquet files. Drag and drop the file and it just works. I would like to see this functionality implemented into ArcGIS Pro.

0 Kudos
Robert_LeClair
Esri Notable Contributor

So in reviewing the internals, I can confirm that this was reported by a customer May 2024 and is a BUG in the software.  I do not see a BUG number as of yet but the team is aware of it.