The use of big tourism data

[SamSmith247]

Hello I am quite new to GIS and I have a question regarding the analysis of big data set.

Basically I was trying to download a big data set from ArcGIS hub ("OpenStreetMap Tourist Attractions for Europe" which has 1,387,754 records) but it took forever for the server to create the file for me to download it (I tried downloading multiple times and sometimes it said the file cannot be created or there is API issue, etc.) so I decided to open it directly with ArcMap and ArcGIS Pro in an attempt to export the data with them.

However, error messages kept popping up (or the progress may be stuck at certain %) when I tried clipping (as I actually only need the data within the UK boundary) or when I tried to export the whole data set so I can deal with it locally.

May I ask if this is an unsolvable problem when dealing with huge datasets? 

[ABishop]

Hello Sam,

Have you tried doing a selection by location within the UK boundary and then exporting the selection set only?  Also, sometimes pausing your data frame can enable you to perform geoprocessing tasks more easily due to the data not redrawing during the process. 

[SamSmith247]

I have tried clipping but it either won't produce a shapefile will all the data within the UK boundary (only some points at one corner are clipped) or return an error code 400 (sometimes together with the 000117 warning)... Is there a way to solve these? 

Thanks for your time!

[ABishop]

OK.  I am testing this on my end.  Which UK boundary are you using?

[SamSmith247]

Administrative boundary, with northern ireland

[ABishop]

This one?

[SamSmith247]

oh I mean i downloaded a shapefile with UK's boundary to do the clipping!

But I guess using your method will do as well?

[ABishop]

I pulled in both of the feature services from the living atlas into ArcGIS Pro.  Because the OSM_Tourism_EU layer is set to a scale visibility, I right-clicked on it and chose properties then in the general tab I made the "Out beyond (min scale)"  set to "<None>.  Then I performed the clip and received 983 records.

[SamSmith247]

I just adjusted the scale visibility of the OSM_Tourism_EU layer and tried to pull in the feature service of "United Kingdom Countries Boundaries 2021" as the clipping feature, but the clipped file returned with an empty attribute table with no records... No idea what is wrong...

[ABishop]

There appears to be an issue with the feature service in the living atlas.  Try getting one of the datasets here:

https://www.arcgis.com/home/item.html?id=69109c4fbbc54f1f9d6e18000031a5fd