Convert GeoJSON to Feature (polyline) takes ages

[WumboB]

Hi folks,

I seem to have an issue with importing GeoJSON files to features (polyline) in ArcGIS pro (version 2.9.5). The JSON file is quite large (about 340 MB). I made sure the JSON file is on my local drive such that no netwerk delays are present. Still the import takes something between 8 and 16 hours.

I'm running this one a descent computer (win 10, Intel(R) Core(TM) i7-10850H CPU (6 core with hyper threading), 32 GB of RAM, a descent 1 TB SSD). When looking at the task manager while importing such an GeoJSON file, the CPU is not loaded at all (up to 25%), ArcGIS uses about 2 GB of memory and also the SSD is hardly transfering any data.

Does anyone know what might cause this long import times?

Kind regards,

Wumbo

[VinceAngelo · Accepted solution]

25% load on a 6-core system means that one CPU is fully engaged and another at 50%. Parsing a big text block like that isn't terribly efficient, and could easily suffer from poor performance (though 8-16 hours seems far too long).

How many features, with how many vertices/feature? What exact procedure are you using to convert the data?  Have you tried using Python to load the JSON to a dictionary then processing it from there?

- V

[WumboB]

Hi VinceAngelo,

Thank you for your quick response. Most of the time the ArcGIS itself only loaded the CPU about 15% according to the Task Manager of windows. So I doubt if your assumption regarding CPU core loads correct.

However, I had a look and, according to the attribute table, there are 128,811 features in the loaded (GeoJSON) databases. Actually, this particular case took eventually about 45 minutes so that's not too bad. In the past, I loaded different GeoJSON databases and these took 8 hours but they might have contained way more features.

Regarding your suggestion to use Python. I tried this and it seems to speed-up the process a lot indeed. Thanks you very much for this suggestion. I'll use this method in the future 🙂