topic Re: High memory usage with 50-600k point geometries in .NET Maps SDK Questions

High memory usage with 50-600k point geometries

Espen_RøvikLarsen — Tue, 10 Oct 2017 09:56:34 GMT

Hi,

The memory usage easily exceeds 2GB and can reach the magic number that crashes a 32bit application on Windows.

I see that I am not the only one experiencing this. I am using the recommended optimizations and I have tried different modes (static, dynamic). The graphics are added to a GraphicsOverlay and I am using a renderer to define symbolization (v10.2.7 / v100.0 / v100.1).

Will this issue be improved upon in a forthcoming release?

Re: High memory usage with 50-600k point geometries

dotMorten_esri — Tue, 10 Oct 2017 21:38:22 GMT

Have you tried using FeatureCollectionLayer with a FeatureCollectionTable instead?

Re: High memory usage with 50-600k point geometries

Espen_RøvikLarsen — Thu, 12 Oct 2017 19:11:22 GMT

Thanks for the tip. I tried it now - still 600MB for 160K items. The load time went from a few seconds to a minute. Probably due to the async API for adding a feature to the feature table - lots and lots of garbage collections.

I know the actual data is about 15MB in memory, so I am curious about what the runtime is doing.

Re: High memory usage with 50-600k point geometries

dotMorten_esri — Thu, 12 Oct 2017 20:25:31 GMT

Did you use the Add method that adds an entire array of them? If you do one-by-one yes it'll be slower. You can't really rely on the memory consumption - it could just be the GC decides it's not low on memory and busy doing other stuff, so will hold off on collecting memory.

Re: High memory usage with 50-600k point geometries

Espen_RøvikLarsen — Fri, 13 Oct 2017 11:07:48 GMT

I see, the load time would surely increase. But the main blocker is the memory usage. I am forcing GC collect, and I can see that most of the memory usage is unmanaged.

Re: High memory usage with 50-600k point geometries

dotMorten_esri — Fri, 13 Oct 2017 16:10:08 GMT

You should see the best performance with large datasets by using a runtime geodatabase

Re: High memory usage with 50-600k point geometries

Espen_RøvikLarsen — Thu, 19 Oct 2017 12:18:46 GMT

That would be an alternative yes, but as the data is coming over a generic REST call (non-ArcGIS) the orchestration would be a bit heavy.

From a programmers perspective I am really curious about how megabytes of data turns into gigabytes inmemory. Is there some kind of bitmap caching behind the scenes?

Re: High memory usage with 50-600k point geometries

dotMorten_esri — Thu, 19 Oct 2017 15:37:59 GMT

Is it possible for you to build a small stand-alone reproducer that reproduces this? (ie something that generates random graphics/features with similar a amount of geometries with similar complexity and attributes)

Re: High memory usage with 50-600k point geometries

Espen_RøvikLarsen — Thu, 16 Nov 2017 11:37:22 GMT

I am changing jobs and I will probably not work with ESRI software for a long time. In the process I do not have the time to make you an app.

That being said there is no complexity to this. It happens for point geometries in large volume.