Possible memory leak

[LarsDomesjö]

Hello,

I've been using the ArcGIS Pro SDK to write an application which updates a Geodatabase.

It seems to be accumulating a lot of memory usage over time though, and it doesn't seem to be my own code which is accumulating, but rather heap memory.

I tried isolating the relavant parts into a smaller PoC for you to observe the issue yourselves.

Basically what this application does it:

• Creates well known binaries (Wkb) from Gml using NetTopologySuite
• Imports the Wkbs into ESRI Geometries using the GeometryEngine
• Creates a union of the geometries using the GeometryEngine
• Stores the Feature (with Geometry) in a Geodatabase

In the real application I get about 20GB of memory usage after storing about 4 million features in the Geodatabase.

I've used the application .NET Memory Profiler to try to look into what's using all the memory and it seems to be the Union-call which isn't getting cleaned up?

Looking at the heap I have alot of these for example:

ntdll!RtlAllocateHeap( HANDLE, DWORD, SIZE_T )
ucrtbase!_malloc_base()
pe!pe_xd_find_patt()
pe!pe_spheroid_ext_new_errext()
pe!pe_spheroid_clone()
pe!pe_datum_clone()
[Managed to native transition]
ArcGIS.Core!ArcGIS.Core.Internal.IGeometryEngineIOP.GEUnionArray( IntPtr[],SpatialReferenceTiny,string,string,ref bool )
ArcGIS.Core!ArcGIS.Core.Geometry.GeometryEngine.Union( IEnumerable<Geometry> )
SampleApp!BALSynchronizer_v2.Helpers.ArcGisHelper.ESRIGeometryFromWKBs( IEnumerable<Wkb> ) ArcGISHelper.cs Line 24
SampleApp!SampleApp.Program.InsertGdb( RowBuffer,Building ) SampleApp.cs Line 117
SampleApp!SampleApp.Program.StoreInGdb( string,List<Building> ) SampleApp.cs Line 135
SampleApp!SampleApp.Program.Main( string[] ) SampleApp.cs Line 187

I cant see any way to dispose the ESRI Geometries manually so I assume that they should be automatically disposed somehow?

Got any suggestions for me? 🙂

Have a great weekend!

/Lars

[KirkKuykendall1]

If you comment out (or remove) the code where you're writing/updating geodatabase, do you still get a leak?

[LarsDomesjö]

I did try this and I am not getting a memory leak if I remove the parts writing/updating the database

[KirkKuykendall1]

Did you try doing the union with NTS instead of Esri?

https://nettopologysuite.github.io/NetTopologySuite/api/NetTopologySuite.Operation.Union.UnaryUnionO...

[LarsDomesjö]

That's a good suggestion, I will try this and see if it resolves my issue, just did a quick try and I seem to be getting faulty polygons when using NTS Union, will need some more investigation. 

[LarsDomesjö]

Seems like NTS-Union is causing some Z-values with NaN for some polygons for some reason.

Created an ugly SequenceFilter for NTS which updates all Z-values with "0.001" just for testing purposes and can confirm that the memory leak is not there when doing the Union with NTS instead of the ArcGIS Pro SDK, so the leak seems to be somewhere in the Union-call in the ArcGIS Pro SDK.

[KirkKuykendall1]

After calling NTS Union, are you able to insert/update without leaking memory?

If so, I suppose Esri's GeometryEngine hangs onto a reference to the unioned geometry in such a way that it doesn't get released after the insert/update.

[LarsDomesjö]

yes. after doing a Union with NTS and exporting it to Wkb, importing the Wkb into an ESRI geometry and doing insert/update no memory leakage appears to be occurring anymore, so seems to be related to doing a Union with the ArcGIS Pro SDK