Memory usage for ArcMap - System.OutOfMemoryException //c#

[MarcinDruzgala]

Hi the problem is common as far as I know from reading post from this and other forums(memory usage for arcgis).
I'm developing a add-in for ArcMap that checks electricity flow in Geometric Network. I've created a loop for over 14 000 start points for my analysis. I'm using Utility Network Analyst and the FindFlowElements method, the algorithm is preety simple:
1. Get Geometric Network IDs for starting point(FCID, FID, subID)
2. Configure Geometric Network
3. FindFlowElements -> as a result i get junctionEIDs and edgeEIDs
4. Create selection from the result
5. Analyze the selection
6. Log info
7. Clearing the result
8. Clearing the flags for network utility
9. Clearing the selection for map
10. Partial refresh
11. Marshal.FinalReleaseComObject(traceFlowSolver); // testing it because of the out of memory exception
and again from the point 1 for next point.

So when we tested this we encountered a lot of errors. I have the log info we are writing to txt file it will be easier to illustrate the problem with example:
INFO 2013-03-19 12:56:12,302 â?? Called FindFlowElements method 1374 time(s)
Number of objects in List<T> SN = 1316, memory usage = 0,06808 MB
Number of objects in List<T> nN = 10068, memory usage = 0,43913 MB
Station number 7545
Process name: ArcMap -> memory use 834,5586 MB || GC TotalMemory -> 80,0527 MB
Processes list:
IEXPLORE ||memory use-> 31,6836 MB
FSM32 ||memory use-> 15,6992 MB
SVCHOST ||memory use-> 26,4336 MB
EXPLORER ||memory use-> 85,4766 MB
SMSVCHOST ||memory use-> 37,5313 MB
DSTERMSERV ||memory use-> 27,0664 MB
FSSM32 ||memory use-> 167,2422 MB
SVCHOST ||memory use-> 48,6758 MB
SVCHOST ||memory use-> 82,5117 MB
FSHDLL32 ||memory use-> 52,4531 MB
BCU ||memory use-> 85,0273 MB
ORACLE ||memory use-> 771,1250 MB
SPLWOW64 ||memory use-> 15,6484 MB
IEXPLORE ||memory use-> 15,7031 MB
ARCMAP ||memory use-> 964,2070 MB
SEARCHINDEXER ||memory use-> 50,6445 MB
TNSLSNR ||memory use-> 36,6602 MB
WINWORD ||memory use-> 22,3398 MB
FSGK32 ||memory use-> 15,4453 MB
SOFFICE.BIN ||memory use-> 16,2188 MB
AUDIODG ||memory use-> 18,5859 MB
IEXPLORE ||memory use-> 19,3359 MB
SPOOLSV ||memory use-> 17,4063 MB
WMPNETWK ||memory use-> 30,1172 MB
SVCHOST ||memory use-> 29,7500 MB
SVCHOST ||memory use-> 234,8750 MB
DWM ||memory use-> 45,6406 MB
SVCHOST ||memory use-> 25,8594 MB
CONNECT.SERVICE.CONTENTSERVICE ||memory use-> 51,8594 MB
Memory usage sum = 2911,0120 MB
=================================================================================================

INFO 2013-03-19 12:56:14,954 â?? Called FindFlowElements method 1375 times
Number of objects in List<T> SN = 1316, memory usage = 0,06808 MB
Number of objects in List<T> nN = 10072, memory use = 0,43931 MB
Station number 7397
Process name: ArcMap -> memory use 838,7539 MB || GC TotalMemory -> 84,8050 MB
Processes list:
x
x
x
x
Memory usage sum = 2899,1410 MB

The FindFlowElementos method was called 1375 times(14 000 expected...) and ArcMap already uses ~840 MB(when it reaches ~2GB arcmap throws an error) of memory and it's growing with every iteration. Sometimes it drops with few MB but no to often so i will propably have to do the cleaning manually, but the question is how?

I will explain now how i calculated/got the memory usage:
1.For ArcMap:

private const float mbyte = 1048576;
Process proc = Process.GetCurrentProcess();
string privMemSize = (proc.PrivateMemorySize64 / mbyte).ToString("0.0000");

2.For Garbage Collector:

string gcTotalMem = (GC.GetTotalMemory(false) / mbyte).ToString("0.0000");

3.Memory usage for Lists<T> I'm calculating with method I took from StackExchange:

private static string GetObjectSize(object TestObject)
{
 try
 {
  BinaryFormatter bf = new BinaryFormatter();
  MemoryStream ms = new MemoryStream();
  byte[] Array;
  bf.Serialize(ms, TestObject);
  Array = ms.ToArray();
  return ((float)Array.Length / mbyte).ToString("0.00000");
 }
 catch (Exception ex)
 {
  log.Error("========================================================================\n"
   +ex.Message + "\n" + ex.StackTrace + "\n"
   + "========================================================================\n");
  return "error";
 }
}

The question is how I can handle the increasing memory usage for ArcMap.exe? Anyone got idea?

[MarcinDruzgala]

Ok Jason I did what you've told me to do and here is the result. I hope this time everything is ok : > Could you please take a look at this logs?

The CLRPRofiler can't connect to arcmap process(?).
"clr profiler waiting for application to start common language runtime" even if i run my add-in nothing happens.

UMDH - this will take much more time for me to understand this. Will give it a shot maybe later if I don't find the solution now with LeakDiag.

Thanks in advance for your help!

[JasonPike]

Ok Jason I did what you've told me to do and here is the result. I hope this time everything is ok : > Could you please take a look at this logs?

The CLRPRofiler can't connect to arcmap process(?).
"clr profiler waiting for application to start common language runtime" even if i run my add-in nothing happens.

UMDH - this will take much more time for me to understand this. Will give it a shot maybe later if I don't find the solution now with LeakDiag.

Thanks in advance for your help!

We still seem to be limited to a stack depth of 5. Did you change the max stack depth to 32? It does look like you got some of the symbols resolved though.

I'd like to know what FeatureDataElements.dll contains. More importantly, we need to know what your code is using from FeatureDataElements.dll. That is one place that needs to be more particular about explicitly releasing COM objects. If we had more stack information, we could probably see what calls your code was making that ended up using the DLL. Another thing you can try is removing FeatureDataElements.dll from your project references (if it is in your project references) and rebuilding to see what lines of code no longer compile.

When I see a stack like the one below, I become concerned that there might be a memory leak in ESRI's product:

AfCore.dll!BString__BString
GpObjects.dll!GPSchemaBase__operator=
GpObjects.dll!GPSchemaBase__operator=
DECoreLib.dll!GPValue__Clone
GeoprocessingLib.dll!GPFeatureLayerBase__Assign

You indicated in a response to one of my posts in a different thread that you'd like to learn as much as possible about this. Check out the link below to better understand why this makes me suspicious:
http://msdn.microsoft.com/en-us/library/xda6xzx7(v=vs.80).aspx

It looks like an object is cloned, which allocates a BSTR that isn't being released. Did you set the OANOCACHE to 1 in your environment variables? If not, we can't trust this stack, but if you did, we should pursue this as well. Where is your code assigning a FeatureLayer? If the object has been properly released and is no longer used within the system, we shouldn't see that stack.

Next up, we have something that is using FdaCore.dll, which in turn is creating a string that is still in memory at the point you last marked. Again, I really need to see more of the stack so we can determine how your code causes this to be called in the first place. I have the same concern here as I do with the previous one: someone allocated a string and didn't release it. Either you have a pointer the object that is keeping it in memory, or you are disconnected from the object and it didn't clean up after itself properly.

MSVCR90.dll!malloc
MSVCR90.dll!operator new
AfCore.dll!String__CreateStringData
AfCore.dll!String__String
FdaCore.dll!??????

Below is another stack that we need to understand how your code is causing to occur. I'm not familiar with NetEngine80.dll, but a few Google searches suggest that it may be related to a doubly-linked list being maintained by one of the ESRI objects.

MSVCR90.dll!malloc
NetEngine80.dll!NE__IDCache__IsEqual3Tuple
NetEngine80.dll!NE__List<NE__LRUNode>__Disconnect
NetEngine80.dll!NE__List<NE__LRUNode>__Disconnect
NetEngine80.dll!NE__List<NE__LRUNode>__Disconnect

As for the CLRProfiler problem you're seeing, you need to get the version that matches the .NET version that your code uses. Also, you'll want to use the x86 version, even if you have an x64 operating system. Regardless of what it reports about connecting, try to create some logs and see if it collected anything.

If you can't get LeakDiag to capture stacks larger than 5 deep, it is time to move on to UMDH. You can see from what I've written above that the granularity is only good for hints, but we want specifics, if possible.

If you can create a small project that reproduces the issue and post it on the forum, I'd be happy to analyze it for you. Otherwise, continue to analyze your code based on the hints and try to identify the calls your code is making that result in the memory allocations from LeakDiag.