cache*C:\dev\symbols;SRV*C:\dev\symbols*http://msdl.microsoft.com/download/symbols;SRV*C:\dev\symbols*http://downloads2.esri.com/Support/symbols/
Hello ESRI developer community,
I've been working with the ESRI ArcObjects for many years now, mainly on an ETL (Extract/Transform/Load) .NET application. This application can run for a long time to transfer data from an ESRI Geodatabase to the target system (for example, 3 consecutive days).
We recently had a situation where our application crashes with an "OutOfMemory" exception. To find the memory leak, we profiled our application with SciTech .NET Memory Profiler. Since ArcObjects is a COM based library and that the application profiler can't give the detail about the instanciated objects, I had to comment the code of our application to narrow the possibilities of the objects used/function calls that was causing this memory leak.
I ended up isolating the IFeatureWorkspace.OpenFeatureClass function being the main source of the problem. This function is called multiple times during the 2-3 days of process and it seems like the FeatureClass object returned is never released.
I read multiple articles about memory leak with ArcObjects and tried everything that was suggested to release the FeatureClass object, i.e.:
The .NET functions:
Marshal.ReleaseComObject
Marshal.FinalReleaseComObject
The ArcObject ComReleaser object with the following logic:
Using comReleaser As New ComReleaser()
esriFeatureClass = _featureWorkspace.OpenFeatureClass(featureClassName)
comReleaser.ManageLifetime(esriFeatureClass)
End Using
All this combined with the .NET Garbage Collector functions:
GC.Collect
GC.WaitForPendingFinalizers
I did a small application that simulates the same routine that our ETL application does to help for the profiling and also validate that the behavior is the same. It turns out it has the same behavior, the amount of memory allocated is similar. With the routine of opening the feature class repeated 450 times, there is 15 Mb that stays allocated at the end (SciTech Memory Profiler screenshot attached). So when the process runs for 3 days, there is more than 200 Mb that stays allocated just for those FeatureClass alone.
I attached the Visual Studio 2010 project that does the routine, so if you want to take a look at the code to see if i'm missing something. Right now, I'm out of solution to fix that. Any help would be greatly appreciated.
I'm working with ESRI ArcGIS 9.3 (9.3.0.1770 is the version number of the ESRI DLLs in Visual Studio)
Thanks in advance,
Joel
Imports ESRI.ArcGIS.ADF Imports ESRI.ArcGIS.DataSourcesFile Imports ESRI.ArcGIS.DataSourcesGDB Imports ESRI.ArcGIS.esriSystem Imports ESRI.ArcGIS.Geodatabase Imports ESRI.ArcGIS.Geometry Imports ESRI.ArcGIS Imports System.Runtime.InteropServices Public Class Form1 Private _featureClassNames As List(Of String) Private _featureWorkspace As IFeatureWorkspace Private Sub InitializeFeatureClassNames() _featureClassNames = New List(Of String) _featureClassNames.Add("ARCFM.Airport") _featureClassNames.Add("ARCFM.Bridge") _featureClassNames.Add("ARCFM.Building") _featureClassNames.Add("etc...") End Sub Private Sub ESRI_Connect() Dim propSet As New ESRI.ArcGIS.esriSystem.PropertySet Dim wsFactory As Geodatabase.IWorkspaceFactory = Nothing Try wsFactory = New DataSourcesGDB.SdeWorkspaceFactory() propSet = New ESRI.ArcGIS.esriSystem.PropertySet() With propSet .SetProperty("SERVER", "put SDE server name here") .SetProperty("INSTANCE", "put SDE service port here") .SetProperty("DATABASE", "put Oracle database service name here") .SetProperty("USER", "put Oracle user name here") .SetProperty("PASSWORD", "put Oracle user password here") .SetProperty("VERSION", "SDE.DEFAULT") End With 'Try to access the ESRI source data _featureWorkspace = wsFactory.Open(propSet, 0) Finally If propSet IsNot Nothing Then Marshal.ReleaseComObject(propSet) If wsFactory IsNot Nothing Then Marshal.ReleaseComObject(wsFactory) End Try End Sub ''' <summary> ''' This is the main loop where the Workspace.OpenFeatureClass function is called. ''' </summary> ''' <remarks></remarks> Private Sub LoopOpenFeatureClasses() For Each featureClassName As String In _featureClassNames Dim esriFeatureClass As Geodatabase.IFeatureClass = Nothing Try esriFeatureClass = _featureWorkspace.OpenFeatureClass(featureClassName) Catch ex As Exception Finally If esriFeatureClass IsNot Nothing Then Marshal.ReleaseComObject(esriFeatureClass) End Try Next End Sub ''' <summary> ''' Initializes the application to use the ArcObjects libraries. ''' </summary> ''' <remarks></remarks> Public Sub InitializeArcObjects() Dim aoInitialize = New AoInitialize() Dim licenseStatus As esriLicenseStatus = aoInitialize.Initialize(esriLicenseProductCode.esriLicenseProductCodeArcView) End Sub Private Sub btnTest_Click(sender As System.Object, e As System.EventArgs) Handles btnTest.Click Try InitializeFeatureClassNames() InitializeArcObjects() ' initializes _featureWorkspace ESRI_Connect() For i As Integer = 1 To 450 LoopOpenFeatureClasses() Next Catch ex As Exception Finally If _featureWorkspace IsNot Nothing Then Marshal.ReleaseComObject(_featureWorkspace) End Try End Sub End Class
One thing that you might be able to do is to try to work around the problem by giving ArcMap more memory.
You can do that by setting the large address aware bit on ArcMap.exe and then run your problem on a 64 bit operating system.
Normally, ArcMap gets 2 GB and the OS gets 2 GB of virtual memory. If you do the above then the OS can move elsewhere leaving ArcMap with 4 GB of memory.
wsFactory = New DataSourcesGDB.SdeWorkspaceFactory()
Dim factoryType As Type = Type.GetTypeFromProgID("esriDataSourcesGDB.SdeWorkspaceFactory") Dim workspaceFactory As IWorkspaceFactory = CType(Activator.CreateInstance(factoryType), IWorkspaceFactory)
Thank you for your help, Jason.
I followed the procedure with the Windows Debugging Tools. To give you an idea of my test, here is the loop that i executed right after I start marking the memory:
For i As Integer = 1 To 5
[INDENT]
'There is about 21 feature classes in that "_featureClassNames" collection
For Each featureClassName As String In _featureClassNames
[INDENT]Dim esriFeatureClass As Geodatabase.IFeatureClass = Nothing
esriFeatureClass = _featureWorkspace.OpenFeatureClass(featureClassName)
Marshal.ReleaseComObject(esriFeatureClass)
[/INDENT]Next
[/INDENT]Next
Then took the second mark after and did the difference. I attached the result file to the thread (results.txt). I did a research with the "ESRI" keyword in the text file and i can find a couple of places where we can see the trace of the memory allocated.
So do you think there is something wrong in my code? Or that the ESRI objects are not released and that I don't have any control on this?
Thanks again, very appreciated!
Joel
I would avoid using Esri's ComReleaser object--it is only useful in very specific situations and will cause problems in every other situation.