Releasing memory using Marshal.FinalReleaseComObject

[USFS_AGOLAdministrator]

Hi all,

I know this is alot of code, but you have to see the whole thing understand the problem. The code below is doing a Reverse Eliminate of sorts by looping through a selected set of sliver polygons, doing a spatial selection to find out what polygons are touching the slivers, getting the 2nd largest polygon that touches the sliver, and merging the sliver to the second largest polygon by unioning them together, and deleting the original sliver polygon.

It works beautifully well, except it bombs out after processing ~900 slivers with a memory error. I've tried everything I can think of to release the memory, as you will see below. Can anyone see anywhere I can release more memory?

Try
     Using comReleaser As ESRI.ArcGIS.ADF.ComReleaser = New ESRI.ArcGIS.ADF.ComReleaser()
Dim pFeat As IFeature = Nothing
Dim pEnumFeat As IEnumFeature = CType(pMap.FeatureSelection, IEnumFeature)
'Dim pEnumFeat As IEnumFeature = CType(pFLayer.SelectionSet, IEnumFeature)
pEnumFeat.Reset()
Dim pFCur As IFeatureCursor = Nothing
Dim pSF As ISpatialFilter
pSF = New SpatialFilter
Dim pTopoOp As ITopologicalOperator
Dim pLargestAdj As IFeature = Nothing
Dim sliverFeat As IFeature = Nothing
sliverFeat = pEnumFeat.Next

Do Until sliverFeat Is Nothing
'this is really the smallest
'---------------------------------------------------------------------
pLargestAdj = GetLargestAdjacent(sliverFeat, True, pSF, pFCur, pFClass)
'---------------------------------------------------------------------

If Not pLargestAdj Is Nothing Then

pTopoOp = CType(pLargestAdj.ShapeCopy, ITopologicalOperator)
pLargestAdj.Shape = pTopoOp.Union(sliverFeat.ShapeCopy)

updateFeature(m_pFLayer.FeatureClass, pLargestAdj.OID, pLargestAdj.Shape)

sliverFeat.Delete()

Marshal.FinalReleaseComObject(sliverFeat)
Marshal.FinalReleaseComObject(pTopoOp)
Marshal.FinalReleaseComObject(pLargestAdj)

Else
Debug.Print("nothing adjacent to: " & sliverFeat.OID)
End If

nCurRecNo = nCurRecNo + 1
My.ArcMap.Application.StatusBar.Message(0) = nCurRecNo.ToString & "/" & n.ToString & " records complete."

Application.DoEvents()

comReleaser.ManageLifetime(sliverFeat)

sliverFeat = pEnumFeat.Next
Loop

'Stop editing and save the edits
pEditor.StopEditing(True)
Runtime.InteropServices.Marshal.FinalReleaseComObject(pLargestAdj)
End Using

Catch ex As Exception
MsgBox("Error: " & ex.StackTrace, , "RElimAddinPB_onClick")
End Try

---------------------------------------------------------------------------------------------------------------------------------------------
'!!!!!!!!!!!!!!!!!!!!!!!This function is where it bombs out after ~900 records!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Function GetLargestAdjacent(ByVal pFeat As IFeature, ByVal getSecondLargest As Boolean, ByVal pSF As ISpatialFilter, ByVal pFcur As IFeatureCursor, ByVal pFC As IFeatureClass) As IFeature

Try

pSF.Geometry = pFeat.Shape
pSF.SpatialRel = esriSpatialRelEnum.esriSpatialRelIntersects

pFcur = pFC.Search(pSF, False)

If pFC.FeatureCount(pSF) < 3 Then 'This is set if there is only polygons on one side of the sliver
getSecondLargest = False
End If

Using comReleaser As ESRI.ArcGIS.ADF.ComReleaser = New ESRI.ArcGIS.ADF.ComReleaser()

Dim pFeat2 As IFeature = Nothing, pLargestFeat As IFeature = Nothing
comReleaser.ManageLifetime(pFeat2)
comReleaser.ManageLifetime(pFcur)
Dim dMaxArea As Double = 0
pFeat2 = pFcur.NextFeature

Do Until pFeat2 Is Nothing 'This loop executes everytime, no matter if there are slivers on both sides or not

If Not pFeat2 Is pFeat Then
If pLargestFeat Is Nothing Then
pLargestFeat = pFeat2

dMaxArea = GetArea(CType(pFeat2.Shape, IArea))

Else
If GetArea(CType(pFeat2.Shape, IArea)) > dMaxArea Then
pLargestFeat = pFeat2
dMaxArea = GetArea(CType(pFeat2.Shape, IArea))
Runtime.InteropServices.Marshal.FinalReleaseComObject(pLargestFeat)
Runtime.InteropServices.Marshal.FinalReleaseComObject(pFeat2)
End If
End If
End If

pFeat2 = pFcur.NextFeature
pFcur.Flush()

Loop

Marshal.ReleaseComObject(pFcur)
'comReleaser.Dispose()

If getSecondLargest Then 'This loop will execute only if there are polygons on both sides of the sliver
'pFCur = pFC.Search(pSF, False)
pFcur = m_pFLayer.FeatureClass.Search(pSF, False)

Dim pSecondLargestFeat As IFeature = Nothing
Dim dSecondMaxArea As Double = 0

pFeat2 = pFcur.NextFeature
Do Until pFeat2 Is Nothing
comReleaser.ManageLifetime(pFcur)
If Not pFeat2 Is pLargestFeat Then
If Not pFeat2 Is pFeat Then
If pSecondLargestFeat Is Nothing Then
pSecondLargestFeat = pFeat2
dSecondMaxArea = GetArea(CType(pFeat2.Shape, IArea))
Else
If GetArea(CType(pFeat2.Shape, IArea)) > dSecondMaxArea Then
pSecondLargestFeat = pFeat2
dSecondMaxArea = GetArea(CType(pFeat2.Shape, IArea))
End If
End If
End If
End If

pFeat2 = pFcur.NextFeature
pFcur.Flush()
Loop

pLargestFeat = pSecondLargestFeat

End If

Marshal.ReleaseComObject(pFcur)
GetLargestAdjacent = pLargestFeat
'comReleaser.Dispose()

End Using

Catch ex As Exception
MsgBox(ex.Message, , "Ex routine")
MsgBox(ex.StackTrace, , "Ex routine")
GetLargestAdjacent = Nothing
End Try


End Function

----------------------------------------------------------------------------------------------

Private Sub updateFeature(ByVal fc As IFeatureClass, ByVal oid As Integer, ByVal newGeom As IGeometry)
Dim queryFilter As IQueryFilter = New QueryFilter() With { _
.WhereClause = Convert.ToString(fc.OIDFieldName) & " = " & oid.ToString() _
}

Dim featureCursor As IFeatureCursor = fc.Update(queryFilter, False)
Dim featureToUpdate As IFeature = featureCursor.NextFeature()

featureToUpdate.Shape = newGeom
featureCursor.UpdateFeature(featureToUpdate)
Runtime.InteropServices.Marshal.ReleaseComObject(featureToUpdate)
featureCursor.Flush()
End Sub

--------------------------------------------------------------------------------------------------------------

Function GetArea(ByVal pArea As IArea) As Double
GetArea = pArea.Area
End Function

[USFS_AGOLAdministrator]

Have you tried trouble shooting the issue  by  monitoring the memory usage in taskmgr while stepping through each statement of your code?

Yes, it stays between50-60% cpu usage the whole time, until it processes around 900 slivers.  Even at the point it bombs out, it never gets higher than that.  Yet it still errors out with a memory issue.

[sapnas]

What aboout Memory in the processes tab of taskmgr? it usually spikes up and down while stepping through the code.

[AlexanderGray]

If you have a two core machine, sounds like your process is tying up one core 100%.  Also ArcGIS desktop and engine are 32 bit.  In ArcGIS 10, it uses large addressing so ~3 GB is the max RAM it can allot to the one process, in ArcGIS 9.3 it is ~1.5GB.  If you monitor the amount of memory used you see if you get the error after you hit those limits.  You can also do tests with less data and monitor the memory and see when it gets released after each change without having to run the code all the way to failure each time you try something new.

[USFS_AGOLAdministrator]

Was this before you added all the ComReleaser stuff? It seems weird that you are creating an object in the main loop and releasing it in a sub function. Maybe create the feature cursor object in GetLargestAdjacent() and then force garbage collection in the main loop using this code:

GC.Collect()
GC.WaitForPendingFinalizers()

More information on the GC class here:
http://msdn.microsoft.com/en-us/library/system.gc.aspx

That was the trick!  Thank you so much for this post, and for everyone else who chipped in.  This tool will be used nation-wide, and will have a huge impact.  Thanks again!

[KenBuja]

I would get rid of using the ComReleaser altogether.  Here is the proper way to get a cursor and loop through it as well as clean up the objects that it uses:

Neil, why do you suggest not using ComReleaser?

[AlexanderGray]

that's great!  You might consider posting the final code so everyone on the forum could benefit from the solution.

Also, if you call gc.collect and waitForPendingFinalizers in main loop, it might slow you down unnecessarily.  If your previous code worked for 900 features before running out of memory, maybe calling the collect and waitForPendingFinalizers every 100 features or more might be just as good and much faster. I would be interested in knowing what the difference would be.

Also I was wondering about Neil's comment about esri's ComReleaser, I use it all the time and never really thought about it, it seems to work ok (I like to use it with a 'using' block.)

[sapnas]

I second that alexander, GC.Collect slows down the process. Also, if the object that is causing memory leak is not marked for garbage collection then GC.Collect wont work. I had cases where I had to literally remove GC.Collect.

[RichWawrzonek]

Glad that worked for you Paul.  As Alex mentioned you'll get better performance to only call GC.Collect() when necessary and not at every feature iteration.

[NeilClemmons]

Neil, why do you suggest not using ComReleaser?

I don't think that the ComReleaser is any help when using objects within loops.  When looping through a cursor, the feature or row object needs to be released before assigning it to the next feature or row in the cursor.  According to the documentation, the ComReleaser doesn't actually clean anything up until the object reference goes out of scope.  These objects within the loop are not out of scope so I don't think any cleanup occurs as the loop iterates.  It's only after the loop that the cleanup occurs so the loop has resulted in possibly thousands or millions of objects waiting to be released.  Releasing these objects yourself within the loop allows any garbage collection that occurs while the loop is running to reclaim those objects.  Also, all the ComReleaser does is perform the equivalent of calling FinalReleaseComObject.  So, in reality, it's just more overhead for something you can easily do yourself.  It can also be dangerous.  For instance, if you get a feature layer reference from the map you don't want to call FinalReleaseComObject on it because that will release the map's reference to it in addition to releasing your own variable's reference.  If I understand the documentation correctly, the ComReleaser would do just that.  What you actually want to do is call ReleaseComObject vs. FinalReleaseComObject.  Calling these methods yourself should make you think about what you are doing and decide which of the methods is the correct one to call.  This, of course, is just my opinion.  I'm not advising against using the ComReleaser class, just saying it isn't really needed.

[AlexanderGray]

According to the documentation, the ComReleaser doesn't actually clean anything up until the object reference goes out of scope

That is interesting I have never seen that documentation.  The documentation I have seen specifies that the ComReleaser uses ReleaseComObject internally (not FinalReleaseComObject mind you...)  I wonder if you declared the variable inside the loop, that should make the variable go out of scope at each increment (.NET can some times be fickle about that sort of thing, however.)
It seems to me the call to FinalReleaseComObject instead of ReleaseComObject (or ComRelease, which is the same thing) makes the biggest difference here.  That and forcing garbage collection.  You would expect the garbage collection to kick in automatically and manage the memory itself though, sort of the point of garbage collection in the first place.

I use ComReleaser's manageLifetime method most often with cursors and the using block.  That way I can call manageLifetime the next line after I instantiate the cursor and that cuts down a lot on forgetting it 20 or 30 lines lower, when I am done with the object.   But I will keep this in mind if I have to release resources for memory management purposes...