Out of Memory problem in VB.NET not in VBA ? why?

[martinmaretta]

hi i have one problem: i have 4 big arrays created as pixelBlock.PixelData in VBA, but when i try to rewrite this code to VB.NET i have Out of Memory exception. So i Try some debug.print Debug.Print(GC.GetTotalMemory(False)) and i found that max memory which i can use in VB.NET is about 401306192  bytes.. (this occur when i create base command in ArcMap and open Form when i have code like this            
            Dim pZ1(5060, 4951) As Single
            Debug.Print(GC.GetTotalMemory(False))
            Dim pZ2(5060, 4951) As Single
            Debug.Print(GC.GetTotalMemory(False))
            Dim pZ3(5060, 4951) As Single
            Debug.Print(GC.GetTotalMemory(False))
            Dim pZ4(5060, 4951) As Single
            Debug.Print(GC.GetTotalMemory(False))
            Dim pZ5(5060, 4951) As Single
            Debug.Print(GC.GetTotalMemory(False))

but when i create normal windows Form application (without ArcObjects) and have the same code..i can use 3 time more memory (about 1102991660 bytes)

do you know where is a problem? is becase of using unmanaged COM object? or what? i tried to use GC.AddMemoryPressure method but nothing hapend (maybe wrong usages).

thanks for all your´s ideas..

[AlexanderGray]

I have had problems with pixelblocks and .Net vs VB 6.  The problem is the pixelblock is a COM array that has to be converted to a .net safe array.  This takes a lot of memory and is slow.  ArcObjects is 32 bits so you will be limited in the amount of memory that can be allocated to the process.  ArcObjects is also very memory intensive so it will eat up the memory very fast.

Potential solutions include rewriting the pixelblock code in C++, you can make it a library to handle just those functions called from your .net code.  Another solution is to limit the size of a pixelblock to something like 100X100 and process the raster in sections.  This option will still be slower than C++ due to the COM/.Net interop but should work.  The best is a combination of both because even in C++ you can run into memory problems creating very large pixelblocks (the limit will just be higher.)

If using .NET I also suggest you release the memory using COMReleaser after processing each pixelblock.  Because of the interop calling GC will not clean up the memory.

[RichardWatson]

ArcMap is strictly a 32 bit process.  That means that the number of bytes which can be addressed is 2 ** 32 or 4 GB.  When you load a .NET assembly/DLL in a 32 bit process then it runs as a 32 bit DLL.

If you write a stand-alone pure .NET program and run that program on a 64 bit OS then the number of bytes which can be addressed is 2 ** 64.

None of this has anything to do with Garbage Collection.

[AlexanderGray]

Actually in ArcGIS 9.3.1 because of addressing issues, a realistic limit for the whole process is around 1.5GB.  In ArcGIS 10 they have changed the memory allocation and you can get closer to 3GB.  Still 32bit though.

[martinmaretta]

thanks for replay I don´t use Safe.array but I cast pixeldata directly to data type :
            Dim pObjectArray(,) As Single
            pObjectArray = CType(pPixelBlock.PixelData(0), Single(,))
by this way a code is very fast (much more then whit safe.array)..and I still don´t understant why my code is running under VBA. Is because there is no conversion from COM to .NET which is memory consuming?

[RichardWatson]

Actually, your best shot is to run ArcMap version 10 on a 64 bit OS with lot's of physical RAM.  In this configuration ArcMap will have 4 GB of RAM available to it because the OS can move elsewhere.

This is possible because ArcMap sets the large address aware bit in version 10.  Of course, if you are running an earlier version of ArcMap then you can set this bit yourself.  Note that this is not the same as the \3GB Windows boot option which ESRI does not recommend.

[AlexanderGray]

rlwatson is right large address will probably fix the problem.  You still have to go through the COM .NET interop even if you are just converting the pixel value directly.  In my experience no matter how much memory you have someone will invariably throw some image at it that it just can't handle regardless of interop so chopping up the image into sections the machine can handle is always a good idea.  The size that the machine can handle depends on the machine, software version, interop, etc.  That is why VBA and C++ will handle larger pixelblocks than .NET.  Large address awareness will make a difference too.