Insert feature causes Out of memory exception

[AlexeySubbotin]

Hi!
I'm developing the 3D modeling GIS software. For 3D city models i use generated from vector data building models and 3d files (Google Earth and SketchUp) with geographic coordinates within. I import such files' geometries with IImport3DFile and store them in multipatch featureclass in file geodatabase.
At last I recieved an exception "Out of memory" during IFeatureCursor.InsertFeature. Have to say that there was no such a problem before (with other files). Error appears constantly after processing definite number of files. Geometries i load from them are correct, i processing their footprints and if i don't save them to my featureclass i process them all with no exceptions. After the exception appears program can't work properly at all until restart (looks like ArcObjects crash).
How you could understand i use insert cursor with featurebuffer. Tried edit sessions, load mode for featureclass, direct write with WriteFeature and so on... No differences. May anybody help me? Any suggestions?
Sorry for my English )

[JasonPike]

Hi!
I'm developing the 3D modeling GIS software. For 3D city models i use generated from vector data building models and 3d files (Google Earth and SketchUp) with geographic coordinates within. I import such files' geometries with IImport3DFile and store them in multipatch featureclass in file geodatabase.
At last I recieved an exception "Out of memory" during IFeatureCursor.InsertFeature. Have to say that there was no such a problem before (with other files). Error appears constantly after processing definite number of files. Geometries i load from them are correct, i processing their footprints and if i don't save them to my featureclass i process them all with no exceptions. After the exception appears program can't work properly at all until restart (looks like ArcObjects crash).
How you could understand i use insert cursor with featurebuffer. Tried edit sessions, load mode for featureclass, direct write with WriteFeature and so on... No differences. May anybody help me? Any suggestions?
Sorry for my English )

Can you post code samples for where you are loading models?

What language are you using for development?

If you are using a .NET language, you can try using CLRProfiler to identify what is consuming the most memory. If it isn't a managed memory issue, you can use UMDH to see what is being allocated on the unmanaged heap.

Let me know if I can help.

[AlexeySubbotin]

Can you post code samples for where you are loading models?

What language are you using for development?

If you are using a .NET language, you can try using CLRProfiler to identify what is consuming the most memory. If it isn't a managed memory issue, you can use UMDH to see what is being allocated on the unmanaged heap.

Let me know if I can help.

Hi, ScJpike!
I use C# with .NET Framework 4.0

There is code fragments of my project for you to test:
Loading each model is made with following class instance (one class instance represents one file):

//class implementation code fragment
public enum GeoReferencedModelAltitudeType { relativeToGround, absolute, clampToGround }

public class GeoReferencedModel
{


public string Filename { get; set; }

//properties are set while reading file
public double Longitude { get; private set; }
public double Latitude { get; private set; }
public double Altitude { get; private set; }
public GeoReferencedModelAltitudeType AltitudeType { get; private set; }
public double XAngle { get; private set; }
public double YAngle { get; private set; }
public double ZAngle { get; private set; }
public double ScaleX { get; private set; }
public double ScaleY { get; private set; }
public double ScaleZ { get; private set; }


public GeoReferencedModel(string filename)
{
Filename = filename;
ScaleX = ScaleY = ScaleZ = 1;
}

public IPoint get_Placement()
{
//read file and assign class properties
//no ArcObjects use

//just for test with a model
Longitude = 51.310704;
Latitude = 35.731657;
Altitude = 0;
AltitudeType = GeoReferencedModelAltitudeType.relativeToGround;
ZAngle = 0;
XAngle = 0;
YAngle = 0;
ScaleX = 1;
ScaleY = 1;
ScaleZ = 1;

IPoint point = new PointClass();
point.PutCoords(Longitude, Latitude);
        return point;
}


private void set_ModelAltitude(IPoint placement)
{
switch (AltitudeType)
{
  case GeoReferencedModelAltitudeType.absolute:
   placement.Z = Altitude;
   break;
  case GeoReferencedModelAltitudeType.clampToGround:
   break;
  case GeoReferencedModelAltitudeType.relativeToGround:
   placement.Z = placement.Z + Altitude;
   break;
}
}

public IGeometry get_MultiPatch(IPoint placement, ISpatialReference spatialReference)
{
IGeometry geometry = null;
IImport3DFile import3DFile = new Import3DFileClass();
try
{
  import3DFile.CreateFromFile(Filename);
  geometry = import3DFile.Geometry;
  ((IImport3DFileRoots)import3DFile).ReleaseCurrentFile();
}
catch
{
  geometry = null;
}
finally
{
  System.Runtime.InteropServices.Marshal.ReleaseComObject(import3DFile);
}

if (geometry != null)
{
  set_ModelAltitude(placement);
  IVector3D xVector = constructVector3D(10, 0, 0);
  IVector3D yVector = constructVector3D(0, 10, 0);
  IVector3D zVector = constructVector3D(0, 0, 10);
  ((ITransform3D)geometry).RotateVector3D(xVector, XAngle * (Math.PI / 180));
  ((ITransform3D)geometry).RotateVector3D(yVector, YAngle * (Math.PI / 180));
  ((ITransform3D)geometry).RotateVector3D(zVector, ZAngle * (Math.PI / 180));
  ((ITransform3D)geometry).Move3D(placement.X, placement.Y, placement.Z);
  ((ITransform3D)geometry).Scale3D(placement, ScaleX, ScaleY, ScaleZ);
  geometry.SpatialReference = spatialReference;
  geometry.SnapToSpatialReference();
}

return geometry;
}

}

Processing all model files with inserting them into file geodatabase featureclass is made by following method:

public static class Tools
{

public static void Import3DFiles(string GDBFilename, string Dataset, List<GeoReferencedModel> Files, ISpatialReference DatasetSpatialReference)
{
IWorkspaceFactory wsf = new FileGDBWorkspaceFactoryClass();
IWorkspace datasetWorkspace=wsf.OpenFromFile(GDBFilename, 0);
System.Runtime.InteropServices.Marshal.ReleaseComObject(wsf);

IFeatureClass objectiveFeatureClass;
exist = ((IWorkspace2)datasetWorkspace).get_NameExists(esriDatasetType.esriDTFeatureClass, Dataset);
if (!exist)
{
 
             IFieldEdit fieldEdit;
                        
  IField shapeField = new FieldClass();
  fieldEdit = (IFieldEdit) shapeField;
  fieldEdit.Name_2 = "Shape";
             fieldEdit.Type_2 = esriFieldType.esriFieldTypeGeometry;

  IGeometryDef geometryDef = new GeometryDefClass();
             IGeometryDefEdit geometryDefEdit;
             geometryDefEdit = (IGeometryDefEdit)geometryDef;
             geometryDefEdit.GeometryType_2 = esriGeometryType.esriGeometryMultiPatch;
             geometryDefEdit.GridCount_2 = 1;
             geometryDefEdit.set_GridSize(0, 0);
             geometryDefEdit.HasZ_2 = true;
             geometryDefEdit.HasM_2 = false;
             if (DatasetSpatialReference != null) geometryDefEdit.SpatialReference_2 = DatasetSpatialReference; else geometryDefEdit.SpatialReference_2 = new UnknownCoordinateSystemClass();
             fieldEdit.GeometryDef_2 = geometryDef;

  IField oidField = new FieldClass();
  fieldEdit = (IFieldEdit) oidField;
  fieldEdit.Name_2 = "OID";
             fieldEdit.Type_2 = esriFieldType.esriFieldTypeOID;
  fieldEdit.Precision_2 = 0;
               
  IFields fcFields = new FieldsClass();
  IFieldsEdit fcFieldsEdit = (IFieldsEdit)fcFields;
  fcFieldsEdit.AddField(shapeField);
  fcFieldsEdit.AddField(oidField);

  IFieldChecker fieldChecker = new FieldCheckerClass();
  IEnumFieldError enumFieldError = null;
  IFields validatedFields = null;
  fieldChecker.ValidateWorkspace = datasetWorkspace;
  fieldChecker.Validate(fields, out enumFieldError, out validatedFields);
           
  IObjectClassDescription objectDescription = new FeatureClassDescriptionClass();
  objectiveFeatureClass = ((IFeatureWorkspace)datasetWorkspace).CreateFeatureClass(dataset, validatedFields, null, objectDescription.ClassExtensionCLSID, esriFeatureType.esriFTSimple, shapeField.Name, "");
}
else
  objectiveFeatureClass = ((IFeatureWorkspace)datasetWorkspace).OpenFeatureClass(dataset);

((ISchemaLock)objectiveFeatureClass).ChangeSchemaLock(esriSchemaLock.esriExclusiveSchemaLock);
((IFeatureClassLoad)objectiveFeatureClass).LoadOnlyMode = true;

ISpatialReferenceFactory srFactory = new SpatialReferenceEnvironmentClass();
IGeographicCoordinateSystem gcs = srFactory.CreateGeographicCoordinateSystem(gcsType);
ISpatialReference geoSR = (ISpatialReference)gcs;
ISpatialReferenceResolution srResolution = (ISpatialReferenceResolution)sr;
        srResolution.ConstructFromHorizon();
        srResolution.SetDefaultMResolution();
        srResolution.SetDefaultXYResolution();
        srResolution.SetDefaultZResolution();
        ISpatialReferenceTolerance srTolerance = (ISpatialReferenceTolerance)sr;
        srTolerance.SetDefaultMTolerance();
        srTolerance.SetDefaultXYTolerance();
        srTolerance.SetDefaultZTolerance();
set_ResolutionTolerance(sr);
System.Runtime.InteropServices.Marshal.ReleaseComObject(srFactory);

featureBuffer = objectiveFeatureClass.CreateFeatureBuffer();
featureCursor = objectiveFeatureClass.Insert(true);

foreach (string geoReferencedModel in Files)
{
  //reading lat,long and other values values from files with own SketchUpReader (for *.skp) and ZipForge, XDocument (for *.kmz)
  //do not use any ArcObjects
  IPoint placement = geoReferencedModel.get_Placement();
  placement.SpatialReference = geoSR;
  (IZAware)geometry.ZAware = true;
  placement.Project(DatasetSpatialReference);
 
  //get Z value from IFunctionalSurface (relief raster) 
  //just for test
  placement.Z = 0;

  if (!placement.Z.Equals(double.NaN))
  {
                 IGeometry multiPatch=null;
   //import file's geometry
                        multiPatch = geoReferencedModel.get_MultiPatch(placement, DatasetSpatialReference);
   if (multiPatch != null)
   {
    featureBuffer.Shape = multiPatch;
                                featureCursor.InsertFeature(featureBuffer);

    /*tried to flush cursor and create new instances cursor and buffer
    featureCursor.Flush();
    featureBuffer = objectiveFeatureClass.CreateFeatureBuffer();
    featureCursor = objectiveFeatureClass.Insert(true);
    */

    /*tried to use cursor with no buffering here and in declaration
    featureCursor.Flush();
    featureBuffer = objectiveFeatureClass.CreateFeatureBuffer();
    featureCursor = objectiveFeatureClass.Insert(false);
    */

    /*tried such variant with no featurebuffer and featurecursor at all
    IFeature newFeature =  objectiveFeatureClass.CreateFeature();
    newFeature.Shape = multipatch;
    ((IFeatureClassWrite)objectiveFeatureClass).WriteFeature(newFeature);
    */

    //here i compute a footprint of multipatch and store it in another featureclass

                                System.Runtime.InteropServices.Marshal.ReleaseComObject(placement);
                                System.Runtime.InteropServices.Marshal.ReleaseComObject(multiPatch);
                                System.Runtime.InteropServices.Marshal.CleanupUnusedObjectsInCurrentContext();
                                GC.Collect();  
   }
  }
 
}

featureCursor.Flush();
System.Runtime.InteropServices.Marshal.ReleaseComObject(featureCursor);

((IFeatureClassLoad)objectiveFeatureClass).LoadOnlyMode = false;
((ISchemaLock)objectiveFeatureClass).ChangeSchemaLock(esriSchemaLock.esriSharedSchemaLock);


}


}

[AlexeySubbotin]

To test my code from previous post i prepared a code fragment

//for test
//may need some correction as was configured and modified on-the-fly from project

ISpatialReferenceFactory3 srFactory = new SpatialReferenceEnvironmentClass();
IProjectedCoordinateSystem pcs = srFactory.CreateProjectedCoordinateSystem(pcsType);

ISpatialReference spatialReference = (ISpatialReference)pcs;
ISpatialReferenceResolution srResolution = (ISpatialReferenceResolution)spatialReference;
srResolution.ConstructFromHorizon();
srResolution.SetDefaultMResolution();
srResolution.SetDefaultXYResolution();
srResolution.SetDefaultZResolution();
ISpatialReferenceTolerance srTolerance = (ISpatialReferenceTolerance)spatialReference;
srTolerance.SetDefaultMTolerance();
srTolerance.SetDefaultXYTolerance();
srTolerance.SetDefaultZTolerance();


IVerticalDatum verticalDatum = srFactory.CreateVerticalDatum((int)esriSRVerticalDatumType.esriSRVertDatum_SeaLevel);
IHVDatum hvDatum = verticalDatum as IHVDatum;
ILinearUnit unit = srFactory.CreateUnit((int)esriSRUnitType.esriSRUnit_Meter) as ILinearUnit;
IVerticalCoordinateSystemEdit vcsEdit = (IVerticalCoordinateSystemEdit)new VerticalCoordinateSystemClass();
int positive = 1;
double shift = 0;
vcsEdit.DefineEx("", "", "", "", "", hvDatum, unit, ref shift, ref positive);
((ISpatialReference3)spatialReference).VerticalCoordinateSystem = (IVerticalCoordinateSystem)vcsEdit;
ISpatialReferenceResolution sprefResolution = (ISpatialReferenceResolution)spatialReference;
sprefResolution.SetDefaultZResolution();
ISpatialReferenceTolerance srTolerance = (ISpatialReferenceTolerance)spatialReference;
srTolerance.SetDefaultZTolerance();
spatialReference.Changed();

System.Runtime.InteropServices.Marshal.ReleaseComObject(srFactory);

List<GeoReferencedModel> models=new List<GeoReferencedModel>();
models.Add(new GeoReferencedModel(path_to_model));

IWorkspaceFactory2 workspaceFactory = new FileGDBWorkspaceFactoryClass();
IWorkspaceName workspaceName = workspaceFactory.Create(System.IO.Path.GetDirectoryName(path_to_fileGDB), System.IO.Path.GetFileName(path_to_fileGDB), null, 0);
IName name = (IName)workspaceName;
IWorkspace workspace = (IWorkspace)name.Open();
System.Runtime.InteropServices.Marshal.ReleaseComObject(workspaceFactory);
System.Runtime.InteropServices.Marshal.ReleaseComObject(workspaceName);

Import3DFiles(path_to_fileGDB, "GeoReferencedObjects3D", models, spatialReference)

I see that the model causing error is large enough (kmz file 13Mb), and may be featureclass can't store such amount of information by igeometry field. If so is, so how i can analyze geometries of 3d files to prevent exceptions with attempts to store them? What is maximum size of such objects?
Can't to attach model file because of size restrictions. If you would like i can email it to you.
Would be very appreciative for your help.

[JasonPike]

To test my code from previous post i prepared a code fragment

//for test
//may need some correction as was configured and modified on-the-fly from project

ISpatialReferenceFactory3 srFactory = new SpatialReferenceEnvironmentClass();
IProjectedCoordinateSystem pcs = srFactory.CreateProjectedCoordinateSystem(pcsType);

ISpatialReference spatialReference = (ISpatialReference)pcs;
ISpatialReferenceResolution srResolution = (ISpatialReferenceResolution)spatialReference;
srResolution.ConstructFromHorizon();
srResolution.SetDefaultMResolution();
srResolution.SetDefaultXYResolution();
srResolution.SetDefaultZResolution();
ISpatialReferenceTolerance srTolerance = (ISpatialReferenceTolerance)spatialReference;
srTolerance.SetDefaultMTolerance();
srTolerance.SetDefaultXYTolerance();
srTolerance.SetDefaultZTolerance();


IVerticalDatum verticalDatum = srFactory.CreateVerticalDatum((int)esriSRVerticalDatumType.esriSRVertDatum_SeaLevel);
IHVDatum hvDatum = verticalDatum as IHVDatum;
ILinearUnit unit = srFactory.CreateUnit((int)esriSRUnitType.esriSRUnit_Meter) as ILinearUnit;
IVerticalCoordinateSystemEdit vcsEdit = (IVerticalCoordinateSystemEdit)new VerticalCoordinateSystemClass();
int positive = 1;
double shift = 0;
vcsEdit.DefineEx("", "", "", "", "", hvDatum, unit, ref shift, ref positive);
((ISpatialReference3)spatialReference).VerticalCoordinateSystem = (IVerticalCoordinateSystem)vcsEdit;
ISpatialReferenceResolution sprefResolution = (ISpatialReferenceResolution)spatialReference;
sprefResolution.SetDefaultZResolution();
ISpatialReferenceTolerance srTolerance = (ISpatialReferenceTolerance)spatialReference;
srTolerance.SetDefaultZTolerance();
spatialReference.Changed();

System.Runtime.InteropServices.Marshal.ReleaseComObject(srFactory);

List<GeoReferencedModel> models=new List<GeoReferencedModel>();
models.Add(new GeoReferencedModel(path_to_model));

IWorkspaceFactory2 workspaceFactory = new FileGDBWorkspaceFactoryClass();
IWorkspaceName workspaceName = workspaceFactory.Create(System.IO.Path.GetDirectoryName(path_to_fileGDB), System.IO.Path.GetFileName(path_to_fileGDB), null, 0);
IName name = (IName)workspaceName;
IWorkspace workspace = (IWorkspace)name.Open();
System.Runtime.InteropServices.Marshal.ReleaseComObject(workspaceFactory);
System.Runtime.InteropServices.Marshal.ReleaseComObject(workspaceName);

Import3DFiles(path_to_fileGDB, "GeoReferencedObjects3D", models, spatialReference)

I see that the model causing error is large enough (kmz file 13Mb), and may be featureclass can't store such amount of information by igeometry field. If so is, so how i can analyze geometries of 3d files to prevent exceptions with attempts to store them? What is maximum size of such objects?
Can't to attach model file because of size restrictions. If you would like i can email it to you.
Would be very appreciative for your help.

Have you tried only loading the model that has the problem? Or, does the problem only occur if you load a bunch of models and then the model that is having trouble?

[AlexeySubbotin]

Have you tried only loading the model that has the problem? Or, does the problem only occur if you load a bunch of models and then the model that is having trouble?

The problem occur with single model too. No differences bunch of models or single model.

[JasonPike]

The problem occur with single model too. No differences bunch of models or single model.

Ok. Can you create and send a project that loads that particular model and reproduces the failure? Please include the model in the zip. I'll try to find out what is consuming the memory.

I'd also report this as a bug to ESRI.

[AlexeySubbotin]

Ok. Can you create and send a project that loads that particular model and reproduces the failure? Please include the model in the zip. I'll try to find out what is consuming the memory.

I'd also report this as a bug to ESRI.

Of course. I don't have such an ability at home (weekend you know). Once i'll get to office, i'll make it. Such a question. How can i send it, if it's size exceeds the limits for attachments? Only model is 12.8 MB, while limit for zip is 2.00 MB.

[JasonPike]

Of course. I don't have such an ability at home (weekend you know). Once i'll get to office, i'll make it. Such a question. How can i send it, if it's size exceeds the limits for attachments? Only model is 12.8 MB, while limit for zip is 2.00 MB.

Could you post it on Dropbox or some equivalent and post a link?

[AlexeySubbotin]

Could you post it on Dropbox or some equivalent and post a link?

Hi, Jason. I created the project.
Despite the common logic of the test project and the source, the test project produces another exception, but it occurs at the same operator (insert a feature to the feature class).
Depending on the type of call (in the current thread or in new one) it can be the RCP_E_SERVERFAULT exception wit error code -2147417851 or the external object's exception with error code -2147467259. Also the project is based on .NET Framework 3.5 (as it should be for ArcObjects 10). Couldn't to base it on .NET Framework 4.0, and honestly don't realy understand how it happened with my source project. Maybe the difference of exceptions raised by the projects explaned by different framework they are based on. After all the exception remains and it raises constantly.

Link to archive http://dl.dropbox.com/u/91176632/Test.zip