Slow C# query performance vs VBA

3845
19
07-27-2011 09:08 AM
CarlosPiccirillo
New Contributor III
Hi everyone,

I am writing an stand alone C# application and I am having a performance issue. The following code was converted from VBA literally line by line. I am doing a spatial query between two layers. The VBA code runs in between 10 and 15 minutes. The C# code below takes just over 3 hours to do the same query on the same layers as the VBA code. The code below doesn't even start ArcMap, just accesses things directly. I also tried changing the code a bit to start ArcMap in code and then run the code but it still takes the same amount of time.

Obviously, I am missing something/doing something wrong but I can't figure it out. Anyone have any ideas?

Thanks,
Carlos

public static void AppsInBoundary()
{ 
    try
    {
 IFeatureLayer pAppLayer = null;
 IFeatureLayer pEnsLayer = null;
 IFeatureCursor pEvgCursor = null;
 IFeatureCursor pAppCursor = null;
 IFeature pAppFeature = null;
 IFeature pEnsFeature = null;
 IArea pFeatureArea;
 int featureCount = 0;
 int appFieldIndex = 0;
 string geometryFieldName = string.Empty;

 //Get a reference to the two layers involved in the query.
 pAppLayer = GetLayerReference("FILEGDB", "App_Merge", Scratch_Local + "temp.gdb", null) as IFeatureLayer;
 pEnsLayer = GetLayerReference("SDE", "RIM.HYHDB_BSN_EVRGLDS_AREA", null, "gerrpsde") as IFeatureLayer;

 //Make sure layers being queried have the same projection.
 //Otherwise, topological operations later in the code (DetermineOverlapArea) fail.
 SpatialReferenceCheck(pAppLayer, pEnsLayer);

 //Find index of app no field in app_merge layer so query can run faster. 
 appFieldIndex = pAppLayer.FeatureClass.FindField("APP_NO");

 //Select only polygons that have a basin name.
 IQueryFilter pEnsQueryFilter = new QueryFilterClass();
 pEnsQueryFilter.WhereClause = "basin_name <> ' '";

 pEvgCursor = pEnsLayer.Search(pEnsQueryFilter, true);
 pEnsFeature = pEvgCursor.NextFeature();

 ISpatialFilter pAppSpatialFilter = new SpatialFilterClass();

 while (pEnsFeature != null)
 {
     m_Label = pEnsFeature.get_Value(pEnsFeature.Fields.FindField("BASIN_NAME")).ToString();

     pAppSpatialFilter.Geometry = pEnsFeature.Shape;
     pAppSpatialFilter.GeometryField = "Shape";
     pAppSpatialFilter.SpatialRel = esriSpatialRelEnum.esriSpatialRelIntersects;

     pAppCursor = pAppLayer.Search(pAppSpatialFilter, true);
     pAppFeature = pAppCursor.NextFeature();

     while (pAppFeature != null)
     {
  if (pAppFeature.get_Value(appFieldIndex) != DBNull.Value)
  {
      m_AppNo = Convert.ToString(pAppFeature.get_Value(appFieldIndex));
      pFeatureArea = pAppFeature.Shape as IArea;
      m_AppArea = pFeatureArea.Area;

      //Set relation type, in or adjacent.
      if (DetermineOverlapArea(pAppFeature, pEnsFeature) == true)
   m_Relation = "AI";   //Application In
      else
   m_Relation = "AA";   //Application Adjacent

      //If app_no is not null, proceed, otherwise skip it.
      if (m_AppNo != "")
      {
   //Check if app status is "no response", if it is not, proceed.
   if (AppStatusCheck(m_AppNo) == true)
   {
       GetAppInfo(m_AppNo);
       PopAppCmgenTables(m_AppNo, m_Relation);
   }
      }

      pAppFeature = pAppCursor.NextFeature();

      //Clear values.
      m_AppNo = string.Empty;
      m_AppArea = 0;

  }   //APP_NO != DBNull.Value
     }   //pAppFeature != null

     pEnsFeature = pEvgCursor.NextFeature();

     //Clear values.
     m_AppNo = string.Empty;
     m_Relation = string.Empty;
     m_Label = string.Empty;
 }   //pEnsFeature != null
    }
    catch (Exception ex)
    {
 LogError(ex.StackTrace, ex.Message, "AppsInBoundary", null);
    }   
}

internal static ILayer GetLayerReference(string layerType, string featureClassName, string path, string serverName)
{
    try
    {
 IWorkspaceFactory pWorkspaceFactory = null;
 IFeatureWorkspace pFeatureWorkspace = null;
 IFeatureClass pFeatureClass = null;

 if (layerType == "SDE")
 {
     IPropertySet pPropertySet = new PropertySetClass();
     pPropertySet = GetSdePropertySet(serverName, null);
     pWorkspaceFactory = new SdeWorkspaceFactoryClass();
     pFeatureWorkspace = pWorkspaceFactory.Open(pPropertySet, 0) as IFeatureWorkspace;
 }
 else if (layerType == "FILEGDB")
 {
     pWorkspaceFactory = new FileGDBWorkspaceFactoryClass();
     pFeatureWorkspace = pWorkspaceFactory.OpenFromFile(path, 0) as IFeatureWorkspace;
 }
 else if (layerType == "PGDB")
 {
     pWorkspaceFactory = new AccessWorkspaceFactoryClass();
     pFeatureWorkspace = pWorkspaceFactory.OpenFromFile(path, 0) as IFeatureWorkspace;
 }

 pFeatureClass = pFeatureWorkspace.OpenFeatureClass(featureClassName);
 IFeatureLayer pFeatureLayer = new FeatureLayerClass();
 pFeatureLayer.FeatureClass = pFeatureClass;
 return pFeatureLayer;
    }
    catch (Exception ex)
    {
 LogError(ex.StackTrace, ex.Message, "GetGroupLayer", null);
 return null;
    }
}
0 Kudos
19 Replies
CarlosPiccirillo
New Contributor III
Aparently, I messed up doing the first test because when I repeated them later trying some of Neil's suggestion, the code took less than a minute. The bottleneck is definitely between GetAppInfo and DetermineOverlapArea. The biggest bottleneck seems to be DetermineOverlapArea. I've been trying alternative code all day and get the same slow results.
0 Kudos
NeilClemmons
Regular Contributor III
The most expensive thing the method does is call the Intersect method.  How long does it take for that one line of code to execute?
0 Kudos
AlexanderGray
Occasional Contributor III
How long do the convert.toint32 functions taking? If they are significant (type conversions are one thing slower because of the .Net interop) you can probably do without it and compare the doubles.
0 Kudos
CarlosPiccirillo
New Contributor III
Neil, Alexander,

Sorry for the late reply. I was out Thursday afternoon and all day Friday.

Neil, you are correct, the bottleneck is the Intersect command in the DeterimeOverlapArea method. I'm trying to figure out what I can use in it's place that might be faster.

Alexander, the Convert.ToInt32 is very fast so that part of the code is okay although I will try your suggestion. Every little bit of speed increase helps.

Carlos
0 Kudos
NehaS1
by
New Contributor II
Carlos,
From my experience arcobjects should never be instantiated  in every iteration of while loop if possible. In your case IArea pFeature1Area,IArea pFeature2Area,ITopologicalOperator2 pTopologicalOperator can be declared once globally and can be reused. You can use pTopologicalOperator.Boundary.SetEmpty(); to release memory
0 Kudos
NehaS1
by
New Contributor II
How many records are stored in  pAppLayerand pEnsLayer? Is it sde?
0 Kudos
CarlosPiccirillo
New Contributor III
Samham,

Thanks for the reply! pAppLayer is in a file geodatabase and pEnsLayer is in SDE.

Not sure I follow you when you say, "From my experience arcobjects should never be instantiated in every iteration of while loop if possible."

Do you have an example of what you mean?

I will be out until Monday so my appologies ahead of time if I am late replying to any replies you post. Thanks again!

Carlos

How many records are stored in  pAppLayerand pEnsLayer? Is it sde?
0 Kudos
NehaS1
by
New Contributor II
For example if pAppLayer  has 100,000 records then you are iterating 100,000 times and creating 100,000  ITopologicalOperator2 pTopologicalOperator. Why not create one ITopologicalOperator2 pTopologicalOperator and reuse it?
0 Kudos
NehaS1
by
New Contributor II
When you comment the method how much time does it take to iterate through all the records?
0 Kudos
DubravkoAntonic
New Contributor III
If you consider your app's performance is more valuable then time you spend for optimizing then this effort is valid one, in other cases let the user wait.

There are two way approach.
ne is to use VB code create COM object that you will refer from you project and use it when needed and you'll be very close to ArcMap speed.

Second:
There try to use IRelationalOperator instead of ITopologicalOperator sinc you are using just a part of ITopologicalOperator functions. Please try to measure how much you will be faster. Neil is again totally right, this time about Simplify,  people use this very often. Since you are not modifying geometry the is no need for this operation.
If you are satisfied with the speed, then you can consider using managed C++. There you'll save interop operation time.
http://help.arcgis.com/en/sdk/10.0/arcobjects_net/conceptualhelp/0001/0001000000wm000000.htm

Regards
0 Kudos