Select to view content in your preferred language

Updating attributes, IFeatureClass.Update vs IFeatureClass.Search

4613
10
08-07-2012 01:20 PM
CarlosPiccirillo
Emerging Contributor
Hi everyone,

Been fighting with this issue for several weeks now and not getting anywhere with it. The goal is to programmatically read attribute values from an Oracle table and update fields in an SDE feature class that is versioned. I am only working on one version currently but eventually, this version will be used to update default once the attributes are updated.

The code below works.... sometimes but not always. I can run it consecutive times and it is hit or miss.

In the code below I have tried updating fields by using both IFeatureClass.Update with IFeatureCursor.UpdateFeature and IFeatureClass.Search with IFeature.Store. On a layer with 24,000 records, using IFeatureClass.Update is slow taking 48 minutes compared to 3 minutes for IFeatureClass.Search. I have tried using recycling and non-recycling cursors. I have even tried reducing the number of fields I update but no matter what I do, sometimes the code updates and sometimes it does not. When it does not, I do not get any errors but the fields are not updated. I am out of ideas on what to try next.

Does anyone see a problem or a better way to do this? I reduced toe number of fields to get the code to fit here. In reality, I have to update over 50 fields on 20 SDE layers with over 300,000 records total!

private static void ProcessLayer(string permitType)
{
 IPropertySet pPropertySet = new PropertySetClass();
 IWorkspaceFactory pWorkspaceFactory = new SdeWorkspaceFactoryClass();
 IFeatureWorkspace pFeatureWorkspace = null;
 IFeatureClass pFeatureClass = null;
 IDataset pDataset = null;
 IWorkspace pWorkspace = null;
 IWorkspaceEdit pWorkspaceEdit = null;
 ITrackCancel pTrackCancel = new CancelTrackerClass();
 IProgressDialogFactory pProgressDialogFactory = new ProgressDialogFactoryClass();
 IProgressDialog2 pProgressDialog2 = null;
 IStepProgressor pStepProgressor = null;
 OracleConnection dataConnection = new OracleConnection();
 OracleCommand dataCommand = new OracleCommand();
 OracleDataReader dataReader = null;
 IFeatureCursor pFeatureCursor = null;
 IFeature pFeature = null;

 try
 {
  pPropertySet.SetProperty("SERVER", "gresssde");
  pPropertySet.SetProperty("INSTANCE", "5195:rim");
  pPropertySet.SetProperty("USER", "GORT");
  pPropertySet.SetProperty("PASSWORD", "Klaatu");
  pPropertySet.SetProperty("VERSION", "RIM.GORT");

  pFeatureWorkspace = pWorkspaceFactory.Open(pPropertySet, 0) as IFeatureWorkspace;
  pFeatureClass = pFeatureWorkspace.OpenFeatureClass(permitType);
  pDataset = pFeatureClass as IDataset;

  pWorkspace = pDataset.Workspace;
  pWorkspaceEdit = pWorkspace as IWorkspaceEdit;

  //If permit type field not found, stop code.
  if (pFeatureClass.FindField("FEE_PERMIT_TYPE") != -1)
  {
   //Start an edit session and operation
   pWorkspaceEdit.StartEditing(false);
   pWorkspaceEdit.StartEditOperation();

   pFeatureCursor = pFeatureClass.Search(null, false);
   //pFeatureCursor = pFeatureClass.Search(null, false);
   //pFeatureCursor = pFeatureClass.Update(null, true);

   //Get the index numbers for every field being updated.
   int permitNoIdx = pFeatureClass.FindField("PERMIT_NO");
   int appNoIdx = pFeatureClass.FindField("APP_NO");
   int projectNameIdx = pFeatureClass.FindField("PROJECT_NAME");
   int appStatusIdx = pFeatureClass.FindField("APP_STATUS");
   int activeModIdx = pFeatureClass.FindField("ACTIVE_MOD");
   string appNo = string.Empty;
   string permitNo = string.Empty;
   string projectName = string.Empty;
   string appStatus = string.Empty;
   string activeMod = string.Empty;

   //Set up and open the progress dialog.
   int featureCount = 1;
   int numberOfFeatures = pFeatureClass.FeatureCount(null);

   pProgressDialog2 = pProgressDialogFactory.Create(pTrackCancel, 0) as IProgressDialog2;
   pProgressDialog2.CancelEnabled = true;
   pProgressDialog2.Description = "Processing Record " + featureCount + " of " + numberOfFeatures;
   pProgressDialog2.Title = "Processing...";
   pProgressDialog2.Animation = esriProgressAnimationTypes.esriProgressGlobe;

   pStepProgressor = pProgressDialog2 as IStepProgressor;
   pStepProgressor.MinRange = 0;
   pStepProgressor.MaxRange = numberOfFeatures;
   pStepProgressor.StepValue = 1;
   pStepProgressor.Message = permitType;

   pProgressDialog2.ShowDialog();
   bool canContinue = true;

   //Read values from Oracle and update each feature with the info.
   dataConnection.ConnectionString = "Data Source=genp;" +
      "Persist Security Info=True;" +
      "User ID=pub;" +
      "Password=pub;" +
      "Unicode=True";

   dataConnection.Open();
   dataCommand.Connection = dataConnection;

   pFeature = pFeatureCursor.NextFeature();

   while (pFeature != null)
   {
    appNo = pFeature.get_Value(appNoIdx).ToString();

    dataCommand.CommandText = "SELECT PERMIT_NO, PROJECT_NAME, APP_STATUS, ACTIVE_MOD, " +
       "FROM REG.REG_GIS WHERE app_no = '" + appNo + "'";

    dataReader = dataCommand.ExecuteReader();

    while (dataReader.Read())
    {
     if (!dataReader.IsDBNull(0))
     {
      permitNo = dataReader.GetString(0);
      permitNo = permitNo.Replace("'", "");
     }

     if (!dataReader.IsDBNull(1))
     {
      projectName = dataReader.GetString(1);
      projectName = projectName.Replace("'", "");
     }

     if (!dataReader.IsDBNull(2))
     {
      appStatus = dataReader.GetString(2);
      appStatus = appStatus.Replace("'", "");
     }

     if (!dataReader.IsDBNull(3))
     {
      activeMod = dataReader.GetString(3);
      activeMod = activeMod.Replace("'", "");
     }

     //Only update fields whose value have changed.
     if (permitNoIdx != -1)
     {
      if (pFeature.get_Value(permitNoIdx) != DBNull.Value)
      {
       string permitNoIn = Convert.ToString(pFeature.get_Value(permitNoIdx));
       if (permitNoIn != permitNo) { pFeature.set_Value(permitNoIdx, permitNo); }
      }
     }

     if (projectNameIdx != -1)
     {
      if (pFeature.get_Value(projectNameIdx) != DBNull.Value)
      {
       string projectNameIn = Convert.ToString(pFeature.get_Value(projectNameIdx));
       if (projectNameIn != projectName) { pFeature.set_Value(projectNameIdx, projectName); }
      }
     }

     if (appStatusIdx != -1)
     {
      if (pFeature.get_Value(appStatusIdx) != DBNull.Value)
      {
       string appStatusIn = Convert.ToString(pFeature.get_Value(appStatusIdx));
       if (appStatusIn != appStatus) { pFeature.set_Value(appStatusIdx, appStatus); }
      }
     }

     if (activeModIdx != -1)
     {
      if (pFeature.get_Value(activeModIdx) != DBNull.Value)
      {
       string activeModIn = Convert.ToString(pFeature.get_Value(activeModIdx));
       if (activeModIn != activeMod) { pFeature.set_Value(activeModIdx, activeMod); }
      }
     }

     pFeature.Store();
     //pFeatureCursor.UpdateFeature(pFeature);
    }   //while (dataReader.Read())

    dataReader.Close();
    dataReader.Dispose();

    pFeature = pFeatureCursor.NextFeature();

    //Increment the progress dialog.
    canContinue = pTrackCancel.Continue();
    if (canContinue == true)
    {
     pStepProgressor.Step();
     featureCount++;
     pProgressDialog2.Description = "Processing Record " + featureCount + " of " + numberOfFeatures;
    }
    else
    {
     pProgressDialog2.HideDialog();
     pWorkspaceEdit.StopEditOperation();
     pWorkspaceEdit.StopEditing(true);
     return;
    }

    //Clear values.
    appNo = string.Empty;
    permitNo = string.Empty;
    projectName = string.Empty;
    appStatus = string.Empty;
    activeMod = string.Empty;
   }

   //Stop editing
   pWorkspaceEdit.StopEditOperation();
   pWorkspaceEdit.StopEditing(true);

   //Close progress dialog.
   pProgressDialog2.HideDialog();
  }   //if (pFeatureClass.FindField("FEE_PERMIT_TYPE") != -1)
 }
 catch (Exception ex)
 {
  MessageBox.Show(ex.StackTrace + "\r\n" + ex.Message);
 }
}


Thanks,
Carlos
0 Kudos
10 Replies
LeoDonahue
Deactivated User

if (pFeatureClass.FindField("FEE_PERMIT_TYPE") != -1)
{
//Start an edit session and operation
pWorkspaceEdit.StartEditing(false);



If you are editing versioned data, don't you need to start an edit session?

How to use cursors in the geodatabase: http://help.arcgis.com/en/sdk/10.0/java_ao_adf/conceptualhelp/engine/0001/0001000003s5000000.htm
"Editing with cursors".

You should relase pFeatureCursor outside of the outer while loop if you don't need it anymore.

And change your passwords.

One more thing,
In reality, I have to update over 50 fields on 20 SDE layers with over 300,000 records total!
I guess I don't understand why you want to update feature attributes with data from an Oracle database. Can't you join or relate this featureclass to the Oracle table to get that information?  In other words, why does this data from Oracle need to reside in two places?  Do you have an application built around the data in this featureclass?
0 Kudos
CarlosPiccirillo
Emerging Contributor
Leo,

Thanks for the reply.

I am familiar with the example on the link you sent me, I used the C# version of this example to do my script. I am starting a edit session on the exact line you mentioned. First I start an edit session on the workspace, then start an edit operation. I am using IWorkspaceEdit instead of IEditor because my code runs outside of ArcMap and using IWorkspace edit for these occasions is the ESRI suggested method.

The reason for having to update attributes is because originally, we only had one field in the feature clasa and all other information was gotten on the fly via a table join like you suggest. Unfortunately, we have seen time and again that table joins slow down the code significantly in our SDE environment. The IT people tried all sorts of stuff trying to speed things up but it never helped so we relunctantly had to add all these fields to the feature class and update the attributes via the code I am working on.

Carlos


If you are editing versioned data, don't you need to start an edit session?

How to use cursors in the geodatabase: http://help.arcgis.com/en/sdk/10.0/java_ao_adf/conceptualhelp/engine/0001/0001000003s5000000.htm
"Editing with cursors".

You should relase pFeatureCursor outside of the outer while loop if you don't need it anymore.

And change your passwords.

One more thing,  I guess I don't understand why you want to update feature attributes with data from an Oracle database. Can't you join or relate this featureclass to the Oracle table to get that information?  In other words, why does this data from Oracle need to reside in two places?  Do you have an application built around the data in this featureclass?
0 Kudos
JamesCrandall
MVP Alum
Carlos,

Have a look/read at: http://help.arcgis.com/en/sdk/10.0/arcobjects_net/conceptualhelp/index.html#/d/0001000002rs000000.ht...

While your approach is similar, there are some minor differences in the actual code implementation that may help (I notice they wrap the ComReleaser in a Using statement).  Also, scroll down towards the bottom of that page to see if the UpdateSearchedRows method using the IRowBuffer offers any benefit or resolution to your issues.

Take Care,

James
0 Kudos
LeoDonahue
Deactivated User
Leo,
First I start an edit session on the workspace, then start an edit operation.

Don't you need to pass true to pWorkspaceEdit.StartEditing() then?  Your code is passing false, or am I seeing that wrong?

Unfortunately, we have seen time and again that table joins slow down the code significantly in our SDE environment
You were using a join before and it was slowing down some other code?  What code?  Or do you mean simply making the join from ArcMap to the Oracle table was slow?
0 Kudos
CarlosPiccirillo
Emerging Contributor
James,

Thanks for the reply. Unfortunately, none of your suggestions worked. If you have other ideas, no matter how crazy, I'm willing to try them.

Carlos,

Have a look/read at: http://help.arcgis.com/en/sdk/10.0/arcobjects_net/conceptualhelp/index.html#/d/0001000002rs000000.ht...

While your approach is similar, there are some minor differences in the actual code implementation that may help (I notice they wrap the ComReleaser in a Using statement).  Also, scroll down towards the bottom of that page to see if the UpdateSearchedRows method using the IRowBuffer offers any benefit or resolution to your issues.

Take Care,

James
0 Kudos
CarlosPiccirillo
Emerging Contributor
Don't you need to pass true to pWorkspaceEdit.StartEditing() then?  Your code is passing false, or am I seeing that wrong?

The true/false parameter of IWorkspaceEdit.StartEditing() only controls whether you want to allow or deny the ability to undo/redo actions, it does not control editing.

You were using a join before and it was slowing down some other code?  What code?  Or do you mean simply making the join from ArcMap to the Oracle table was slow?


The table joins were not slowing down code, they were slowing down the performance of the layer. For example, layer redraws or opening the attribute table. At times, it was taking 2 or 3 minutes to open the attribute table when the layer was joined to a table and only seconds when not joined.
0 Kudos
JamesCrandall
MVP Alum
Don't you need to pass true to pWorkspaceEdit.StartEditing() then?  Your code is passing false, or am I seeing that wrong?


I believe that the False parameter is for specifying undo/redo on the WorkspaceEdit.
0 Kudos
JamesCrandall
MVP Alum
The table joins were not slowing down code, they were slowing down the performance of the layer. For example, layer redraws or opening the attribute table. At times, it was taking 2 or 3 minutes to open the attribute table when the layer was joined to a table and only seconds when not joined.


Admittedly I have not travelled far down this road, but you may have to implement version edit sessions.

http://help.arcgis.com/en/sdk/10.0/arcobjects_net/conceptualhelp/index.html#/Editing_with_the_geodat...

"...A versioned edit session can be started on an ArcSDE geodatabase using the IMultiuserWorkspaceEdit.StartMultiuserEditing method. Versioned edit sessions allow applications to edit a version of the geodatabase as opposed to the base tables of datasets (which is the case with local geodatabase editing and non-versioned edit sessions)."

Perhaps even if no other concurrent edits are being made to the version, you still may be required to setup the edit session to accomodate for such a thing?
0 Kudos
LeoDonahue
Deactivated User
Recap:

Problem:


  • Table join to Oracle data is slow.


Proposed Solution:


  • Loop through features in multiple feartureclasses and update  those feature attributes by pulling data out of Oracle and storing them  in the respective featureclasses.

  • This featureclass is a child version of default, that will be reconciled and posted back to default at some point in the future?


This doesn't sound like a good process, especially on versioned data.

Something  sounds like you should revisit the table join.  I've joined 1.5 million  records to a featureclass and it was never a problem.

I have a parcel featureclass with 1.5 million features and one field.  I joined that featureclass to a table containing Address and owner info.  The join happens instantly and the attribute table opens instantly.

Are you indexing the joined fields in your featureclass and Oracle table?

(ok, 1,490,000 +/- records.)

[ATTACH=CONFIG]16771[/ATTACH]

and for further disclosure, that is SQL Express.
0 Kudos