nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Field Calculator and Calculate Geometry EXTREMELY SLOW!!!

ThomasEmery

We manage land records for a county that contains 191,532 parcels. Within the attribute table there are 4 fields which require us the calculate the geometry and 3 fields which which require the use of the field calculator. This took less than 5 minutes per field in 9.3.1 but takes over 20 minutes in 10.
This could potentially cause us to go back to 9.3.1 until something is resolved.

What in the world is going on? Has anyone encountered this and found a solution?

Thanks

Finde mehr Beiträge mit dem Tag

Accepted answers

All comments

DarrenWiens2

Are you, by chance, working on a remote desktop? We experienced very slow calculations until we changed a network setting (particularly one instructing the remote session to use our local machine's Documents and Settings folder).

ThomasEmery

No, I was working within a personal geodatabase on my local machine. This slow-down with the field calculator and calculate geometry really hurts our productivity. In 9.3.1 I could publish a new Parcel feature class in about an hour, now, in 10, it takes about 4 hours.

BruceNielsen

Have you considered moving the data to a File Geodatabase? Personal GDBs tend to slow things down as they get larger, and can possibly corrupt the data when the size approaches 2Gb.

ThomasEmery

I guess I can try a file geodatabase next time but I created a new personal geodatabase for this task so I know size isn't the problem in this case.

Thanks for your idea though.

BenClay1

I recently had about 55000 records and also took a very long time for calculation. I finally gave up and calculated smaller selection of a few thousand and it worked fine, although a bit tedious to perform.

ThomasEmery

Yeah, that's basically what i had to do too. Pretty ridiculous if you ask me, but not all too surprising I guess. I should have just stayed with 9.3.1. First no snake pin tool unless you go to the fabric and now very sub par geoprocessing. Yikes.

KenHartling

No, I was working within a personal geodatabase on my local machine. This slow-down with the field calculator and calculate geometry really hurts our productivity. In 9.3.1 I could publish a new Parcel feature class in about an hour, now, in 10, it takes about 4 hours.

There is a bug in the 10.0 GDB where insert performance for PGDB is 3 times slower than it was in 9.3.1 and earlier. I'm wondering if what you are seeing is related to this issue. If you can, please try with FGDB and let us know if your process starts to perform with acceptable performance. If it does not, I'd like to get a reproducible case from you (exact command with parameters and data if possible) so that I can investigate what you are seeing.

Thanks,
Ken
ESRI Geoprocessing Product Engineer

JimPomplun

this problem is not unique to person geodatabase format. I see the calculation slow down in shapefiles, file geodatabases and in enterprise level geodatabases when comparing 10 vs 9.3.1. 9.3.1 wins!

curtvprice

this problem is not unique to person geodatabase format. I see the calculation slow down in shapefiles, file geodatabases and in enterprise level geodatabases when comparing 10 vs 9.3.1. 9.3.1 wins!

Something to check that I didn't see mentioned in this thread yet: if you have a join active, you should disable it - or use the VBScript option in Field Calculator.

Bug" rel="nofollow" target="_blank">http://support.esri.com/en/bugs/nimbus/TklNMDYwNjc1]Bug NIM060675 Field Calculate performs very slowly on a layer with a join when using Python as the parser.

[post]45792[/post]

MattGoodman

I'm running a field calculation concatenating two fields, in an SDE geodatabase point file with 62,000 records. Elapsed time was 2 hours 17 minutes 31 seconds.

There is no join.

I saw some forum discussion recommending using VBA parsing to speed up field calcs when there is a join involved - does that mean that when there is not a join involved that Python parsing is faster?

Any tips/tricks for helping this process run a little faster?

Thanks, -Matt

KenHartling

Our testing in house has not shown an overall slow down in performance between ArcGIS 9.3.1 and 10.0 for the CalculateField tool. With selection sets we've seen improvements in performance at 10.0. We have seen some slowdown when updating field values in an entire feature class (no selection set) when there are tens of millions of records to be updated. We are investigating this for 10.1. If the feature class contains less than 10 million records we've seen improved performance when updating a field value for an entire feature class.

If any of you on this thread have specific examples and data you can share with us we would love to be able to have them so we can investigate what you are reporting. There may be something different about your case that we have not considered in our testing. Any help you can give us in figuring this out would be greatly appreciated. If you are unable to share your case and data through the forum, you can:
1. Submit a technical support incident.
2. Use our ftp site to upload the data and let me know it is there. (I can provide instructions)

When providing us with instructions on how to reproduce the issue you are seeing, please provide:
- The exact parameters you are using in the CalculateField tool when you encounter the issues you have reported.
- The patch level of ArcGIS you have applied.
- If you have comparison numbers between 9.3.1 and 10.0 please provide us with your performance numbers for each version using your test case.
- Please provide the machine specifications for the machines that were used for each version of ArcGIS in your testing.

A few things of note for the CalculateField tool ...

We've addressed a number of performance issues for CalculateField in 10.0 SP3 (coming soon). Once SP3 is available, you may want to apply it and retest to see if what you are reporting has been helped by these fixes.

A fix for the PYTHON parser option slowness when compared to VB is planned to be ported back to 10.0 SP4 (release date TBD).

10.1 (release date TBD) contains a very substantial improvement to working with selection sets in GP tools. CalculateField performance when updating field values using a selection set has been improved dramatically in 10.1. Unfortunately, these changes cannot be ported back to earlier releases.

The ArcSDE team is investigating slow performance when the CalculateField tool is used against ArcSDE. Something to be aware of in our testing of ArcSDE with the CalculateField tool is direct connect connections appear substantially slower than when using application server connections for very large updates using the CalculateField tool.

Thanks for your help.
Ken
ESRI - Senior Geoprocessing Product Engineer

AnnHowland

I also am experiencing the same slowness with ArcMAP 10. I have a direct connection SDE feature class layer with about 70,000 records that I have to use the Toolbox to join this address layer to the update parcels layer to update three fields. The Field Calculation used to take about 30 - 45 minutes for all three fields to recalculate WITH the join. Now it takes ONE field calculation 3-4 HOURS to recalculate. So I spend nine to twelve hours of work time waiting on my computer to recalculate. Because there are 70,000 records, I can't run anything else in the meantime or it takes even longer. What a waste of time and productivity.

KenHartling

Would you be able to share your data (as a fgdb) and the exact steps of setting up the join and running the calculateField tool so we can investigate? If you don't want to share it through the forum just let me know and I can send you instructions for uploading it all to our secure ftp site.

Thanks, Ken
ESRI
Senior GP Product Engineer

I also am experiencing the same slowness with ArcMAP 10. I have a direct connection SDE feature class layer with about 70,000 records that I have to use the Toolbox to join this address layer to the update parcels layer to update three fields. The Field Calculation used to take about 30 - 45 minutes for all three fields to recalculate WITH the join. Now it takes ONE field calculation 3-4 HOURS to recalculate. So I spend nine to twelve hours of work time waiting on my computer to recalculate. Because there are 70,000 records, I can't run anything else in the meantime or it takes even longer. What a waste of time and productivity.

NathenHarp

I am also having trouble with Add Joins and Calculate fields in Arc10.0. Have developed very simple model and script, no changes between database env., .dbf, .gdb. But there is a problem with processing record counts. The model/script is successful with very small datasets, i.e., 50-100.

I ran the model/script in Arc9.3.1 and at ~600,000 records, ~20 minutes. When seperating out the processes and calculating the field in ArcMap, took < 20 secs.

Has this bug been resolved in Arc10.1?

SampleModel.png

CarolWickenheiser

Would you be able to share your data (as a fgdb) and the exact steps of setting up the join and running the calculateField tool so we can investigate? If you don't want to share it through the forum just let me know and I can send you instructions for uploading it all to our secure ftp site.

Thanks, Ken
ESRI
Senior GP Product Engineer

Ken, I would be glad to share our data. We are using the parcel fabric, calculating from a joined field, in ArcMap version 10.0, SDE version 10.0, SQL 2008 R2. I have exported the fabric into a file gdb to upload to ESRI for a separate issue so it is ready to go if you would like me to send it to you. Please send me instructions. Thank you. - Carol

StephaniePatterson

I am having the same problem. I have a parcel geodatabase in a personal geodatabase v10 and it now takes hours to calculate the geometry for the acreage. This is really slowing down our update process.

Stephanie

JohnFell

We are using ArcSDE 10.0 SP 4, SQL 2008 R2, direct connections and using python to execute the CalculateField tool. It takes about 12 minutes to calculate about 70 rows within a child version of default in a parcel fabric. I was curious if there was a performance enhancement request pending or a workaround to this behavior. Thanks.

MathewCoyle

We are using ArcSDE 10.0 SP 4, SQL 2008 R2, direct connections and using python to execute the CalculateField tool. It takes about 12 minutes to calculate about 70 rows within a child version of default in a parcel fabric. I was curious if there was a performance enhancement request pending or a workaround to this behavior. Thanks.

Does using a cursor to calculate the geometry work better? Something like this?

import arcpy

layer = "some_layer_or_path_to_fc"
desc = arcpy.Describe(layer)
shapeField = desc.shapeFieldName
field = "Area_field_to_update"

ucurs = arcpy.UpdateCursor(layer)
for row in ucurs:
    area_set = row.getValue(shapeField).area
    row.setValue(field,area_set)
    ucurs.updateRow(row)

JohnFell

mzcoyle,

After applying your solution the script tool was able to execute successfully in a fraction of the time it took to calculate the selected features. Thank you for the suggestion!

MaryCoburn1

I know this is an old thread, but has anyone found a good workaround this. I am using 10.6 and have created a model builder tool. I put a ‘calculate area’ into my model builder tool to update the area and it takes more time to run this one calculation than it does to run the rest of the model. It started calculating the area 15 hours ago and is still only 32% done. It usually takes 6-8 hours to run the entire model.

I have found that running things in Python is usually faster than using the tools through the toolbox, but this Model Builder tool is for someone else who doesn’t know python so I am trying to make it using tools they can easily change if they need to later.

KenHartling

Hello Mary,

Would I be able to get the model and data from you to investigate?

If so, I'll contact you with info on how to get it to us securely.

Thx. Ken

MaryCoburn1

I will be able to send you the data, but it must only be used for trouble shooting purposes.

KenHartling

After taking a look at Mary's case I was unable to reproduce the performance issues using her data and model. She was able to work around the issue by modifying the order of operations a bit in the model, however, I would guess that there is something on the system being used that is causing the issue. I recommended following some of the troubleshooting issues here when possible.

Mary, thanks for helping us attempt to investigate what you were seeing.

Ken

Nimm an der Diskussion teil

Angemeldete Mitglieder können Beiträge verfassen, Updates folgen und mehr. Neu hier? Registriere ein kostenloses Konto.

Schnellzugriffe

Community Help
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.