Feature class size slowing Down Geoprocessing

1044
7
12-16-2016 03:23 PM
JohnPapageorgiou
New Contributor III

Hi,

I have a polygon feature class that is 170MB in size.  Originally it was 708MB but using simplify and dissolve I managed to get it down to its current size.  The problem is that at times it still takes forever to process.  I created a feature class of 120 buffers and I need to use that clip features from the 170MB feature class.  Is there a way I further reduce the size of the 170MB feature class without sacrificing data integrity?  I need this to be more for functional.  It's been over an hour and the process is only 33% complete and seems to have frozen.  Any help is much appreciated.  I'm using ArcGIS desktop, Advanced 10.3.1.

thanks,

John

0 Kudos
7 Replies
RickeyFight
MVP Regular Contributor

One option is to do that kind of work in ArcCatalog. It really speeds up processing time. 

JohnPapageorgiou
New Contributor III

Thank you for your response.  I have tried arcCatalog but it is still slow. 

thanks,

John

0 Kudos
JayantaPoddar
MVP Esteemed Contributor

Is the data located in local drive or a network drive? (latter one would be much slower). Closing all other applications might help. Run the geoprocessing in Foreground (Backgroud processing disabled).

Foreground and background processing—Help | ArcGIS for Desktop 

I recommend trying the workflow using stand-alone Python script. I hope it would get much faster.

Clip—Help | ArcGIS for Desktop 



Think Location
JohnPapageorgiou
New Contributor III

It's on a network drive.  I'll give the local drive a try.  I have background geoprocessing disabled.  I have the process in a standalone python script right now.  But I'll try the local.  Thanks!

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

This is one of those issues where users eventually learn the best practice, or better practice, but Esri doesn't want to come right out and document it.  Although hosting file geodatabases on network drives/shares is "supported," it is also discouraged in many ways.

Over the years there have been Esri, Microsoft, VMware, and possibly other bugs that impact the functionality or integrity of file geodatabases on network shares, e.g., Problem: NIM090672: Specific Microsoft updates may result in data corruption with file geodatabases ... and Error: Failed to create feature class. The item was not found, [ItemType:{70737809-852C-4A03-9E22-2C...

Additionally, there is a reason for warning 10027 (10027: Layer's data source is referenced via a UNC path) when preparing GIS services for publication.

There may be a performance disadvantage when using UNC paths because network file access is typically slower than direct disk access on your local computer (using absolute paths), which can affect map drawing performance.

However much map drawing performance might be affected, geoprocessing performance will be affected even more because it is more I/O intensive than simply rendering maps.

I am not saying don't use file geodatabases on network shares, we do in the organization I work for now and have since as long as I can remember, but it is important to be informed and aware of the tradeoffs when doing so.  For heavy geoprocessing, I tend to avoid file geodatabases on network shares if at all possible, both for performance and stability reasons.

JohnPapageorgiou
New Contributor III

Thank you for your response Joshua, interesting information, the errors and problems you shared are helpful to know.  I think we may have encountered some of these issues without realizing the cause. 

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor
0 Kudos