topic Re: Feature class size slowing Down Geoprocessing in Geoprocessing Questions

Feature class size slowing Down Geoprocessing

JohnPapageorgiou — Fri, 16 Dec 2016 23:23:29 GMT

Hi,

I have a polygon feature class that is 170MB in size. Originally it was 708MB but using simplify and dissolve I managed to get it down to its current size. The problem is that at times it still takes forever to process. I created a feature class of 120 buffers and I need to use that clip features from the 170MB feature class. Is there a way I further reduce the size of the 170MB feature class without sacrificing data integrity? I need this to be more for functional. It's been over an hour and the process is only 33% complete and seems to have frozen. Any help is much appreciated. I'm using ArcGIS desktop, Advanced 10.3.1.

thanks,

John

Re: Feature class size slowing Down Geoprocessing

RickeyFight — Fri, 16 Dec 2016 23:38:29 GMT

One option is to do that kind of work in ArcCatalog. It really speeds up processing time.

Re: Feature class size slowing Down Geoprocessing

JayantaPoddar — Fri, 16 Dec 2016 23:45:56 GMT

Is the data located in local drive or a network drive? (latter one would be much slower). Closing all other applications might help. Run the geoprocessing in Foreground (Backgroud processing disabled).

Foreground and background processing—Help | ArcGIS for Desktop

I recommend trying the workflow using stand-alone Python script. I hope it would get much faster.

Clip—Help | ArcGIS for Desktop

Re: Feature class size slowing Down Geoprocessing

JohnPapageorgiou — Fri, 16 Dec 2016 23:46:03 GMT

Thank you for your response. I have tried arcCatalog but it is still slow.

thanks,

John

Re: Feature class size slowing Down Geoprocessing

JohnPapageorgiou — Fri, 16 Dec 2016 23:49:13 GMT

It's on a network drive. I'll give the local drive a try. I have background geoprocessing disabled. I have the process in a standalone python script right now. But I'll try the local. Thanks!

Re: Feature class size slowing Down Geoprocessing

DanPatterson_Retired — Sat, 07 Jan 2017 03:08:26 GMT

moved to https://community.esri.com/community/gis/analysis/geoprocessing?sr=search&searchId=dfe532ad-7710-443d-b103-1a774c2fc340&searchIndex=0‌

Re: Feature class size slowing Down Geoprocessing

JoshuaBixby — Sat, 07 Jan 2017 14:58:36 GMT

This is one of those issues where users eventually learn the best practice, or better practice, but Esri doesn't want to come right out and document it. Although hosting file geodatabases on network drives/shares is "supported," it is also discouraged in many ways.

Over the years there have been Esri, Microsoft, VMware, and possibly other bugs that impact the functionality or integrity of file geodatabases on network shares, e.g., Problem: NIM090672: Specific Microsoft updates may result in data corruption with file geodatabases or shapefiles created on a network share and Error: Failed to create feature class. The item was not found, [ItemType:{70737809-852C-4A03-9E22-2CECEA5B9BFA}].

Additionally, there is a reason for warning 10027 (10027: Layer's data source is referenced via a UNC path) when preparing GIS services for publication.

There may be a performance disadvantage when using UNC paths because network file access is typically slower than direct disk access on your local computer (using absolute paths), which can affect map drawing performance.

However much map drawing performance might be affected, geoprocessing performance will be affected even more because it is more I/O intensive than simply rendering maps.

I am not saying don't use file geodatabases on network shares, we do in the organization I work for now and have since as long as I can remember, but it is important to be informed and aware of the tradeoffs when doing so. For heavy geoprocessing, I tend to avoid file geodatabases on network shares if at all possible, both for performance and stability reasons.

Re: Feature class size slowing Down Geoprocessing

JohnPapageorgiou — Mon, 09 Jan 2017 23:41:12 GMT

Thank you for your response Joshua, interesting information, the errors and problems you shared are helpful to know. I think we may have encountered some of these issues without realizing the cause.