Buffer tool not finishing on large feature class

[MarcCusumano]

I am trying to create a new "Cartway" feature class using the buffer tool on a centerline feature class with ~750,000 features in ArcGIS 10.1 SP1. The tool has basic parameters as inputs, such as "FULL", "ROUND", and "ALL", and background geoprocessing is disabled.

The problem I am having is the tool is stuck at 100% completion and does not finish. However the ArcCatalog.exe process is still working (13% CPU, or 100% / 8 cores). I even let it run over the weekend and it is still not done.

The data and scratch workspace are all on solid state drives (SSD's). There are no "spinners" in this workstation to slow things down, so there should be no bottleneck there. The workstation is a Dell T5500 Precision workstation. The specifications are:

2 x Intel Xeon X5677 (2 physical quad-core CPUs, 8 physical cores total, HT disabled).

20GB RAM

2 x NVIDIA Quadro 600 GPUs

120GB Corsair V12 SSD

120GB Corsair Force GT SSD

Any ideas as to how I can get this to finish?

[MarcCusumano]

I tried breaking up the input feature class and running dissolve on each part, then merging back together. This worked (except for one of the parts, which throws error 99999, even after repairing geometry), but now running dissolve on the merged feature class gives an "out of memory" error.

Does anybody else have any suggestions?

[DuncanHornby]

So you've managed to narrow it down to a single feature? This is good. You could copy that out into its own feature class and manually chop it up into smaller geometries then run that through the buffer and dissolve tools then merge that back into all your other merged data.

[MarcCusumano]

I was able to run the dissolve on 4 / 5 feature classes. The one that fails has 170,000 feature inside of it.

[TOMKONG]

Since the ESRI replaced the data model from Coverage to SHAP to Geodatabase, the ArcGIS software geoprocessing tools can't handle the large dataset, and the error message (memory issue...) is wrong also.

One better solution is to convert the data format to the Coverage format, then running the coverage geoprocessing tool -- the coverage geoprocessing tool can deal with the large dataset.

In order to run coverage tool, you should to have the ArcINFO Workstation installed in your PC, then you can add the Coverage Tools in the Toolbox and to active it to run.

Actually the coverage data model is robust format and much better than SHP & GDB.    

[MarcCusumano]

Still struggling with this. Chopping up the feature class did not work, I am still getting topoengine errors and 9999 general function failures. Also this method is creating gaps between feature all over the state, which is unacceptable.

[MarcCusumano]

We require support from esri on this issue. Who can I contact?

[DanPatterson_Retired]

Up at the very top of this page is the Support and Services link.  If you are the owner of the software, you can report it yourself...if it is a corporate account...someone there (sometimes unknown) can do it for you.

[RichardFairhurst]

Did you read this help file?‌  It specifically mentions the Dissolve tool and road casings as an example of what causes the kinds of failures you are experiencing.  The Dice tool‌ may be a more effective way of removing the errors you are seeing than the partitioning techniques you are using, since it accounts for vertices counts that create the issues you are seeing.

Also, given that this is a buffer, use of the Cartography generalization tools should be considered acceptable, since these buffers are probably not exact boundaries that represent real legal boundaries or any other real world precise information.  Buffering is already a form of generalization, and some additional cartographic generalization may be useful specifically for avoiding the errors you are receiving without necessarily making radical changes to the final output.

For example, Dissolving the lines on name attributes and then using Simplify Line at the start of the process prior to creating buffers could have a significant effect on the vertice count of the buffer output, which is the issue you are running into.  At least it should eliminate vertices that contribute no real geometry angles if you use a small tolerance value, but retaining those extraneous vertices increases the likelihood of tool failure.  If you try the Simplify Line tool review this help on that topic as well.  In addition, or as an alternative, you could also try the Simplify Polygons tool prior to using the Dissolve tool to reduce the vertice count of the buffers.  That may eliminate large numbers of very small true curve segments, which may cause tolerance errors when they are being split during the Dissolve process.

In addition to the issues mentioned in the help about shapefiles and personal geodatabase size limitations, if you are outputting to shapefile or personal geodatabase, the vertice counts will explode to convert true curves at end caps into nodes only to dissolve them away, so I hope you are only doing this with a file geodatabase.  Reducing vertice counts with the Simplify Lines or Simplify Polygon tools also probably helps eliminate many very short true curve segments created by the Dissolve which may also contribute to the geometry errors being reported if you are using a file geodatabase already.  Retaining such very small curves will contribute very little to your buffer data analysis in the end and make any other geoprocessing you may do based on the output problematic.

[MarcCusumano]

Thank you for the suggestions everyone, although still no luck. Every avenue ends up with 9999 general function failure, topoengine errors, or the tool simply does not finish after allowing over 3 days of processing time.

I have now merged the entire street centerline feature class into one single feature and am attempting to run the buffer tool on that without the dissolve option checked, although it is still at 0% after a half hour.

If this does not work I will open a ticket with esri.

[TOMKONG]

Did you try to use Coverage Tool? It can handle large dataset input. I tried few times to run large dataset input by using Coverage Tool (After installing ArcInfo Worksatation, you can have the Coverage Tool in the Toobox), there are no any problems.