How to force Dissolve to put ALL features together?

[MatthewNoteboom]

I'm trying to dissolve a layer with a LOT of polygons to a single multipart so I can place random points throughout (but at less than 1pt/polygon).

However, ArcMap seems to have decided it can't dissolve ALL of the features, just subsets. I've tried in ArcMap, and with a standalone Python script. I've tried without using a dissolve field, and with a dissolve field that has the same value for every feature. Multi-part is on. I can't see anything blocking it from completing the process.

[DanPatterson_Retired]

can you show what is NOT being dissolved?

A visual example of the map and table might bring some suggestions

[Aaron_G_Jones]

Have you tried the Multipart to Singlepart tool in ArcCatalog first?  Located in Data Management Tools/Features/Multipart To Singlepart.  Once you've completed the Dissolve you can "Explode" the features back if needed.

[MatthewNoteboom]

Thanks guys. From reading around (albeit older posts) I think maybe the scale of these layers hits a memory limit when using the Dissolve function.

I have a layer of man-made cover for Canada that was WAY to unwieldy, so I broke it into provinces (which is how we want to handle the next part of analysis anyway), but I think there's still too many features. I think they're all single-part features to start with, I'm actually trying to make one multipart, but I think it might be too ambitious.

I've attached a windows snip of the British Columbia layer, with one of the output features selected. This layer started with 496924 features, and looks to have basically just dissolved 2 blocks. With denser provinces, the output breakdown is 4+ tiles like this.

[KenHartling]

Hi Matthew,

It sounds like you have reached some memory limitations. 

http://desktop.arcgis.com/en/arcmap/latest/tools/data-management-toolbox/dissolve.htm

The availability of physical memory may limit the amount (and complexity) of input features that can be processed and dissolved into a single output feature. This limitation could cause an error to occur, as the dissolve process may require more memory than is available. To prevent this, Dissolve may divide and process the input features using an adaptive tiling algorithm. To determine the features that have been tiled, run the Frequency tool on the result of this tool, specifying the same fields used in the dissolve process for the Frequency Field(s) parameter. Any record with a frequency value of 2 has been tiled. Tile boundaries are preserved in the output features to prevent the creation of features that are too large to be used by ArcGIS.

Caution:

Running Dissolve on the output of a previous dissolve run will rarely reduce the number of features in the output when the original processing divided and processed the inputs using adaptive tiling. The maximum size of any output feature is determined by the amount of available memory at run time; therefore, output containing tiles is an indicator that dissolving any further with the available resources would cause an out-of-memory situation or result in a feature that is unusable. Additionally, running the Dissolve tool a second time on output that was created this way may result in very slow performance for little to no gain and may cause an unexpected failure.

When you see this type of output in ArcMap, please make sure to try the same process using Background 64bit ( http://desktop.arcgis.com/en/arcmap/latest/analyze/executing-tools/64bit-background.htm ). Pro being 64bit should also help but it appears there may have been issues there as well. 

Let me know if you would be willing to share your data with me and I'll send you info on how to share it with us securely.

Thx. Ken

[MatthewNoteboom]

Thanks Ken, the data is all public Canadian topographic/land use data, but I'm not with that organisation anymore so don't have it on hand.

That's the conclusion I came to as well as the classes had silly numbers of features, although it was mostly guesswork as the tool seemed to indicate everything worked - it would be useful if the tools logged when they hit a memory limit. I found a workaround doing some steps in python that probably worked better than my original workflow anyway.

[MatthewNoteboom]

Another follow-up comment:

With ArcGIS Desktop, I was able to handle the two smallest provinces (by feature count), then passed the database onto a colleague with ArcGIS Pro to try the Pairwise dissolve tool which seems to have a bit more capability. That sorted out 4 more provinces, but the 4 largest (by feature count) come out of Pairwise dissolve with 1 feature, but with null geometry. So any ideas for that are welcome.

[DanPatterson_Retired]

Dice  or do a manual 'subsectioning' to reduce the size

The tool in arctoolbox.  You may not have any choice.  I have had issues with some datasets from Ontario and Quebec, 'split" the north off, do it first and move southward in whatever chunks work.  Then merge the results together and do one final dissolve on the merged resultant.