How can I speed up a simple dissolve?

Question

I have a singlepart featureclass of polygons (2.5M) with an indexed key that I want to convert to multipart polygons. There are less than 0.5% (12,480) that are actually multipart, the rest are single parts. They are all reasonable multiparts, nothing silly that spans the whole dataset, no universal polygon or unreasonably large polygon such as a huge road casing.

Since there are only a few to process I thought: "Aha, select the polygons with multiple keys, process these and merge them back with the remaining singles". That dissolve of the subset does complete in human time.

But I can't complete the process because merging single and multipart features is not permitted. Who knew that? I thought multiparts were to be avoided for analysis, now I know.

Is there a way of merging multiparts without rewriting the entire tool in Python? I didn't think that multipart was a different feature type from single part. I had thought that a single part feature was just a multipart with one part. Is that not so?

The simple way (Dissolve_management) on the whole featureclass takes hours for sample sets (and fails to complete when it runs out of memory on the full set), giving me time to write this... ah here it is: 'unable to read memory, crashed'.

I should expect this by now, nearly any process on a large dataset seems to run out of memory far too often. Just a simple relate with a selection (keyfile select) takes hours just to select and fails to write a subset, compared to Workstation which takes seconds. I have tried using a Python set and creating an SQL query as a layer but this is also very slow when it comes to writing out the records.

Maybe I should just write out the records when running a cursor to test the key against the set and write them out manually? But that is rewriting the tool which I hoped to avoid. I am using local file geodatabases and 9.3, Duo Quad (8 processors) and 4 GB of memory,  TB disk.

Any other ideas?

ChrisSnyder · Answer

parcel labels as multiparts

I don't understand why they have to be multipart... Is it becasue the parcels themselves are (sometimes) multipart polygons)? Why not just first break the parcels into singleparts (MultipartToSinglePart tool) and then get rid of any resulting labelpoints that share the same PARCEL_ID?

KimOllivier · Answer

Kim, I don't seem to have any issues creating a FC with mixed states of single and multipart (made a singlepart dissolve, and then a multipart dissolve, and merge together - no error). What error do you get? I think maybe I'm not grasping your issue?

Polygons are inherently multipart so the dissolved and undissolved subsets merged successfully.

It applies to points. There is a Point featuretype and Multipoint featuretype which cannot be merged. 
I now have the difficult decision to have all parcel labels as multiparts or create two layers which then makes searching a single table, easy joins, counting all more difficult. I could abandon points and store the attributes on the polygons directly, but that has other inconveniences such as polygon label placement and text rotation is not possible.

KathrinSchulte-Braucks · Answer

Hm, unfortunately in my case setting the affinity did not solve my problem. Another tip is to set in msconfig that only one core should be used.