Combining Polygon Shapefile issues

[ChristopherClary]

I am trying to spatially combine 47 polygon shapefiles that contain county parcel data in ArcMap 10. The most obvious choice to me is to merge the shapefiles and I have had some success merging some of them together. The problem lies in the fact that I continuously get errors, especially with the Zip Code field.

I am only interested in merging 7 of the sometimes 50 fields in each shapefile attribute table. Often, I had to create new fields and field calculate over the data in order to have common named fields among the 47 shapefiles. The seven fields contain PIN, acreage, owner name, owner address, owner city, owner state, and owner zip code data.

Most of the time I get errors, they are attributed to the zip code field or long address fields. I was able to fix some of the errors (i.e. creating a VBScript to remove the dash and +4 from some zip codes and addressing any blank fields). Additionally, moving the merging files into the same folder has also alleviated some of the previous problems I experienced. I solved some of the long address issues by making sure all of the created fields allowed 254 characters (I mistakenly started out creating the address fields with only the 50 character limit selected). All of the fields are text except Acres (Double).

At this point in time, I am just overly frustrated with the process and wonder if there is a better or easier way to achieve my intended result. I have tried performing a union and a spatial join (not even sure if it would have achieved what I wanted) and have received errors with them as well.

Questions I do not know the answer to:

Will removing the unnecessary fields from all of the 47 attribute tables help? I have been removing them in the Field Map section of the Merge tool in order to save the effort of having to manually remove them from each attribute table.

Are blank fields to blame? The errors typically stall on fields that have names to them though.

Why can I not merge several of the shapefiles at once? I understand that some of them have over 300,000 rows and I am trying to merge a considerable amount of information but my computer is only a few months old and should be able to handle pretty much any geoprocess in Arc (especially when I turn off background processing).

Thank you for any assistance anyone can provide. I have searched up and down and left and right for a possible solution

[ChristopherClary]

I have been having much more recent success (of course it happens once I create a post about my problems). The only thing I really changed was having the output end up in the same folder as the files I'm merging. Also, I have been loading the files into the TOCs and then dragging them into the Merge tool rather than loading them from their folder location.

[ChristopherClary]

The main error I keep receiving is "000278 : <value> error(s) have been detected for layer <value>. Errors are described in file GL<value>.log in your temp directory."

I was able to merge 7 of the shapefiles and another 8 separately. When I tried to combine the 15 together, I received error 000278 on a field that had no problems in the previous separate merge.

[JoeBorgione]

If you google error 000278 you'll see a few responses.  One of them mentions field length being too short as a problem that tosses this error.

Merging feature classes with different data schemas is painful, as you have discovered.  With so many to do, loading them into a standardized feature class may not be optimal, but the simple data loader is one of my favororite tools.

[ChristopherClary]

I just don't understand why I am getting the errors since I created the fields and made sure that all of them had the same field text length attributed to them. The particular solution I'm discovering now is the order of the datasets in the merge is key. Also, the majority of the errors I'm receiving are for fields in a second dataset that have more characters in their fields than the first dataset. If I switch the merge order around, it is solving some problems, especially the zip code problem.

[JoeBorgione]

Hang in there; it sounds like you'll have a killer piece of data once your are done!

[KenCarrier]

You could write a model to loop through the folder where the shapefiles resides and delete the fields.

When geoprocessing runs it takes into account the entire table first I believe, unless otherwise specified in a tools parameters.

Limiting fields should help things merge more smoothly.

I would try working with iterators inside model builder or try to write a script in python.

Merging the files together should not be so much of a pain if the schemas match exactly, I know you did not want to have to delete each field manually but you can actually do this programmatically or through a model using the iterators which are new at 10.0.

[ChristopherClary]

Thanks for the suggestions and support. It would likely take me as long to create a model/python script for the process than to do it all manually thanks to my inexperience in those areas. I've had some pretty good luck just going in and managing the merges one shapefile at a time and I think I'll keep testing my luck in that direction.