Merging multiple featureclasses...

Question

I was wondering if anyone could help me out:

I am trying to merge a large amount of line featureclasses - 100+ - and I am getting 'ERROR 999999: Workspace or data source is read only; Failed to execute merge'

Now, I have tried using the Merge tool and using Python and I get the same results with both methods. However, if I try it with only say, 10 or so featureclasses instead of 100+ it works fine. And I don't believe the error message about 'read only' because this happens not matter where I put the folder I am working in; the root of the C-drive, on my desktop, over the network to another drive, etc...I am guessing it has something to do with memory maybe?

I have a 64-bit system, w/ 4GB of RAM, and way too much hard drive space 🙂

Any ideas or suggestions would be much appreciated.

KimOllivier · Answer

In your script parameters ask for a folder or workspace not featureclasses.
Also ask for a wildcard string, default to "*".
This avoids the long pathnames because you are using a workspace setting.
It also removes the tedious cut and paste of hundreds of featureclasses (or shapefiles)
Note that the wildcard string can be something like "B*" or even an exact match to limit the list.
import arcpy,sys
wkspace = sys.argv[1] # or arcpy.GetParameterAsText(0)
wild = sys.argv[2] # or arcpy.GetParameterAsText(1)
arcpy.env.workspace = wkspace
lstFC = arcpy.ListFeatureClasses(wild)
# process the list...

shawnherrick · Answer

Great, thanks!

It was definitely the long paths causing this particular problem. As soon as I shortened them, it worked like a charm - that is, I was able to access the data from another path which was much shorter.

However, I am not sure what you mean by, "...always write my scripts to set a folder with a wildcard string...". Could you explain this a bit more or give an example?

I appreciate the help,
Thank you 🙂

KimOllivier · Answer

I doubt your are running out of memory, just overflowing the buffer that contains the list of input files.

Unfortunately the input form expands every featureclass to the full path. I am guessing that you have excessively long paths so you will have a buffer length of say 100 x 100 characters = 10,000 chars which is more than a likely 8K block limit in the code.

This was a problem when we used sys.argv[] to pass parameters, and the arcpy.getParameter is supposed to be unlimited.
You could test this by making your paths much smaller.

I always write my scripts to set a folder and a wildcard string, and then assemble the list of files without the full path in the script.

Then set the workspace to the folder to execute the merge. Saves specifying the list in the input form, which is more general, but a nuisance for a special case.

I have found other limits merging rasters, max 20 at a time. In that case I did a staged merge.
First merge all the images in a row, then merge the rows into a block, then merge the blocks.
It was easy to do from the names of the image tiles which had a good pattern. Works much faster too.