ArcPy Partial/Incomplete Shapefile Union Output

[JaronSwain1]

I'm having difficulty running batch unions using arcpy consistently and reliably. There is a portion in my batch script where I'm running 3 unions however the union output creates partial/incomplete shapefiles that are not complete, thus corrupt themselves when moving onto the next part of the script. The union runs and no errors are generated but the outputs are broken. Only .sbn, .sbx, .xml files are created.

The weird thing is when I run my files separately (not batched) the outputs are complete and normal. Another weird thing is that each time I run the batch scripts different files generate the corruption. It's not the same input files each time.

I'm using a basic license in Python 2.7 and am running 3 unions separately on relatively large datasets (the size of North America).

I'm wondering if there is something I can do to make the script more reliable because sometime it works flawlessly and other times it doesn't? I've tried repairing the geometry on all the data sources as well as switching the input sources from shapefiles to feature classes with no luck.

If anyone has any ideas it would be much appreciated.

Thanks.

[JoeBorgione]

Are you really working on shapefiles or are you using that term to describe file geodatabse features?  If the former, use the latter.  If the latter try performing the union on a feature class where the file geodatabase local rather than file shared.

[JaronSwain1]

The first union inputs are all geodatabase features and the subsequent 2 unions use the output of the previous union with a geodatabase feature for the next union. I'll try having all the inputs and outputs as geodatabase features and then export the final shapefiles from there at the end.

[JoeBorgione]

Can you run the  union tool manually and get the desired results?

[JaronSwain1]

Yes, when I run the tool manually in Desktop or Pro it works. When I run my script on smaller/one-off batches of data it produces desired results. It's only when I run it on my batches of 300+ feature iterations it produces unpredictable results

[JoeBorgione]

 I'll try having all the inputs and outputs as geodatabase features and then export the final shapefiles from there at the end.

Hopefully that'll work for you.

[JaronSwain1]

Thanks Joe! I'll give it a go.