Error 000210 and 000354 when running Union analysis in a Python script.

822
12
01-17-2020 08:31 AM
MattBeal
New Contributor III

Hello everyone! I've come across an issue that has completely stumped our entire organization. I'm currently working through the issue with ESRI support (who also seems stumped) as well but wanted to see if the broader community had any experience with this issue as well.

We have a python script that compares two cities and extracts the differences and doctors them up a bit for project standards. Naturally, the first stage of the script is calls for a union. On the first set of cities that I wanted to compare, I got the following error: 


PS Microsoft.PowerShell.Core\FileSystem::\\havc.local\misc\GIS\Regional\Boundary and Annexation Survey (BAS)\2020 BAS> python BASupdate.py
If the city is two words, do not add a space.

Enter city:
Aiken


Input Folder: \\havc.local\misc\GIS\Regional\Boundary and Annexation Survey (BAS)\2020 BAS\Aiken
MXD Path: \\havc.local\misc\GIS\Regional\Boundary and Annexation Survey (BAS)\2020 BAS\Aiken\AikenBAS.mxd
env.workspace: \\havc.local\misc\GIS\Regional\Boundary and Annexation Survey (BAS)\2020 BAS\Aiken\Aiken_2020.gdb
Creating the change polygons...
Traceback (most recent call last):
File "BASupdate.py", line 56, in <module>
arcpy.Union_analysis([censusData, havcData], unionFC)
File "C:\Program Files (x86)\ArcGIS\Desktop10.5\ArcPy\arcpy\analysis.py", line 628, in Union
raise e
arcgisscripting.ExecuteError: ERROR 000210: Cannot create output \\havc.local\misc\GIS\Regional\Boundary and Annexation Survey (BAS)\2020 BAS\Aiken\Aiken_2020.gdb\union.shp.shp
ERROR 000354: The name contains invalid characters
Failed to execute (Union).

PS Microsoft.PowerShell.Core\FileSystem::\\havc.local\misc\GIS\Regional\Boundary and Annexation Survey (BAS)\2020 BAS>

As you can see, it looks like it is trying to export a shapefile into a geodatabase, but that is not what the code is telling it to do at all. Here is the entire code snippet up until that point: 

import os
import sys
import arcpy
from arcpy import env
import shutil
import subprocess


# Folder location will need to be changed every year
# There are two instances of the year in the script. Use the search function to find them.
basFolder = '\\\\havc.local\\misc\\GIS\\Regional\\Boundary and Annexation Survey (BAS)\\2020 BAS\\'


print "If the city is two words, do not add a space.\n"


# user to input the city name for the script
userInput = raw_input('Enter city:\n')
inputFolder = os.path.join(basFolder, userInput)
basReturnDirectory = os.path.join(basFolder, userInput + '\\BASReturn')


# check if folder exists
if os.path.exists(inputFolder):

print "\n"
# print out the input folder as specified by the user
print "Input Folder: " + inputFolder

# create BASReturn folder for final deliverables
if not os.path.exists(basReturnDirectory):
os.makedirs(basReturnDirectory)

# set the environment to list the mxds
arcpy.env.workspace = inputFolder

for file in arcpy.ListFiles("*.mxd"):

mxd_path = os.path.join(inputFolder, file)
print "MXD Path: " + mxd_path

# reset the environment to the database
env.workspace = inputFolder + '\\' + userInput + '_2020.gdb'
print "env.workspace: " + env.workspace
# map variables
mxd = arcpy.mapping.MapDocument(mxd_path)
df = arcpy.mapping.ListDataFrames(mxd, "*")[0]
havcData = arcpy.mapping.ListLayers(mxd, "HavcData", df)
censusData = arcpy.mapping.ListLayers(mxd, "CensusData", df)
unionFC = "union"
union_m2s = "union_multisingle"

# create the change polygons
print "Creating the change polygons..."

arcpy.Union_analysis([censusData, havcData], unionFC)

It seems to not be recognizing the env.workspace as being a geodatabase for some reason. 

As if that wasn't strange enough, I decided to try the tool on a different city and it worked flawlessly. That city's polygon's came out of the same feature classes that the one that failed did as well. Does anyone have any suggestions on what may be causing this? If I can provide any more information, please let me know! Thanks so much!

0 Kudos
12 Replies
DanPatterson_Retired
MVP Esteemed Contributor

Cannot create output \\havc.local\misc\GIS\Regional\Boundary and Annexation Survey (BAS)\2020 BAS\Aiken\Aiken_2020.gdb\union.shp.shp

periods, spaces, brackets also there are single backslashes in the path name and the workspace is indeed a geodatabase, but the output featureclass(?) is a shapefile (?) with a double .shp.shp extension.

You can't test on a simple folder can you?

also...    "\union.shp.shp" yields a "unicode" error in python since \u is reserved... you could try renaming it so it doesn't begin with an a, b, t, g (plus a few others).

Path names and filenames are now complicated in the world of unicode

MattBeal
New Contributor III

So I may have had a breakthrough. I'm still testing, but I think the issue was a corrupt file geodatabase. My train of thought was that it seemed to be trying to export a .shp file into a file geodatabse, thus it must not recognize that the workspace is a file geodatabase (despite) having the .gdb extension. So I created a brand new, empty file gdb and hardcoded that as the workspace and bam. Ran flawlessly. That leaves a new mystery, however. Why did multiple (brand new) file geodatabases get corrupted every time I ran through the workflow leading up to the script for this one city, but not another? I started the workflow from scratch several times with completely new data, and each time led to the same results. Very strange!

0 Kudos
MattBeal
New Contributor III

So I have confirmed that it is indeed an issue with the workspace. If I replace this line: 

env.workspace = inputFolder + '\\' + userInput + '_2020.gdb'

 with a hardcoded test geodatabase, the script runs perfectly. I just can't figure out why this issue is only occuring for one particular city. It seems like if it is a workspace problem, then it should affect all cities. I'm genuinely stumped. 

0 Kudos
JoshuaBixby
MVP Esteemed Contributor

What is the city?  Maybe it is a Unicode character issue.

DanPatterson_Retired
MVP Esteemed Contributor

Matt did you explore my comment about parts of paths that begins with certain letters?

Like 

a, b, t, g 

they yield unicode errors.

"\aplace"

'\x07place'
MattBeal
New Contributor III

Holy cow. That was it. Replaced the first letter with a P since I ran the tool successfully on a city that started with P previously and it ran. One thing I still don't quite understand, however is that this tool worked the last several years using the same cities. The unicode character didn't apparently become an issue until this year. The script hasn't changed at all aside from replacing 2019 with 2020. Can you think of what might have caused it to all of a sudden become a problem now? 

More importantly, is there a way I can prevent this? I'm not familiar with unicode at all (I'm just barely above familiar with python. Lol). So how can I ensure that it is not going to use unicode for certain user inputs that just happen to start with a unicode character?

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

switching from arcmap to Pro would do it since the python versions changes and more unicode issues began to surface.

To mitigate this you need to provide paths in 1 of 3 ways

  • double backslashes  "c:\\A_folder\\agdb.gdb"
  • raw encoding (little r goes a long way)   r"c:\A_folder\agdb.gdb"
  • forward slashes   "c:/A_folder/agdb.gdb"
  • there are combinations of characters that translate to non-visible characters in unicode so what you see isn't necessarily what is there
  • \t, \b, \a are a couple of the obvious ... but I will leave that for you to explore
0 Kudos
MattBeal
New Contributor III

The goal for next year is to have the scripts working in Pro, however we have not had time to make the change this year.

Given that the workspace, is a concatenation of variables and strings, how can I make this work? The segment of code causing the problem is below. 

env.workspace = inputFolder + '\\' + userInput + '_2020.gdb'

Specifically, the '\\' + userInput was causing a problem when combined with certain characters as you pointed out. changing the userInput variable to something that started with a 'P' fixed the problem. But how can I account for this in the code and ensure that it is looking at the whole workspace as one file path?

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

userInput = "hello"

inputFolder = r"{}\{}_2020.gdb".format(r"c:\SomeFolder", userInput)

inputFolder
'c:\\SomeFolder\\hello_2020.gdb'

raw encoding and python mini-format language

0 Kudos