Deleting double quotation " in field, find/replace errors - script

[TurnerNowak]

I have several shape files (all Parcels.shp) that I run scripts on in an attempt to clean and standardize the address field (SIT_FULL_S).  Whenever a record has a double quotation, I get an error and the script shuts down.

Here is what I am using:

gp.CalculateField_management(fc + "\\Parcels.shp", "SIT_FULL_S", "!SIT_FULL_S!.replace(u'\u0022',u'\u0027')", "PYTHON", "")

example 1 - Script works with this text:          125 South  Main Street
example 2 - Script shuts down with this text:  125 South "Main Street"

When I try example 1, the scripts runs.
When I try example 2, the script quits with Error 000539

Long story short, how can I remove any occurances of " from my field so that I can run my address scrubbing scripts ?


Or perhaps I should ask how to modify my below script to ignore the " and continue running. Here is an example of one of my address scrubbing scripts and the log showing the error:

gp.CalculateField_management(fc + "\\Parcels.shp","SIT_FULL_S", "!SIT_FULL_S!.lstrip('0')", "PYTHON")
arcgisscripting.ExecuteError: ERROR 000539: Error running expression: "9030 W SR 2 HWY "A"".lstrip('0') <type 'exceptions.SyntaxError'>: invalid syntax (<string>, line 1)
Failed to execute (CalculateField).

[ChrisMathers]

Well with python you can use a cursor to go over the features and when it finds a " it can take it out. I had to do that recently on one of my feature classes as well.

[TurnerNowak]

Well with python you can use a cursor to go over the features and when it finds a " it can take it out. I had to do that recently on one of my feature classes as well.

Great, would you be able to share that with me ?

[ChrisMathers]

I think I just did it in the python window.

import arcpy
cursor=arcpy.UpdateCursor(your feature class)
for feature in cursor:
    if '"' in feature.YourField: #if a double quote is in the string returned,
        feature.YourField=feature.YourField.replace('"','') #replace that double quote with an empty string (removes it)
        cursor.UpdateRow(feature)

That should work. Let me know if it doesnt.

[TurnerNowak]

I think I just did it in the python window.

import arcpy
cursor=arcpy.UpdateCursor(your feature class)
for feature in cursor:
    if '"' in feature.YourField: #if a double quote is in the string returned,
        feature.YourField=feature.YourField.replace('"','') #replace that double quote with an empty string (removes it)
        cursor.UpdateRow(feature)

That should work. Let me know if it doesnt.

Could not get that to work so I modified my original replace script but I am getting a EOL error on the following:

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('"',''), "PYTHON", "")

[ChrisMathers]

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('"','')", "PYTHON", "")

Looks like you forgot the closing quote on the formula string. Not sure why the cursor didnt work though.

[TurnerNowak]

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('"','')", "PYTHON", "")

Looks like you forgot the closing quote on the formula string. Not sure why the cursor didnt work though.

Yeah, still won't work !  I'll give up on that for now. How about modifying the Python script :

gp.CalculateField_management(fc + "\\Parcels.shp","SIT_FULL_S", "!SIT_FULL_S!.lstrip('0')", "PYTHON")to ignore all rows that contain a " and continue processing the rest of the table ?

I had no idea this would stump so many people. Seems like such a basic function..........

[JasonDeMunck]

I am running into a similar problem with strings containing double quotes (") when just trying to do a simple calculate field to remove "whitespace" with Python.  I would hope there is an elegant solution for dealing with these issues as I would believe they could come up quite regularly.  I am using the following code:

" ".join(!FIELDA!.split())

A workaround when the quote never falls on the last character could be ""+!FIELDA+"" which puts the string in triple quotes, this however fails once again if the last character of the string is a double quote.

Why does this process not bracket the string automatically with single quotes when double quotes are present or automatically escape the quotes like a .read() would do?

[KimOllivier]

I can see that you are not working through the quote characters in a logical way. You cannot hide the double quote string in the middle of a double quoted string with single quotes. You have to use the wonderful escape character (\).

So your calculate expression must be like this:

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('\"','')", "PYTHON", "")

Just visually "unpack" the string to see what is happening. Even better make a separate expression string and print it out to see if you get what is expected.

expression = "!SIT_FULL_S!.replace('\"','')"
print expression
gp.CalculateField_management(fc + "\\Parcels.shp", expression, "PYTHON", "")

the expression without the backslash would be:

"!SIT_FULL_S!.replace('"

...followed by a syntax error:  ',")" - an unclosed single quote which you don't find because CalculateField does not run TabNanny before it attempts to run it. So it does not compile and "doesn't work". Actually this is picked up by TabNanny. I recommend that you use it in Pythonwin to trap this sort of syntax error. It is the little tick icon.

This is why it is not a good idea to be sucked into using CalculateField instead of a cursor.
Use a cursor in scripts. Leave CalculateField for Modelbuilder.

With a cursor you can pre-check the syntax with TabNanny (I hope you always do that!), you can expand out the steps to make it clearer, you can test for unexpected data and correct it, and you can add a try/except block to trap more unexpected errors without stopping the whole process. you can print out current variables when it crashes to see what the faulty data was.

[TurnerNowak]

kimo;69183 wrote:
So your calculate expression must be like this:

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('\"','')", "PYTHON", "")

Thanks for the tip Kimo, but the above and all other ways of escaping still don't work.  I tried to research how to do a find/replace using a cursor and came up with this script, but I'm getting an Error: 999999 at the the line:  name = row.GetValue("Name").

import arcgisscripting
import logging 
logger = logging.getLogger()

gp = arcgisscripting.create(9.3)
gp.OverWriteOutput = True

gp.Workspace = "C:\ZP4"
fcs = gp.ListWorkspaces("*","Folder")

for fc in fcs:
    print fc
    rows = gp.UpdateCursor("//Parcels.shp")
    row = rows.Next()
    while row:    
        name = row.GetValue("Name")        
        dash = string.find(name, "-")     
        if dash != -1:
            fix = name.replace("-", "")
    del row, rows