topic Re: Deleting double quotation " in field, find/replace errors - script in Python Questions

Deleting double quotation " in field, find/replace errors - script

TurnerNowak — Wed, 12 Jan 2011 21:01:01 GMT

I have several shape files (all Parcels.shp) that I run scripts on in an attempt to clean and standardize the address field (SIT_FULL_S). Whenever a record has a double quotation, I get an error and the script shuts down.

Here is what I am using:

gp.CalculateField_management(fc + "\\Parcels.shp", "SIT_FULL_S", "!SIT_FULL_S!.replace(u'\u0022',u'\u0027')", "PYTHON", "")

example 1 - Script works with this text: 125 South Main Street
example 2 - Script shuts down with this text: 125 South "Main Street"

When I try example 1, the scripts runs.
When I try example 2, the script quits with Error 000539

Long story short, how can I remove any occurances of " from my field so that I can run my address scrubbing scripts ?

Or perhaps I should ask how to modify my below script to ignore the " and continue running. Here is an example of one of my address scrubbing scripts and the log showing the error:

gp.CalculateField_management(fc + "\\Parcels.shp","SIT_FULL_S", "!SIT_FULL_S!.lstrip('0')", "PYTHON")
arcgisscripting.ExecuteError: ERROR 000539: Error running expression: "9030 W SR 2 HWY "A"".lstrip('0') <type 'exceptions.SyntaxError'>: invalid syntax (<string>, line 1)
Failed to execute (CalculateField).

Re: Deleting double quotation " in field, find/replace errors - script

ChrisMathers — Thu, 13 Jan 2011 11:44:51 GMT

Well with python you can use a cursor to go over the features and when it finds a " it can take it out. I had to do that recently on one of my feature classes as well.

Re: Deleting double quotation " in field, find/replace errors - script

TurnerNowak — Thu, 13 Jan 2011 16:32:37 GMT

Well with python you can use a cursor to go over the features and when it finds a " it can take it out. I had to do that recently on one of my feature classes as well.

Great, would you be able to share that with me ?

Re: Deleting double quotation " in field, find/replace errors - script

ChrisMathers — Sat, 11 Dec 2021 21:55:55 GMT

I think I just did it in the python window.

import arcpy
cursor=arcpy.UpdateCursor(your feature class)
for feature in cursor:
    if '"' in feature.YourField: #if a double quote is in the string returned,
        feature.YourField=feature.YourField.replace('"','') #replace that double quote with an empty string (removes it)
        cursor.UpdateRow(feature)

That should work. Let me know if it doesnt.

Re: Deleting double quotation " in field, find/replace errors - script

TurnerNowak — Sat, 11 Dec 2021 21:55:58 GMT

I think I just did it in the python window.

import arcpy
cursor=arcpy.UpdateCursor(your feature class)
for feature in cursor:
    if '"' in feature.YourField: #if a double quote is in the string returned,
        feature.YourField=feature.YourField.replace('"','') #replace that double quote with an empty string (removes it)
        cursor.UpdateRow(feature)

That should work. Let me know if it doesnt.

Could not get that to work so I modified my original replace script but I am getting a EOL error on the following:

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('"',''), "PYTHON", "")

Re: Deleting double quotation " in field, find/replace errors - script

ChrisMathers — Fri, 14 Jan 2011 11:46:10 GMT

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('"','')", "PYTHON", "")

Looks like you forgot the closing quote on the formula string. Not sure why the cursor didnt work though.

Re: Deleting double quotation " in field, find/replace errors - script

TurnerNowak — Fri, 14 Jan 2011 16:47:54 GMT

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('"','')", "PYTHON", "")

Looks like you forgot the closing quote on the formula string. Not sure why the cursor didnt work though.

Yeah, still won't work ! I'll give up on that for now. How about modifying the Python script :

gp.CalculateField_management(fc + "\\Parcels.shp","SIT_FULL_S", "!SIT_FULL_S!.lstrip('0')", "PYTHON")to ignore all rows that contain a " and continue processing the rest of the table ?

I had no idea this would stump so many people. Seems like such a basic function..........

Re: Deleting double quotation " in field, find/replace errors - script

JasonDeMunck — Sun, 16 Jan 2011 02:45:50 GMT

I am running into a similar problem with strings containing double quotes (") when just trying to do a simple calculate field to remove "whitespace" with Python. I would hope there is an elegant solution for dealing with these issues as I would believe they could come up quite regularly. I am using the following code:

" ".join(!FIELDA!.split())

A workaround when the quote never falls on the last character could be ""+!FIELDA+"" which puts the string in triple quotes, this however fails once again if the last character of the string is a double quote.

Why does this process not bracket the string automatically with single quotes when double quotes are present or automatically escape the quotes like a .read() would do?

Re: Deleting double quotation " in field, find/replace errors - script

KimOllivier — Sat, 11 Dec 2021 21:56:00 GMT

I can see that you are not working through the quote characters in a logical way. You cannot hide the double quote string in the middle of a double quoted string with single quotes. You have to use the wonderful escape character (\).

So your calculate expression must be like this:

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('\"','')", "PYTHON", "")

Just visually "unpack" the string to see what is happening. Even better make a separate expression string and print it out to see if you get what is expected.

expression = "!SIT_FULL_S!.replace('\"','')"
print expression
gp.CalculateField_management(fc + "\\Parcels.shp", expression, "PYTHON", "")

the expression without the backslash would be:

"!SIT_FULL_S!.replace('"

...followed by a syntax error: ',")" - an unclosed single quote which you don't find because CalculateField does not run TabNanny before it attempts to run it. So it does not compile and "doesn't work". Actually this is picked up by TabNanny. I recommend that you use it in Pythonwin to trap this sort of syntax error. It is the little tick icon.

This is why it is not a good idea to be sucked into using CalculateField instead of a cursor.
Use a cursor in scripts. Leave CalculateField for Modelbuilder.

With a cursor you can pre-check the syntax with TabNanny (I hope you always do that!), you can expand out the steps to make it clearer, you can test for unexpected data and correct it, and you can add a try/except block to trap more unexpected errors without stopping the whole process. you can print out current variables when it crashes to see what the faulty data was.

Re: Deleting double quotation " in field, find/replace errors - script

TurnerNowak — Sat, 11 Dec 2021 21:56:03 GMT

kimo;69183 wrote:
So your calculate expression must be like this:

gp.CalculateField_management(fc + "\\Parcels.shp", "!SIT_FULL_S!.replace('\"','')", "PYTHON", "")

Thanks for the tip Kimo, but the above and all other ways of escaping still don't work. I tried to research how to do a find/replace using a cursor and came up with this script, but I'm getting an Error: 999999 at the the line: name = row.GetValue("Name").

import arcgisscripting
import logging 
logger = logging.getLogger()

gp = arcgisscripting.create(9.3)
gp.OverWriteOutput = True

gp.Workspace = "C:\ZP4"
fcs = gp.ListWorkspaces("*","Folder")

for fc in fcs:
    print fc
    rows = gp.UpdateCursor("//Parcels.shp")
    row = rows.Next()
    while row:    
        name = row.GetValue("Name")        
        dash = string.find(name, "-")     
        if dash != -1:
            fix = name.replace("-", "")
    del row, rows

Re: Deleting double quotation " in field, find/replace errors - script

Anonymous User — Sat, 11 Dec 2021 21:56:06 GMT

Couple of things. One method I use for this is...

gp.CalculateField_management(fc + "\\Parcels.shp", '"%s" % (!SIT_FULL_S!.replace('"','')'), "PYTHON", "")

Regarding the code and your error. The 999999 is very generic.

Looking at this code, I don't see how it could possibly work as intended.

Why are you looping through all the objects in ListWorkspaces() and then doing rows = gp.UpdateCursor("//Parcels.shp")? Aren't you just reopening the same shape file?

Couldn't you just do?

rows = gp.UpdateCursor("//Parcels.shp")
for row in rows:
    row.name = row.GetValue("Name").replace("-", "")
    rows.updateRow(row)
del row
del rows

import arcgisscripting
import logging 
logger = logging.getLogger()

gp = arcgisscripting.create(9.3)
gp.OverWriteOutput = True

gp.Workspace = "C:\ZP4"
fcs = gp.ListWorkspaces("*","Folder")

for fc in fcs:
    print fc
    rows = gp.UpdateCursor("//Parcels.shp")
    row = rows.Next()
    while row:    
        name = row.GetValue("Name")        
        dash = string.find(name, "-")     
        if dash != -1:
            fix = name.replace("-", "")
    del row, rows

Re: Deleting double quotation " in field, find/replace errors - script

TurnerNowak — Fri, 21 Jan 2011 16:00:18 GMT

Thanks for the suggestions Huey.

When I try your calculate field script I get this:

C:\Documents and Settings\Andrew>"C:\Python24\New Text Document.py"
File "C:\Python24\New Text Document.py", line 19
    gp.CalculateField_management(fc + "\\Parcels.shp", '"%s" % (!SIT_FULL_S!.rep
lace('"','')'), "PYTHON", "")

                      ^
SyntaxError: invalid syntax

And when I try your update cursor script I get this:

C:\Documents and Settings\Andrew>"C:\Python24\update cursor test.py"
C:\ZP4\TEST2
Traceback (most recent call last):
File "C:\Python24\update cursor test.py", line 15, in <module>
    for row in rows:
TypeError: 'geoprocessing cursor object' object is not iterable

Re: Deleting double quotation " in field, find/replace errors - script

LoganPugh — Sat, 11 Dec 2021 21:56:08 GMT

The "for row in rows" syntax was introduced with arcpy in ArcGIS 10. For 9.3 and earlier you'll need to use the following construct:

row = rows.Next()
while row:
   <do stuff>
   row = rows.Next()
del row, rows

Re: Deleting double quotation " in field, find/replace errors - script

TurnerNowak — Sat, 11 Dec 2021 21:56:11 GMT

Thanks lpugh01 , I made those changes and modified a few other lines to come up with a script that works.
Here is what I have now :

import arcgisscripting
import logging 
logger = logging.getLogger()

gp = arcgisscripting.create(9.3)
gp.OverWriteOutput = True

gp.Workspace = "C:\ZP4"
fcs = gp.ListWorkspaces("*","Folder")

for fc in fcs:

    print fc
    rows = gp.UpdateCursor(fc + "//Parcels.shp")
    row = rows.Next()
    while row:
        Name = row.GetValue('APN').replace('"', '')
        print Name
        row.SetValue('Apn', Name)
        rows.updateRow(row)
        row = rows.Next()
    del row
    del rows

Re: Deleting double quotation " in field, find/replace errors - script

PaulBrandt — Fri, 06 May 2011 21:01:44 GMT

From the code suggestions in this thread (credit all who shared) I put together a model tool that you can use straight up with python code blocks to calculate fields when you have double quotes in a field.