topic Re: Multiple replacements with Field Calculations using Python in Petroleum Questions

Multiple replacements with Field Calculations using Python

LeonardSpringer — Wed, 26 Aug 2015 13:18:26 GMT

I am able to make replacements one at a time following these instructions:

Simple calculations

Simple string examples

Strings are supported by a series of Python string functions, including capitalize, rstrip, and replace.

Replace any occurrences of "california" with "California" found in the field STATE_NAME.

!STATE_NAME!.replace("california", "California")

How do I make several replacements at a time? For example, how do I change “california” and “cal” and “CA” and “Cal” to “California” and “florida” and “fla” and “FL” and “Fla” to “Florida” without having to create a new field for STATE NAME for each individual replacement?

Thanks, Len Springer

Re: Multiple replacements with Field Calculations using Python

ChrisDonohue__GISP — Wed, 26 Aug 2015 15:08:32 GMT

Just an idea - I guess part of this could come down to whether one could use wildcards in Python. I don't know the exact answer, but can outline the concept. For example, if you know your data will only include state names expressed in various abbreviations, something like:

!STATE_NAME!.replace("ca#", "California")

where # is the wildcard allowing unlimited characters after it (whatever the appropriate Python wildcard would be).

I'm pretty sure the Python Guru's here on GeoNet would know. I'm curious to see what folks come up with.

Chris Donohue, GISP

Re: Multiple replacements with Field Calculations using Python

ChrisDonohue__GISP — Wed, 26 Aug 2015 15:14:50 GMT

A quick search revealed this thread regarding Python wildcards. Apparently in Python * is the one for 'one or more characters'.

They also mention other ways to do the search that may be usable.

http://stackoverflow.com/questions/11427138/python-wildcard-search-in-string

Chris Donohue, GISP

Re: Multiple replacements with Field Calculations using Python

ChrisDonohue__GISP — Wed, 26 Aug 2015 18:39:49 GMT

Hmmm, maybe it is time to call in some Python experts:

Xander Bakker

Dan Patterson

Darren Wiens

Chris Donohue, GISP

Re: Multiple replacements with Field Calculations using Python

IanMurray — Sat, 11 Dec 2021 07:45:13 GMT

I used a function to do something like this and it should be adaptable for what you are needing. I had a table with mixed upper and lower cases no's and Nones, that I all wanted "No". Also replaced all "yes" with "Yes". The function allows you to use loops and other python tools, instead of a single line of code.

If you know all the exact strings you need replaced, you can put them in lists, otherwise wildcards could be used.

Python Parser, click Show Code Block

Pre-Logic Script Code

def Rename(field):
  type1_replace = [ "no" , "None"]
  type2_replace = "yes"
  for n in type1_replace:
    field = field.replace(n , "No")
  field = field.replace(type2_replace , "Yes")
  return field

New Field = Rename(!YourFieldHere!)

Edit: Ah heck, I'll post a relevant to you example.

def Rename(field):
  type1_replace = ["california" ,  "CA" , "cal" ]
  type2_replace = [ "florida" , "FL" , "fl"]
  for n in type1_replace:
    field = field.replace(n , "California")
  for n2 in type2_replace:
    field = field.replace(n2 , "Florida")
  return field

Re: Multiple replacements with Field Calculations using Python

IanMurray — Wed, 26 Aug 2015 19:56:29 GMT

Also, why would you need to create a new field each time? You could run the field calculator over and over again on the existing field if you wanted. Calculate Field can be done on an existing field or a new empty field.

Re: Multiple replacements with Field Calculations using Python

ChrisDonohue__GISP — Wed, 26 Aug 2015 20:22:11 GMT

Thanks for the code example, Ian.

In regards to the new fields, I think Leonard means making a new line for each replacement, not the literal making a new field. For example, having to do this to account for all the possible combinations:

!STATE_NAME!.replace("california", "California")

!STATE_NAME!.replace("ca", "California")

!STATE_NAME!.replace("cal", "California")

!STATE_NAME!.replace("cali", "California")

etc....

Chris Donohue, GISP

Re: Multiple replacements with Field Calculations using Python

DarrenWiens2 — Sat, 11 Dec 2021 07:45:15 GMT

You can do this using a dictionary, where the key is the state name, and the value is a list of possible matches:

def customReplace(search):
    dict = {'California':['c','ca','cal'], 'Florida': ['f','flor','fla']} # build your custom dictionary
    for k,v in dict.iteritems(): # loop through key/value pairs
        for abbrev in v: # loop through the abbreviation list
            if search == abbrev: # compare
                return k # return the state
    return search # else return original

Re: Multiple replacements with Field Calculations using Python

ChrisDonohue__GISP — Wed, 26 Aug 2015 20:59:39 GMT

Darren - is there also a way to do wildcards instead of a dictionary? Like if the values to be evaluated can only be state names, so any combination starting with "Wy...." is going to be "Wyoming". This would save having to type out every possible combination people use to abbreviate Wyoming (and other states).

Chris Donohue, GISP

Re: Multiple replacements with Field Calculations using Python

DanPatterson_Retired — Sat, 11 Dec 2021 07:45:18 GMT

no fancy function but the logic is the same as dictionaries...what is easiest to convert your data to??? If your table has missing keys...I have opted to leave out the flawed entry. You can alter this behaviour.

>>> data = ['CA', 'Fla', 'Cal', 'FL', 'Ontario']
>>> innies = [[["california", "cal","CA","Cal"],"California"],[["florida","fla","FL","Fla"],"Florida"]]
>>> outies=[]
>>> for j in data:
...  for i in innies:
...  if j in i[0]:
...    outies.append(i[1])
...   
>>> outies
['California', 'Florida', 'California', 'Florida']
>>>

Re: Multiple replacements with Field Calculations using Python

LeonardSpringer — Wed, 26 Aug 2015 21:09:39 GMT

Thank you, Ian. That works for me.

Re: Multiple replacements with Field Calculations using Python

DarrenWiens2 — Sat, 11 Dec 2021 07:45:21 GMT

Welllll, you're welcome to explore the regex library, but I've only done so cursorily. Easier, you could make a dictionary with the full name paired with the shortened name, then compare the first two characters against that. Note that you'll have issues in cases like Arkansas and Arizona:

dict = {'California':'ca', 'Florida':'fl'} # and so on
for k,v in dict.iteritems(): # loop through key/value pairs  
        for abbrev in v: # loop through the abbreviation list  
            if search[:2] == abbrev: # compare first two characters
                return k # return the state  
    return search # else return original

Re: Multiple replacements with Field Calculations using Python

ChrisDonohue__GISP — Wed, 26 Aug 2015 21:31:30 GMT

I'll have to have a word with Congress about our state names. Maybe if we renamed them to be more distinctly unique, think of all the money saved and confusion avoided....

Chris Donohue, GISP

Re: Multiple replacements with Field Calculations using Python

IanMurray — Wed, 26 Aug 2015 21:47:12 GMT

Definitely a more pythonic solution and in the case of having more states would be significantly improved over my version.

I should really use dictionaries more often, but since I only use python with arcgis I haven't used them alot(exception of storing data from cursors with them).

Re: Multiple replacements with Field Calculations using Python

JoshuaBixby — Sat, 11 Dec 2021 07:45:23 GMT

There are numerous ways to address this issue, some of which have already been raised here. A Google search for "python replace multiple strings" will yield even more ideas, some of them with fairly in-depth discussions.

If you want to stick with using the Field Calculator, then you are going to have to use code blocks. Another option would be to use ArcPy cursors, the Data Access ones are the newest and best.

An approach I have used in the past that is simple, and seems to perform well enough with my data sets, is a variation of a suggestion from a Stack Exchange/Overflow post: How can I do multiple substitutions using regex in python?

import re
repls = {
    'california': 'California',
    'cal': 'California',
    'ca': 'California',
    'florida':'Florida',
    'fl':'Florida'
}
def multiple_replace(text):
    regex = re.compile("(%s\\b)" % "\\b|".join(map(re.escape, repls.keys())),re.I)
    return regex.sub(lambda mo: repls[mo.string[mo.start():mo.end()].lower()], text)

The above code goes in the code block, and the field box would be multiple_replace(!State!).

A couple of notes.

You will need some kind of mapping between what you want to replace and what you are replacing it with. In this example, a dictionary is being used but other Python data structures could be used as well.
The posted code works on upper or lower case, but the dictionary keys need to be lowercase or you will get a Key Error potentially.

Re: Multiple replacements with Field Calculations using Python

JoshuaBixby — Wed, 26 Aug 2015 22:18:36 GMT

Regular expressions are extremely worthwhile, especially since they are a language in and of themselves so your knowledge can be transferred beyond Python.