Python code block in calculate field problem

Question

Ok I am having a problem with a python code block. It works for about 1300 rows and then fails without error. The calculate field tool runs for 15-20 minutes and then without returning any error returns to the shell. The problem is stopping the whole script without giving any sort of error. Also it seems to be skipping value.replace("'",'') though I think that the need to escape those two " are confusing the GP. Perhaps there is a better way to convert a string to a set  
expression='getvalue(!parce_test_spatial_join.FLD_ZONE!)'
codeblock="def getvalue(string):
\
    valuelist=string.split(',')
\
    valueset=set(valuelist)
\
    value=str(valueset)[5:-2]
\
    value.replace("'",'')
\
    return value"

I am thinking of replacing the whole code block with this but I still have to escape those quotes or python gets mad.
 
codeblock="def(string):
\
    value=str(set(string.split(','))).replace("'",'')[5:-2]
\
    return value"

ChrisMathers · Answer

Thanks to both of you. After much deliberation last night I decided "ef it Ill just forget the set()." The whole point of this was that the output of a spatial join with the important field set to Join as its merge rule gave for some features things like 'x,x,x,a,ae,ve,ve,ve,a,ae,x,x.' I wanted to make that a set so that only the unique values copied over to the other table. At this point though Ill just copy the whole string and be done with it. It was more of an aesthetic change anyways. Not being able to use an update cursor on join tables is very irritating though.

KimOllivier · Answer

You could look at the MakeQueryTable tool. This will effectively join a number of tables very efficiently. I have tried it with extremely large tables that would fail with a Join.
Haven't tried a cursor on the resulting layer....

RichardFairhurst · Answer

Python absolutely sucks with joined data.  Python is up to 5000% slower with a join in place than VB script, so you should absolutely avoid doing calculations using Python with a join in place whenever possible.  Joins and the field calculator should only be used in scripts where a data transfer is occurring from the child table to the parent table (and if it is a simple transfer use VB Script syntax).  I don't see that your calculation in this case is dependent on the join (but I may be missing something).  Joins with Python also can cause out ot memory errors that cause unexpected failures as you can see.

If this is scripted and the calculation you want to do is possible outside of the join reorganize your operations to do the calculation outside the join and then create the join.  Everything will perform much much faster and failure will be minimized.  If the join is necessary for transferring data, do just that operation and then break the join to do any other operations that transform the data.  You will save an immense amount of time.  (Python is necessary for advanced expressions, but never perform them with a join in place).  Breaking the transfer and the transformation into separate operations will actually save you time if you manage teh joins so that they are removed prior to and advanced calculations.

ESRI has confirmed the slow performance of Python on joined tables and has no fix at either ArcGIS 9.3.1 or ArcGIS 10.  VB Script syntax always performs faster for simple transfers between joined tables, so for any simple transfers use VB syntax (simple means only without any VB specific functions or methods, just [field] tranfer syntax).  The VB syntax on joined tables always takes advantage of indexed join fields, but the Python syntax does not seem to, which is why the calculations are always faster with VB script.  (Of course this means you should create indexes on your joined fields before doing the calculations and use VB script syntax)

I also have never been able to use an updatecursor on a joined table, so I don't know of a way to make that work (seems like ESRI somehow does it, but they have never indicated how as far as I have seen).

Hope this helps.