topic Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise. in Python Questions

Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

mhouts_ku — Tue, 20 Apr 2021 20:56:21 GMT

I want to join a table (dbf from a zonal stats) to a FC and then calculate the mean from the dbf into a FC field. I can do it manually, but am hoping in Pro and ArcPy its faster than ArcMap and manual method. With over 1 million polygons joined, it takes about a week to calculate. Please review code below and advise what wrong. Just learning by pulling together examples of familiar processes, please speak noobie. THX!

Got the "add field" to work, the subset below is what fails...

# Set local variables
inFeatures = "f:/GreatPlains_NHF/Join_calc_test/105_45_532.gdb/Hex_subset105_45_532"
joinTable = "f:/GreatPlains_NHF/Join_calc_test/105_45_532_summary.dbf"
joinField = "hexagon_id"
expression = "Mean"

# Join the feature layer to a table
arcpy.AddJoin_management(inFeatures, joinField, joinTable, joinField)

# Calculate field from joined table ("Mean" attribute) to FC table.
arcpy.CalculateField_management(inFeatures, "mean_HD_ensamble", expression)

Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

DanPatterson — Tue, 20 Apr 2021 21:05:16 GMT

Not sure what you are wanting to do. A complete error message would be very useful.

And what "mean" are you trying to calculate?

across rows?

down a column?

partitioned by some other attribute?

Details and other notes on the rest of the code would be useful. Also

Code formatting ... the Community Version - Esri Community

helps with readability and syntax checking.

Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

mhouts_ku — Tue, 20 Apr 2021 21:09:57 GMT

The "Mean" I am calculating is actually the field name from the zonal stats table. Basically use calculate value to copy values from field "Mean" in the joined table to the "mean_HD_ensamble" field in the source FC that the table is joined to.

Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

BlakeTerhune — Tue, 20 Apr 2021 21:20:50 GMT

Could you describe the join? Are there multiple records in the dbf for each hexagon_id in the feature class? are you calculating the mean of some value for each group of hexagon_id in the dbf?

Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

DanPatterson — Tue, 20 Apr 2021 21:20:58 GMT

If your destination field is empty and you are going to calculate values over. A quicker way is to save the joined data to a new featureclass. The expression has to be python supported "Mean" isn't correct "!Mean!" would be if it is a field's name

Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

mhouts_ku — Tue, 20 Apr 2021 21:28:06 GMT

The destination (recieving) field for the calculate is anew field just created in a previous step before the join ("mean_HD_ensamble"). There are many other fields present, so I dont want to just save after the join and have the default min/max/mean field names from the zonal stats table...need to move those into a better named column. Thanks, will try put ! around source and destination field names.

Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

mhouts_ku — Tue, 20 Apr 2021 21:29:22 GMT

its a one to one relationship. just trying to basically copy values from one filed "mean" to another field "mean_HD_ensamble".

Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

BlakeTerhune — Tue, 20 Apr 2021 21:37:01 GMT

Try some of the techniques described by @RichardFairhurst in Turbo Charging Data Manipulation with Python Curso... - Esri Community

Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

mhouts_ku — Thu, 22 Apr 2021 20:45:03 GMT

Thanks for the suggestion. His process (example 1) of using Joins and Updates worked great. Instead of a traditional Join and Calculate Field process that took about 5 days to run on 900k records, the Join/Update process copied the desired values over in 57 seconds. THANK YOU! Wish I took the time to figure this out two months ago.

Re: Just learning ArcPy for a very familiar process, but I cant get it run, please advise.

Tomasz_Tarchalski — Mon, 03 May 2021 04:37:46 GMT

Hi, it might be a bit too late, but as a notice. 1mln features, I would not even try to join. It is much simplier to create dictionary in memory. using the attriutbuttes from 'join' feature create dictionnary - key:[values] where key is your unique id, and values can be 1 value, or list of values. Next run update cursor in your main feature, loop through the key field. if that record is in dictionnary, you can use its values (for example if rec[0] in mydico.keys()): rec[1] = mydico[rec[0]] ).

sorry no script, but there are some articules about that in internet.