topic Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results? in Python Questions

How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

VishalShah2 — Sat, 11 Dec 2021 15:08:44 GMT

Hello,

I am working with a for loop where a summary statistics is created. I am able to store the output to a custom file gdb using TableToTable conversion. However, I'm not able to append data to it on the following iterations. How can I go about this?

if hubDict[0]:
    arcpy.TableToTable_conversion(in_rows, out_path, out_name)
else:
    arcpy.Append_management(inputs, target, "NO_TEST")
    ‍‍‍‍‍

hubDict is a dictionary i have created earlier on. in_rows, out_path, out_name, inputs, target are all variables that are defined to the correct files and output locations.

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

JoshuaBixby — Thu, 10 May 2018 18:31:13 GMT

Are you getting an error or unexpected results? If the former, what error, exactly (include the traceback). If the latter, what do you expect and what are you getting?

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

VishalShah2 — Sat, 11 Dec 2021 15:08:47 GMT

Hi Joshua,

Sorry about the delay in getting back to you. So my script first creates a dictionary per feature in a feature class. Then the script runs a for loop per feature in the feature class using the values in the dictionary. Rather than exporting several tables and their excel sheets, I am trying to create a table with the first iteration using table to table (conversion) and then for the rest of the iterations, I want their tables to append to the created table from the first iteration. The way I have it set up is the following:

if hubDict[0]:
    in_rows = 'table'
    out_path = folderPath + '/' + 'Results.gdb'
    out_name = 'Results'

    arcpy.TableToTable_conversion(in_rows, out_path, out_name)

else:
    inputs = 'table'
    target = folderPath + '/' + 'Results.gdb' + '/' + 'Results'

    arcpy.Append_management(inputs, target, "NO_TEST")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The Results table is being created on the first iteration but on the second one, the data isn't getting appended.

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

JoshuaBixby — Tue, 15 May 2018 17:52:01 GMT

Are you sure Append is being executing? If so, capture the Result—Help | ArcGIS Desktop and see what messages it is giving.

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

VishalShah2 — Tue, 15 May 2018 18:01:01 GMT

It's not being executed because of the else statement. I don't think I set that up correctly nor do I know how to fix that. For some reason that is not registering on the second iteration and would love it if you have any ideas on how to approach this scenario..

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

VishalShah2 — Tue, 15 May 2018 18:56:15 GMT

When I run the steps outside of the main for loop, the append is being executed correctly. My issue is not sure how to script it for the script to read it as the first iteration or Xth iteration. I want the first iteration to create the table and the following iterations to be appended to it.

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

JoshuaBixby — Tue, 15 May 2018 19:13:23 GMT

How are you iterating over hubDict? If hubDict is a list, which it appears to be, you have to loop over it in some way, shape, or form. Knowing a bit more about the structure of hubDict would be helpful too.

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

VishalShah2 — Sat, 11 Dec 2021 15:08:49 GMT

So hubDict is actually a dictionary. To loop over over it, I'm doing the following:

for hub in hubDict:
    valueHub = dict.get(hubDict, hub)
    hubQuery = "HubName = " + "'" + str(valueHub) + "'"
    nodeQuery = "ProjectHub = " + "'" + str(valueHub) + "'"
    hubLayer.definitionQuery = hubQuery
    nodeLayer.definitionQuery = nodeQuery‍‍‍‍‍‍

hubLayer and nodeLayer are variables that point to the hub and node layers in my data frame. After this, in the loop I do a network analysis and use that to do some joins and run a summary statistics. I'm trying to use the output table from the summary statistics to get into the results FGDB.

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

VishalShah2 — Tue, 15 May 2018 20:18:58 GMT

Apparently the append worked this time around but the original data was overwritten with the data that was appended. I think the issue lies with how the if statement is set up in the for loop. How would you set it up?

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

JoshuaBixby — Tue, 15 May 2018 21:33:53 GMT

Prior to Python 3.6 (one could argue it is really prior to 3.7, but that is a separate discussion), insertion order was not preserved with Python dictionaries. If the "first" record in the dictionary means something special, you should either be using an OrderedDict or some other data structure that does maintain insertion order.

If insertion order doesn't matter and the first item is simply meant to create the table while the other items are to get appended, then it simplifies the situation a bit. Does your first code snippet come immediately after your second one or is there other code involved?

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

VishalShah2 — Wed, 16 May 2018 13:42:05 GMT

Insertion order doesn't matter as long as the first item creates the table while the remaining items append to it. Not entirely sure what you mean by code snippets and what comes after but I do some analysis work after I start the iteration through the dictionary using network analysis and the last step in the iteration is what this thread is for, either creating the table or appending to it and then repeating the steps. Hope that helps.

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

JoshuaBixby — Wed, 16 May 2018 14:22:56 GMT

You have provided two code snippets, or probably better stated code fragments, during our discussion. Between the two different blocks of code, there is only a single variable that is in common, hubDict. Since I have no idea how the two blocks of code relate to each other, I can't provide specific suggestions on how to loop over hubDict and accomplish what you want.

Overall, I suggest you use dict.iteritems() to return an iterator of the items in hubDict. You can use next() to retrieve the first item and build the table, and then loop over the rest of the iterator with for to append the rest to the tables.

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

VishalShah2 — Sat, 11 Dec 2021 15:08:52 GMT

Ah...I get where you were coming from. For the code snippets, here is the overall picture:

for hub in hubDict:
    valueHub = dict.get(hubDict, hub)
    hubQuery = "HubName = " + "'" + str(valueHub) + "'"
    nodeQuery = "ProjectHub = " + "'" + str(valueHub) + "'"
    hubLayer.definitionQuery = hubQuery
    nodeLayer.definitionQuery = nodeQuery
    
    if hubDict[0]:
        in_rows = 'table'
        out_path = folderPath + '/' + 'Results.gdb'
        out_name = 'Results'

        arcpy.TableToTable_conversion(in_rows, out_path, out_name)

    else:
        inputs = 'table'
        target = folderPath + '/' + 'Results.gdb' + '/' + 'Results'

        arcpy.Append_management(inputs, target, "NO_TEST")
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

I've never worked with dict.iteritems() or next(). How would you put that together with the code snippet I have?

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

JoshuaBixby — Sat, 11 Dec 2021 15:08:55 GMT

After looking at your code, I changed my approach to take the path of least modifying your code:

for i, hub in enumerate(hubDict):
    valueHub = dict.get(hubDict, hub)
    hubQuery = "HubName = " + "'" + str(valueHub) + "'"
    nodeQuery = "ProjectHub = " + "'" + str(valueHub) + "'"
    hubLayer.definitionQuery = hubQuery
    nodeLayer.definitionQuery = nodeQuery
    
    if i == 0:
        in_rows = 'table'
        out_path = folderPath + '/' + 'Results.gdb'
        out_name = 'Results'

        arcpy.TableToTable_conversion(in_rows, out_path, out_name)

    else:
        inputs = 'table'
        target = folderPath + '/' + 'Results.gdb' + '/' + 'Results'

        arcpy.Append_management(inputs, target, "NO_TEST")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: How to store first iteration of loop in a defined location and the rest of the iterations to append to first iteration results?

VishalShah2 — Wed, 16 May 2018 16:38:19 GMT

Thank you so much Joshua!!