topic Re: Python For Loop Error in Python Questions

Python For Loop Error

Anonymous User — Sat, 11 Dec 2021 16:37:24 GMT

*EDIT*

I realized after posting this question that the code didn't quite come through as formatted as I remember the last time I posted something. Has the formatting changed, or did I do something wrong? Thanks!

Hello,

I have a script that I am using to iterate through a list of all fields in a feature class, run summary statistics on itself to get a table of all entries, and then creates a file geodatabase copy as well as an excel copy of the table. The tool that I have is working seemingly well, but I end up getting an error at the end.

The code:

# ###################################################################################################
# Name: Database Check
# Author: Coy Potts
# Description: This tool runs a summary on all feature class fields to check for inappropriate entries.
# ###################################################################################################


# ###################################################################################################
# Import system modules
# ###################################################################################################
import arcpy
from arcpy import env
import datetime


# ###################################################################################################
# Set universal variables
# ###################################################################################################
generalFeatures = r'OSP Mapping Layers\General Features'
dateToday = datetime.date.today()
outPath = r'"local_shared_drive"
generalVar = "general"


# ###################################################################################################
# Create the workspace for the statistics documents
# ###################################################################################################

# ########################
# Create a dated folder
# ########################

# Set local variables
datedFolder = str(dateToday)

# Create a dated folder to store the tables
arcpy.CreateFolder_management(outPath, datedFolder)


# ########################
# Create a dated geodatabase
# ########################

# Set local variables
outGDB = str(dateToday) + ".gdb"

# Create a dated file geodatabase
arcpy.CreateFileGDB_management(datedFolder, outGDB, 'CURRENT')


# ###################################################################################################
# Run Statistics on the General Features feature class
# ###################################################################################################

# Set local variables
inFC = generalFeatures
fields = [f.name for f in arcpy.ListFields(inFC)]
fieldCount = len(fields)

i = 0


# ########################
# Iterate through each General Features field, exporting an associated table and excel sheet
# ########################

for f in fields:
    while i < fieldCount:
        # Set local variables
        i += 1
        fieldVar = fields[i]
        statsFields = [[fieldVar,"COUNT"]]
        caseFields = [fieldVar]
        outTable = generalVar + "_" + fieldVar

        # Execute SummaryStatistics
        arcpy.Statistics_analysis(inFC, outTable, statsFields, caseFields)

        # Set local variables
        inTable = inFC
        saveGDB = outPath + str(dateToday) + "/" + outGDB

        # Execute TableToTable
        arcpy.TableToTable_conversion(inTable, saveGDB, outTable, "", "", "")

        arcpy.AddMessage("Summary table created for the " + fieldVar + " field!")

        # Set local variables
        inTable = outTable
        outXLS = outPath + str(dateToday) + "/" + generalVar + "_" + fieldVar + ".xls"

        # Execute TableToExcel
        arcpy.TableToExcel_conversion(inTable, outXLS)

        arcpy.AddMessage("Excel sheet created for the " + fieldVar + " field!")
print("General Features summary tables have been created for all fields!")



‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

This script runs through and does exactly what it is meant to do for each field, but then I get this error:

I have 4 other feature classes that I would like to eventually add onto this script to run through the exact same process for each of the feature classes, but I can't get it to work on the second feature class with this error getting triggered after the first feature class wraps up. Any ideas why it is giving me a field type error even though the tool has technically already exported out each of the fields to tables?

*SIDE NOTE* (let's fix the problem above first)

The tool needs another minor tweak as well in the fact that in its current state, it creates a copy of the table in my default database before it creates a copy in my created database via the table to table function, which obviously poses the problem of having pre-existing tables when running subsequent times after the first. I had to try various workarounds when trying to get this to work initially. For instance, whenever I have an env.workspace declared under the initial while loop's local variables, I would later get an error stating that the table already existed when I tried the table to table function. However, if I removed the table to table function and relied on the env.workspace table created down in the table to excel function, I would get an error saying that the table couldn't be read.

Re: Python For Loop Error

DarrenWiens2 — Thu, 17 Nov 2016 23:09:15 GMT

I think you need fieldVar.name (a string) rather than fieldVar (a field object).

Re: Python For Loop Error

Anonymous User — Fri, 18 Nov 2016 06:18:10 GMT

Even though the script runs through the summary of each field and exports an associated database table and excel sheet, as intended? This process iterates through 23 separate fields before sending back this error, but the very last field in the feature class has exported just fine. I'll give your suggestion a try, just seems strange given that it works appropriately up until the end.

I'm at home and can't test right now, but I was thinking maybe it had to do with my while loop. There are 23 fields total, but that's counting all default fields as well, so the very last field is technically a shape field, thus possibly kicking back the error...? I'll have to try tomorrow when I'm back in the office.

Re: Python For Loop Error

NeilAyres — Fri, 18 Nov 2016 07:10:53 GMT

Is the input a feature class, ie has a geometry column?

Perhaps you are trying the generate statistics on a field which makes no sense.

Re: Python For Loop Error

Anonymous User — Fri, 18 Nov 2016 07:19:28 GMT

Yes, it's a feature class. That's what I referencing in my comment above. Maybe it gets through all the other fields, but then once it hits that last field then it doesn't know what to do because it's a shape field.

Re: Python For Loop Error

NeilAyres — Fri, 18 Nov 2016 07:32:21 GMT

So, why not try :

fields = [f.name for f in arcpy.ListFields(inFC) if not f.required]‍

To skip over the OID, SHAPE@ etc

Re: Python For Loop Error

Anonymous User — Fri, 18 Nov 2016 07:47:01 GMT

I will give that a try tomorrow as well. Just so I understand, is f.required literal, or would I need to define something there? I have never seen that one before is why I ask.

Thanks!

Re: Python For Loop Error

DanPatterson_Retired — Fri, 18 Nov 2016 08:13:19 GMT

It is in the help Field—Help | ArcGIS for Desktop

everything about arcpy is found here at the top of its help file tree What is ArcPy?—Help | ArcGIS for Desktop

Re: Python For Loop Error

NeilAyres — Fri, 18 Nov 2016 09:32:19 GMT

f.required is a property of f (field object) and is True or False.

Hence

if not f.required‍

So you don't need anything else.

Re: Python For Loop Error

Anonymous User — Fri, 18 Nov 2016 14:06:33 GMT

Great, thank you!

Re: Python For Loop Error

Anonymous User — Fri, 18 Nov 2016 14:06:58 GMT

Thanks, Dan! I'll bookmark that resource.

Re: Python For Loop Error

Anonymous User — Fri, 18 Nov 2016 15:26:34 GMT

I tried this with the f.required as you suggested, and now I am getting this error at the end:

Just as before, the script runs through exporting everything just fine, but it gets to the end and returns the message above. I ran a print on my fieldCount variable and it shows as 21 fields now, which is right since the previous was 23 including the OID and SHAPE fields. Print on i returns 21, and print of fieldVar returns the name of the very last field. It seems like it is all checking out to be correct in regards to variables.

Re: Python For Loop Error

Anonymous User — Fri, 18 Nov 2016 15:35:46 GMT

Darren,

Using fieldVar.name instead returns this error:

Re: Python For Loop Error

NeilAyres — Sat, 11 Dec 2021 16:37:26 GMT

Is this the same line 77 as in the script above. I don't think so somehow.

Firstly in you script, you don't need the whole while loop. For f in fields will already iterate through the fields in the fields list. So you don't need i. And the whole of the while inner block should come in one indentation.

for field in fields:

    statsFields = [[field,"COUNT"]]
    caseFields = [field]
    outTable = generalVar + "_" + field

    # Execute SummaryStatistics
    arcpy.Statistics_analysis(inFC, outTable, statsFields, caseFields)

    # Set local variables
    inTable = inFC
    saveGDB = outPath + str(dateToday) + "/" + outGDB

    # Execute TableToTable
    arcpy.TableToTable_conversion(inTable, saveGDB, outTable, "", "", "")

    arcpy.AddMessage("Summary table created for the " + field + " field!")

    # Set local variables
    inTable = outTable
    outXLS = outPath + str(dateToday) + "/" + generalVar + "_" + field + ".xls"

    # Execute TableToExcel
    arcpy.TableToExcel_conversion(inTable, outXLS)

    arcpy.AddMessage("Excel sheet created for the " + field + " field!")
print("General Features summary tables have been created for all fields!")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

As this appears to be run inside the python window in ArcMap, you don't need all the addmessage stuff either. I suppose you want to turn this into a script tool, but this is not doing anything at the moment.

Just use some more print statements. What does the list fields look like at the moment?

What is line 77 of the current script?

Re: Python For Loop Error

DanPatterson_Retired — Sat, 11 Dec 2021 16:37:29 GMT

I would take some time to see if you need the field itself or the field's name or some other property.

In the following example, the actual field object is collected and is later used to print its name and type. Sometimes, you need the actual field, sometimes not.

import arcpy

def demo():
    inFC = r"c:\!AV\!Data\Polylines\emst1.shp"
    flds = [i for i in arcpy.ListFields(inFC) if not i.required]
    for i in flds:
        print("Field: {}, type {}".format(i.name, i.type))
    return flds

if __name__ == '__main__':
    flds = demo()‍‍‍‍‍‍‍‍‍‍‍

Result

>>> 
Field: ID, type Integer
Field: BETWEEN, type String‍‍‍

Re: Python For Loop Error

Anonymous User — Fri, 18 Nov 2016 16:29:51 GMT

I realize the error in my code now. I was grabbing a count of all fields, which was 21 and then having the i variable increment as long as it was less than that number. The i variable, however, should start at 0, so 21 field options starting from 0 should end at 20. Whenever i was equal to 21, it was effectively out of the given range.

Re: Python For Loop Error

Anonymous User — Sat, 11 Dec 2021 16:37:32 GMT

I'm simply testing in the Python window within ArcMap. I will eventually add this to a custom toolbar once it's complete. I try to see the endgame results by adding those things in as I go instead of trying to go back later and figure out what I need to put in.

I mentioned in a comment above that I have actually gotten this to work as desired, but I don't doubt that I have some inefficiencies throughout.

Overcoming the out of range error I simply created a new countVar that reduced the list of the fields by 1, since the i variable starts at 0 and not 1.

I temporarily got past the default geodatabase tables by adding a delete segment to the code.

# Set local variables
inFC = generalFeatures
fields = [f.name for f in arcpy.ListFields(inFC) if not f.required]
fieldCount = len(fields)
countVar = fieldCount -1

i = 0


# ########################
# Iterate through each General Features field, exporting an associated table and excel sheet
# ########################

for f in fields:
    while i < countVar:
        # Set local variables
        i += 1
        fieldVar = fields[i]
        statsFields = [[fieldVar,"COUNT"]]
        caseFields = [fieldVar]
        outTable = generalVar + "_" + fieldVar
        env.workspace = outPath + outGDB

        # Execute SummaryStatistics
        arcpy.Statistics_analysis(inFC, outTable, statsFields, caseFields)

        # Set local variables
        inTable = defaultGDB + "/" + outTable
        saveGDB = outPath + str(dateToday) + "/" + outGDB

        # Execute TableToTable
        arcpy.TableToTable_conversion(inTable, saveGDB, outTable, "", "", "")

        arcpy.AddMessage("Summary table created for the " + fieldVar + " field!")

        # Set local variables
        inTable = defaultGDB + "/" + outTable
        outXLS = outPath + str(dateToday) + "/" + outTable + ".xls"

        # Execute TableToExcel
        arcpy.TableToExcel_conversion(inTable, outXLS)

        # Set local variables
        inTable = defaultGDB + "/" + outTable

        # Execute Delete
        arcpy.Delete_management(inTable, "")

        arcpy.AddMessage("Excel sheet created for the " + fieldVar + " field!")
print("General Features summary tables have been created for all fields!")‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Python For Loop Error

NeilAyres — Sat, 19 Nov 2016 08:56:21 GMT

As I said above, you don't need the "while i < countvar" bit.

Simply iterating through the fields would be enough. Then the pointer would not have gone out of range either.

Re: Python For Loop Error

JoshuaBixby — Sat, 19 Nov 2016 14:52:42 GMT

I agree with Neil Ayres‌. My guess is that you come to Python from another/different programming language because you are applying an unPythonic programming pattern in Python. It isn't that looping/iterating this way won't work, it will, but it will be more prone to errors in the long run because you are not using the natural patterns of the language.

If you plan on programming in Python more, I suggest learning a bit more about looping/iterating in Python. For someone like yourself who already knows programming, but from a different language, I have found Ned Batchelder's Loop like a native presentation well received. Ned does a really good job of explaining how looping is handled in different languages and how to do it in Python.

Re: Python For Loop Error

DanPatterson_Retired — Sat, 19 Nov 2016 15:00:06 GMT

I also noticed that this doesn't make sense

statsFields = [[fieldVar,"COUNT"]]‍

wouldn't just a plain list be needed not a list of a list