Need help with my python script

DMac · ‎08-28-2024

Hey everyone! I am a python novice and am having trouble with a script I'm trying to develop which will then be put into a toolbox. I'm creating the script with the Python IDLE version 3.9.18 and running it in ArcPro 3.2.

The basics of the tool are it should take in the user parameters and spit out a .csv file with the counts of features for each feature class in each selected feature dataset.

The tool has 3 user parameters:

1. Geodatabase - Geodatabase where data is located

2. Feature Datasets - Selected feature datasets from the above geodatabase

3. Output location - Location for output .csv file

The output of the tool should be a .csv file that has 3 columns: DataSet (name of dataset), FeatureClass(name of feature class), and NumFeatures(number of features in the feature class)

When I try to run the below script it gives me an error " Traceback (most recent call last): File "D:\Users\dmckay\Documents\FeatureCountAutomation_OG_Test\CODE\FeatureCount9UtilFDs_R1.py", line 29, in <module>
for FCs in thisFD:
TypeError: 'NoneType' object is not iterable"

Here is the script. Any help is appreciated!

# Custom script to extract feature counts for each featureclass in a hard coded list of feature data sets
# Output = CSV

import arcpy

# Get production workspace path from first user prompt
gdbpath = arcpy.GetParameterAsText (0)
arcpy.env.workspace = gdbpath.replace("\\", "/")
arcpy.env.overwriteOutput = True

# List all feature datasets in the geodatabase
feature_datasets = arcpy.ListDatasets(feature_type='Feature')

# Get the user-selected feature datasets (multi-value parameter) - Second user prompt
UtilClass = arcpy.GetParameterAsText(1).split(";")

# Get output file path and name from second user prompt
outfile_path = arcpy.GetParameterAsText(2)

# Open the output file in write mode - Python
with open(outfile_path, "w") as outfile:
# Write column headings
outfile.write("DataSet,FeatureClass,NumFeatures\n")

for FDs in UtilClass:
# Update workspace path for each feature dataset
arcpy.env.workspace = gdbpath + "/" + FDs
thisFD = arcpy.ListFeatureClasses()
for FCs in thisFD:
if "." in FCs:
FCs = FCs.split(".")
FCs = FCs[len(FCs)-1]
arcpy.AddMessage("Working on {0}: {1}".format(FDs, FCs))
# Write Feature Dataset Name
outfile.write(format(FDs))
# Write Field Delimiter
outfile.write(",")
# Write Featureclass Name
outfile.write(format(FCs))
# Write Field Delimiter
outfile.write(",")
# Write Feature Count
outfile.write(format(arcpy.GetCount_management(FCs)))
# Write New Line
outfile.write("\n")
#Close the output file
arcpy.SetParameter(1, outfile)

DavidSolari · ‎08-29-2024

I figured I'd use this as way to not only fix your script tool, but demonstrate some common newbie Python mistakes and a way to fix them. This isn't a condemnation of your skills or anything, I too was a new programmer once! Anyways:

import arcpy
from os import path
from csv import writer

from typing import Iterable, Iterator


def get_rows(dataset: str) -> Iterator[tuple[str, str, str, str]]:
    """
    Yield the workspace name, dataset name, feature class name and feature count for
    every feature class in a dataset.
    """
    workspace, dataset_name = path.split(dataset)
    # EnvManager doesn't work, save and restore workspace like this
    old_workspace = arcpy.env.workspace
    arcpy.env.workspace = workspace
    for feature_class in arcpy.ListFeatureClasses(feature_dataset=dataset_name):
        name = feature_class.split(".")[-1]
        count = arcpy.management.GetCount(feature_class)[0]
        yield workspace, dataset_name, name, count
    arcpy.env.workspace = old_workspace   


def main(datasets: Iterable[str], output: str):
    """
    Write a table of feature counts from the selected datasets.
    """
    with open(output, "w", newline="") as f:
        csv_writer = writer(f)
        csv_writer.writerow(("Geodatabase", "DataSet", "FeatureClass", "NumFeatures"))
        for dataset in datasets:
            for row in get_rows(dataset):
                csv_writer.writerow(row)


def execute(params: list[arcpy.Parameter]):
    datasets = params[0].valueAsText.split(";")
    output = params[1].valueAsText
    main(datasets, output)


if __name__ == "__main__":
    execute(arcpy.GetParameterInfo())

Now for the notes:

I've split this script into roughly 3 parts: main, execute and the "main guard". The main function is where the real work starts, but all parameters are expressed as standard Python data types. The execute function translates a list of Parameter objects into standard Python objects and calls main; this would also set derived outputs using values returned from main if that were useful. This separates "doing the work" from "dealing with parameter objects" which makes it easier to reason about your code. Finally, the "main guard" will check if this script is being executed directly and will pass all parameters as a list of Parameter objects into execute, kickstarting the process. This means I can independently test the main function by, say, replacing that last line with a direct call to main, which lets me use a Python IDE like Visual Studio Code and its debugger without any hassle. The lesson in short: try to apply a separation of concerns to your code and it'll be easier to write, test, rewrite etc.
Use type annotations whenever you can! Pro 3.3 has pretty good type hints for arcpy now so applying hints to your own code will let you run type analysis through an IDE. This isn't a big issue for a small script like this, but if your code gets into the thousands of lines range they can save a ton of headaches. In this example I used some imports from the "typing" module to make my types more generic, sticking to the standard types like "list[str]" is perfectly fine if you're just getting into this.
Speaking of imports, Python has a massive standard library and Pro comes with dozens of third-party packages to leverage. Here I'm using two standard modules: "os.path" to handle path string manipulation and "csv.writer" to write CSV data without worrying about text formatting, escape codes etc. etc. Thanks to this the multiple file write calls in your original example have been condensed to one foolproof call per feature class.
Your original tool had separate parameters for the workspace and the feature datasets. I tried this myself and discovered that it doesn't play well with the parameter's data types. The script above just has one parameter for a list of feature datasets. All the user has to do is use the file picker that comes with the parameter definition and the tool gets a list of full dataset paths which we can easily extract the workspace and dataset name from using "os.path.split". The lesson: try to work with the various parameter types as best you can, you'll discover with practice which ones are helpful and which ones are not.
The stuff I'm doing with the environment workspace is very hacky, unfortunately my copy of Pro has a bug with the new EnvManager context manager object so I have to write it this way to temporarily set the workspace. That said, in general you should not overwrite environment variables in your script unless it's 100% necessary, and if you do try to restore the old value. These variables are set by the user and it's expected that all tools will honor their values.
My version of the script has exactly three comments: the docstring for the main function, the docstring for get_rows, and an aside for that EnvManager workaround. As counter-intuitive as it may seem, your code should strive to have as few comments as possible. This is an extremely common new programmer mistake that seems to be drilled into people's heads, mostly by teachers and administrators who either write very little code or are applying obsolete doctrine. Comments do not make your code do the correct thing: your code does that. Excess comments can obscure mistakes very easily and make your code difficult to read. If you've written a block of code and feel it needs to be documented, extract that code into a function and write a docstring like I've done here. Save your "#" comments for hacky workarounds, complex chains of logic and other nuances that can't be expressed in the function's docstring.

Whoops, wrote a whole essay! Hope this provided some guidance for your future work, if anyone takes umbrage with these tips please reply, I'm sure I'm making some mistakes of my own.

View solution in original post

CodyPatterson · ‎08-28-2024

Hey @DMac

It appears that one of your iterables in thisFD are None, so we would have to get around that, optimally reporting the error as well, here is something to try:

if thisFD = None:
     print('thisFD is None, skipping')
else:
     for FCs in thisFD:
          {rest of your code}

I can't really tell where your indentation should start and end, the formatting seems to have not came through, so everything that should be within for FCs in thisFD should be contained in {rest of your code}.

AllenDailey1 · ‎08-28-2024

Hi there,

It would help to see your code in the code-style font/format. You can do that by clicking the button to expand the toolbar (when you are writing a reply here), then click the </> icon to insert code.

I see that you are using the variable "thisFD" for arcpy.ListFeatureClasses(). For me, this variable name could cause a little confusion and obscure a possible reason for your error. I would expect "thisFD" to represent the feature dataset name or object, so I would not expect it to end up as None, because how would you have a feature dataset name for a nonexistent feature dataset? But since the variable is representing the list of feature classes within the dataset, you could have a feature dataset with nothing in it. So the variable could represent None (no feature classes). That's where I find it could cause confusion. Personally, I find it helpful to use variable names that very literally indicate what they represent. I think if the variable name was something like "fc_list", it would very clearly indicate what it represents, and it would indicate that it could be empty/None, because a list can be empty. I agree with what the previous person said, to add handling to deal with an empty list.

DMac · ‎08-28-2024

# Custom script to extract feature counts for each featureclass in a hard coded list of feature data sets
# Output = CSV

import arcpy

# Get production workspace path from first user prompt
gdbpath = arcpy.GetParameterAsText (0)
arcpy.env.workspace = gdbpath.replace("\\", "/")
arcpy.env.overwriteOutput = True

# List all feature datasets in the geodatabase
feature_datasets = arcpy.ListDatasets(feature_type='Feature')

# Get the user-selected feature datasets (multi-value parameter) - Second user prompt
UtilClass = arcpy.GetParameterAsText(1).split(";")

# Get output file path and name from second user prompt
outfile_path = arcpy.GetParameterAsText(2)

# Open the output file in write mode - Python
with open(outfile_path, "w") as outfile:
    # Write column headings
    outfile.write("DataSet,FeatureClass,NumFeatures\n")
    
    for FDs in UtilClass:
        # Update workspace path for each feature dataset
        arcpy.env.workspace = gdbpath + "/" + FDs
        thisFD = arcpy.ListFeatureClasses()
        for FCs in thisFD:
            if "." in FCs:
                FCs = FCs.split(".")
                FCs = FCs[len(FCs)-1]
            arcpy.AddMessage("Working on {0}: {1}".format(FDs, FCs))
            # Write Feature Dataset Name
            outfile.write(format(FDs))
            # Write Field Delimiter
            outfile.write(",")
            # Write Featureclass Name
            outfile.write(format(FCs))
            # Write Field Delimiter
            outfile.write(",")
            # Write Feature Count
            outfile.write(format(arcpy.GetCount_management(FCs)))
            # Write New Line
            outfile.write("\n")
    #Close the output file
    arcpy.SetParameter(1, outfile)

DMac · ‎08-28-2024

Above is the code re-displayed in the correct format. Sorry for the newbie mistake!

CodyPatterson · ‎08-28-2024

Hey @DMac

This is how I would reformat it to check for these errors:

# Custom script to extract feature counts for each featureclass in a hard coded list of feature data sets
# Output = CSV

import arcpy

# Get production workspace path from first user prompt
gdbpath = arcpy.GetParameterAsText (0)
arcpy.env.workspace = gdbpath.replace("\\", "/")
arcpy.env.overwriteOutput = True

# List all feature datasets in the geodatabase
feature_datasets = arcpy.ListDatasets(feature_type='Feature')

# Get the user-selected feature datasets (multi-value parameter) - Second user prompt
UtilClass = arcpy.GetParameterAsText(1).split(";")

# Get output file path and name from second user prompt
outfile_path = arcpy.GetParameterAsText(2)

# Open the output file in write mode - Python
with open(outfile_path, "w") as outfile:
    # Write column headings
    outfile.write("DataSet,FeatureClass,NumFeatures\n")
    
    for FDs in UtilClass:
        # Update workspace path for each feature dataset
        arcpy.env.workspace = gdbpath + "/" + FDs
        arcpy.AddMessage("Current FDs value: {}").format(FDs)
        thisFD = arcpy.ListFeatureClasses()
        arcpy.AddMessage("Current thisFD value: {}").format(thisFD)
        if thisFD == None:
            arcpy.AddMessage("Error encountered, thisFD is None")
        else:
            for FCs in thisFD:
                if "." in FCs:
                    FCs = FCs.split(".")
                    FCs = FCs[len(FCs)-1]
                arcpy.AddMessage("Working on {0}: {1}".format(FDs, FCs))
                # Write Feature Dataset Name
                outfile.write(format(FDs))
                # Write Field Delimiter
                outfile.write(",")
                # Write Featureclass Name
                outfile.write(format(FCs))
                # Write Field Delimiter
                outfile.write(",")
                # Write Feature Count
                outfile.write(format(arcpy.GetCount_management(FCs)))
                # Write New Line
                outfile.write("\n")
    #Close the output file
    arcpy.SetParameter(1, outfile)

In this, I've added a few print statements to record the current FDs and thisFD values, this should clear up when the program is causing the error, such as the supporting error message, but that will now be avoided.

Cody

DMac · ‎08-28-2024

Hey @CodyPatterson thanks for the reply! When I run it with your changes I get this error "Traceback (most recent call last): File "D:\Users\dmckay\Documents\FeatureCountAutomation_OG_Test\CODE\FeatureCount9UtilFDs_R1.py", line 28, in <module>
arcpy.AddMessage("Current FDs value: {}").format(FDs)
AttributeError: 'NoneType' object has no attribute 'format'"

This NoneType error is confusing me because I know there is data inside the datasets that I am choosing but my code must be wrong somewhere.

CodyPatterson · ‎08-28-2024

Hey @DMac

That is my bad, instead, use f strings:

# Custom script to extract feature counts for each featureclass in a hard coded list of feature data sets
# Output = CSV

import arcpy

# Get production workspace path from first user prompt
gdbpath = arcpy.GetParameterAsText (0)
arcpy.env.workspace = gdbpath.replace("\\", "/")
arcpy.env.overwriteOutput = True

# List all feature datasets in the geodatabase
feature_datasets = arcpy.ListDatasets(feature_type='Feature')

# Get the user-selected feature datasets (multi-value parameter) - Second user prompt
UtilClass = arcpy.GetParameterAsText(1).split(";")

# Get output file path and name from second user prompt
outfile_path = arcpy.GetParameterAsText(2)

# Open the output file in write mode - Python
index = 0
with open(outfile_path, "w") as outfile:
    # Write column headings
    outfile.write("DataSet,FeatureClass,NumFeatures\n")
    
    for FDs in UtilClass:
        # Update workspace path for each feature dataset
        arcpy.env.workspace = gdbpath + "/" + FDs
        arcpy.AddMessage(f"Current FDs value: {FDs} index: {index}")
        thisFD = arcpy.ListFeatureClasses()
        arcpy.AddMessage(f"Current thisFD value: {thisFD} index: {index}")
        if thisFD == None:
            arcpy.AddMessage(f"Error encountered, thisFD is None index: {index}")
        else:
            for FCs in thisFD:
                if "." in FCs:
                    FCs = FCs.split(".")
                    FCs = FCs[len(FCs)-1]
                arcpy.AddMessage("Working on {0}: {1}".format(FDs, FCs))
                # Write Feature Dataset Name
                outfile.write(format(FDs))
                # Write Field Delimiter
                outfile.write(",")
                # Write Featureclass Name
                outfile.write(format(FCs))
                # Write Field Delimiter
                outfile.write(",")
                # Write Feature Count
                outfile.write(format(arcpy.GetCount_management(FCs)))
                # Write New Line
                outfile.write("\n")
        index += 1
    #Close the output file
    arcpy.SetParameter(1, outfile)

The reason there is Nonetype may be due to an access issue in the layer, maybe there are two or more maps to pull from, give this a shot and lets see where the issue would be at.

Cody

DMac · ‎08-28-2024

The code ran. Seems like it isn't able to see any of the data. Here is the message:

"Current FDs value: D:\Users\dmckay\Documents\FeatureCountAutomation_New\GeodatabaseForCodeTesting.gdb\Telecommunication index: 0
Current thisFD value: None index: 0
Error encountered, thisFD is None index: 0
Current FDs value: D:\Users\dmckay\Documents\FeatureCountAutomation_New\GeodatabaseForCodeTesting.gdb\Utilities_Electrical index: 1
Current thisFD value: None index: 1
Error encountered, thisFD is None index: 1"

AlfredBaldenweck · ‎08-28-2024

I'm confused as to why it would be returning a None at all? It should be returning an empty list.

See: I test on an empty feature dataset.

Actually.

The reason you're having a problem is this:

gdbpath + "/" + FDs

It's not joining the GDB path and the feature dataset name, it's joining the gdb path and feature dataset path.

C:\Users\local_abaldenweck\Temp\ArcGISProTemp9892\Untitled\Default.gdb/C:\Users\local_abaldenweck\Temp\ArcGISProTemp9892\Untitled\Default.gdb\testFD

Your UtilsClass is returning the full path of the Dataset. I'm not 100% sure what you're doing here, but you need to either use the FDs variable as the workspace environment. (Or maybe replace part of the path with the working gdb in the first parameter? idk)

Also, it will be less headache for you to use the os module for path manipulations in the future.

fdPath = os.path.join(gdb, fdname)

is cleaner than

fdPath = gdb + "\\" + fdname

Also if you want to do a replace for the path in this case, I'd use

# Get the name of the file or folder or featuredataset
fdname = os.path.basename(full fd path) 
fdPath = os.path.join(gdb,fdname)

I hope this helps!