Need help with my python script

[DMac]

Hey everyone! I am a python novice and am having trouble with a script I'm trying to develop which will then be put into a toolbox. I'm creating the script with the Python IDLE version 3.9.18 and running it in ArcPro 3.2. 

The basics of the tool are it should take in the user parameters and spit out a .csv file with the counts of features for each feature class in each selected feature dataset.

The tool has 3 user parameters:

1. Geodatabase - Geodatabase where data is located

2. Feature Datasets - Selected feature datasets from the above geodatabase

3. Output location - Location for output .csv file

The output of the tool should be a .csv file that has 3 columns: DataSet (name of dataset), FeatureClass(name of feature class), and NumFeatures(number of features in the feature class)

When I try to run the below script it gives me an error " Traceback (most recent call last): File "D:\Users\dmckay\Documents\FeatureCountAutomation_OG_Test\CODE\FeatureCount9UtilFDs_R1.py", line 29, in <module>
for FCs in thisFD:
TypeError: 'NoneType' object is not iterable"

Here is the script. Any help is appreciated!

# Custom script to extract feature counts for each featureclass in a hard coded list of feature data sets
# Output = CSV

import arcpy

# Get production workspace path from first user prompt
gdbpath = arcpy.GetParameterAsText (0)
arcpy.env.workspace = gdbpath.replace("\\", "/")
arcpy.env.overwriteOutput = True

# List all feature datasets in the geodatabase
feature_datasets = arcpy.ListDatasets(feature_type='Feature')

# Get the user-selected feature datasets (multi-value parameter) - Second user prompt
UtilClass = arcpy.GetParameterAsText(1).split(";")

# Get output file path and name from second user prompt
outfile_path = arcpy.GetParameterAsText(2)

# Open the output file in write mode - Python
with open(outfile_path, "w") as outfile:
# Write column headings
outfile.write("DataSet,FeatureClass,NumFeatures\n")

for FDs in UtilClass:
# Update workspace path for each feature dataset
arcpy.env.workspace = gdbpath + "/" + FDs
thisFD = arcpy.ListFeatureClasses()
for FCs in thisFD:
if "." in FCs:
FCs = FCs.split(".")
FCs = FCs[len(FCs)-1]
arcpy.AddMessage("Working on {0}: {1}".format(FDs, FCs))
# Write Feature Dataset Name
outfile.write(format(FDs))
# Write Field Delimiter
outfile.write(",")
# Write Featureclass Name
outfile.write(format(FCs))
# Write Field Delimiter
outfile.write(",")
# Write Feature Count
outfile.write(format(arcpy.GetCount_management(FCs)))
# Write New Line
outfile.write("\n")
#Close the output file
arcpy.SetParameter(1, outfile)

[DMac]

Hey @AlfredBaldenweck thanks for the reply. After doing some testing I think what's confusing me is the arcpy.ListDatasets() function. When I run a test on this just in the arcpy window I get the same NoneType error. 

But when I set the workspace to one of the feature datasets and run the arcpy.ListFeatureClasses I get back a list of all the feature classes. 

Is there something about the ListDatasets function that I don't get? Shouldn't it list the datasets in the geodatabase workspace? 

[AlfredBaldenweck]

It does.

(BTW I don't think you actually do anything with you feature_datasets variable, so idk why it's there)

The thing with the list functions (ListDatasets, ListFeatureClasses) is that it relies on your workspace. That is, you can't feed it a geodatabase and have it figure it out; you have to set the workspace environment first.

For reasons I do not understand or agree with, a feature dataset is considered a workspace. So, in this example, I have a file gdb with a feature dataset containing one feature class and nothing else.

If you're feeding it a path that exists, both ListDatasets() and ListFeatureClasses() should return a list, even if it's empty (see part one of the second picture). Like I said, I think your issue is that you're feeding it a bad path, and when you manually do it Pro you're explicitly feeding it a good path.

[DMac]

Yes I understand that. I think the problem lies in the location of my workspace. I'm trying to run this on a versioned SDE and I'm wondering if the path I'm using isn't the "actual" path but some kind of shortcut for lack of a better word.

Still seems weird that I can list the feature classes using the SDE datasets as the workspace but I can't list the datasets using the geodatabase as the workspace. 

I was able to run the List functions on the datasets using a regular file geodatabase so the issue must be with my sde connection. Thanks for the help!

[AlfredBaldenweck]

hmmmm, weird.

I did the same thing on an SDE file (connected to default; I'm having issues editing my SDE files to a specific version today) and didn't get the error:

(This eGDB doesn't have any feature classes outside of a feature dataset)

[AlfredBaldenweck]

Update yeah I connected to a specific version with that SDE file and it worked fine.

[AlfredBaldenweck]

Also, reviewing the documentation:ListFeatureClasses—ArcGIS Pro | Documentation

If you know the feature dataset you want, it appears you can feed that to ListFeatureClasses() without having to change the environment?

[DMac]

This looks interesting. Thanks @AlfredBaldenweck !

[DavidSolari · Accepted solution]

I figured I'd use this as way to not only fix your script tool, but demonstrate some common newbie Python mistakes and a way to fix them. This isn't a condemnation of your skills or anything, I too was a new programmer once! Anyways:

import arcpy
from os import path
from csv import writer

from typing import Iterable, Iterator


def get_rows(dataset: str) -> Iterator[tuple[str, str, str, str]]:
    """
    Yield the workspace name, dataset name, feature class name and feature count for
    every feature class in a dataset.
    """
    workspace, dataset_name = path.split(dataset)
    # EnvManager doesn't work, save and restore workspace like this
    old_workspace = arcpy.env.workspace
    arcpy.env.workspace = workspace
    for feature_class in arcpy.ListFeatureClasses(feature_dataset=dataset_name):
        name = feature_class.split(".")[-1]
        count = arcpy.management.GetCount(feature_class)[0]
        yield workspace, dataset_name, name, count
    arcpy.env.workspace = old_workspace   


def main(datasets: Iterable[str], output: str):
    """
    Write a table of feature counts from the selected datasets.
    """
    with open(output, "w", newline="") as f:
        csv_writer = writer(f)
        csv_writer.writerow(("Geodatabase", "DataSet", "FeatureClass", "NumFeatures"))
        for dataset in datasets:
            for row in get_rows(dataset):
                csv_writer.writerow(row)


def execute(params: list[arcpy.Parameter]):
    datasets = params[0].valueAsText.split(";")
    output = params[1].valueAsText
    main(datasets, output)


if __name__ == "__main__":
    execute(arcpy.GetParameterInfo())

Now for the notes:

• I've split this script into roughly 3 parts: main, execute and the "main guard". The main function is where the real work starts, but all parameters are expressed as standard Python data types. The execute function translates a list of Parameter objects into standard Python objects and calls main; this would also set derived outputs using values returned from main if that were useful. This separates "doing the work" from "dealing with parameter objects" which makes it easier to reason about your code. Finally, the "main guard" will check if this script is being executed directly and will pass all parameters as a list of Parameter objects into execute, kickstarting the process. This means I can independently test the main function by, say, replacing that last line with a direct call to main, which lets me use a Python IDE like Visual Studio Code and its debugger without any hassle. The lesson in short: try to apply a separation of concerns to your code and it'll be easier to write, test, rewrite etc.
• Use type annotations whenever you can! Pro 3.3 has pretty good type hints for arcpy now so applying hints to your own code will let you run type analysis through an IDE. This isn't a big issue for a small script like this, but if your code gets into the thousands of lines range they can save a ton of headaches. In this example I used some imports from the "typing" module to make my types more generic, sticking to the standard types like "list[str]" is perfectly fine if you're just getting into this.
• Speaking of imports, Python has a massive standard library and Pro comes with dozens of third-party packages to leverage. Here I'm using two standard modules: "os.path" to handle path string manipulation and "csv.writer" to write CSV data without worrying about text formatting, escape codes etc. etc. Thanks to this the multiple file write calls in your original example have been condensed to one foolproof call per feature class.
• Your original tool had separate parameters for the workspace and the feature datasets. I tried this myself and discovered that it doesn't play well with the parameter's data types. The script above just has one parameter for a list of feature datasets. All the user has to do is use the file picker that comes with the parameter definition and the tool gets a list of full dataset paths which we can easily extract the workspace and dataset name from using "os.path.split". The lesson: try to work with the various parameter types as best you can, you'll discover with practice which ones are helpful and which ones are not.
• The stuff I'm doing with the environment workspace is very hacky, unfortunately my copy of Pro has a bug with the new EnvManager context manager object so I have to write it this way to temporarily set the workspace. That said, in general you should not overwrite environment variables in your script unless it's 100% necessary, and if you do try to restore the old value. These variables are set by the user and it's expected that all tools will honor their values.
• My version of the script has exactly three comments: the docstring for the main function, the docstring for get_rows, and an aside for that EnvManager workaround. As counter-intuitive as it may seem, your code should strive to have as few comments as possible. This is an extremely common new programmer mistake that seems to be drilled into people's heads, mostly by teachers and administrators who either write very little code or are applying obsolete doctrine. Comments do not make your code do the correct thing: your code does that. Excess comments can obscure mistakes very easily and make your code difficult to read. If you've written a block of code and feel it needs to be documented, extract that code into a function and write a docstring like I've done here. Save your "#" comments for hacky workarounds, complex chains of logic and other nuances that can't be expressed in the function's docstring.

Whoops, wrote a whole essay! Hope this provided some guidance for your future work, if anyone takes umbrage with these tips please reply, I'm sure I'm making some mistakes of my own.

[DMac]

Thanks @DavidSolari !! This really helped and made it easier for me to understand. I appreciate your explanations, they make sense and now I can use them for future projects. Thanks again!