'NoneType' Object has no Attribute 'modules' AttributeError the Second Time Script Tool is Run

4511
14
06-15-2018 09:52 AM
PaulMcCord1
New Contributor II

I am developing a script tool that will be part of a suite of tools in an ArcToolbox. This script tool is intended to plot FAO food security indicators.

When I run this tool the first time from ArcToolbox (ie, I run the tool from ArcMap), it successfully completes without an issue (ie, the Pygal module (which I import) plots the food security indicator that the user selects for the country selected by the user). However, the second time that I run the tool, it fails with the following error:

Traceback (most recent call last):
File "C:\Users\Paul McCord\Google Drive\Toolbox\Tool_First\FoodSecurity\Content_Post052418\TestScripts\foodSecurity_v1.py", line 5, in <module>
import pygal as pg
File "C:\Python27\ArcGIS10.5\lib\site-packages\pygal\__init__.py", line 28, in <module>
import pkg_resources
File "C:\Python27\ArcGIS10.5\lib\site-packages\pkg_resources\__init__.py", line 46, in <module>
from pkg_resources.extern import six
File "C:\Python27\ArcGIS10.5\lib\site-packages\pkg_resources\extern\__init__.py", line 43, in load_module
mod = sys.modules[extant]
AttributeError: 'NoneType' object has no attribute 'modules'

Referring to a separate post that suggested removing the 'Try-Except' statement, I've tried removing this in order to resolve the issue, but it persists.

A similar GeoNet post exists (https://community.esri.com/thread/44095 ); however, in this post, the issue appears to arise from 'ImportToolbox.' I believe my issue arises from a module that I'm importing. Nevertheless, it does not appear that the issue was resolved in the earlier post.

Does anyone have experience with this issue while developing a Python script for a script tool?

Thanks,

Paul  

0 Kudos
14 Replies
DanPatterson_Retired
MVP Esteemed Contributor

mod = sys.modules[extant]

do you have....

import sys

at the top of your script in the imports section?  sys is the NoneType, so I suspect it isn't being imported.

Perhaps you might want to show your code just in case it is somewhere else.

PaulMcCord1
New Contributor II

Thanks Dan,

I appreciate your suggestion. I do have import sys at the beginning of my script (sorry for not initially including my code). The script is fairly long, but I'm including the primary parts of it here:

#Import modules
import os
import pandas as pd
import numpy as np
import pygal as pg
import sys
import arcpy

arcpy.env.overwriteOutput = True
arcpy.env.outputCoordinateSystem = arcpy.SpatialReference(3857)

def plotFAO(df, iso_code, fao_indicator, itemCodeFS, countrySelection, outputLocation):
     arcpy.AddMessage("IN FAO LOOP")
     #Select the appropriate indicator
     df_selected = df[df["Item Code"] == itemCodeFS]
     #color code for plots
     customColorStyle = pg.style.Style(colors = ['#D35400', '#979A9A', '#797D7F', '#5F6A6A', '#515A5A'], title_font_size = 18)
     arcpy.AddMessage("added Style")
     if itemCodeFS == 21010:
          #Creat the line chart plot
          line_chart = pg.Line(title = u"Average Dietary Energy Supply Adequacy - \n" + countrySelection, x_title = "End Year of '3-Year Average'", y_title = "% - 3-Year Average", style = customColorStyle, show_dots = False, stroke_style = {'width': 3}, show_minor_y_labels = True)
          line_chart.x_labels = map(str, range(2001, 2018))
          #Get just the Value column for the country of interest and the country groupings
          country_series = df_selected.loc[df_selected["ISO_code"] == iso_code, "Value"]
          low_income_series = df_selected.loc[df_selected["ISO_code"] == "lowestIncome", "Value"]
          mid_lower_income_series = df_selected.loc[df_selected["ISO_code"] == "midLowerIncome", "Value"]
          mid_upper_income_series = df_selected.loc[df_selected["ISO_code"] == "midUpperIncome", "Value"]
          high_income_series = df_selected.loc[df_selected["ISO_code"] == "uppermostIncome", "Value"]
          #Add Pd Series to the line chart
          line_chart.add(countrySelection, country_series)
          line_chart.add("Low-Income", low_income_series)
          line_chart.add("Lower-Middle-Income", mid_lower_income_series)
          line_chart.add("Upper-Middle-Income", mid_upper_income_series)
          line_chart.add("High-Income", high_income_series)
          #Output the plot
          line_chart.render_to_file(os.path.join(outputLocation, 'test_pygal3.svg'))

def foodSecurity_v1():

     #FAO Data
     fao_clean = "C:\\Users\\Paul McCord\\Google Drive\\Toolbox\\Tool_First\\FoodSecurity\\Content_Post052418\\Data_Tabular\\FAOSTAT\\fao_clean.csv"
     #Country ISO codes
     country_iso_csv = "C:\\Users\\Paul McCord\\Google Drive\\Toolbox\\Tool_First\\FoodSecurity\\Content_Post052418\\Data_Tabular\\country_codes\\FAO_Area_Export.csv"

     #Set tool parameters
     #Local variables
     out_src_FeatureSet = r"in_memory\src_feature_set"
     #User specified parameters
     src_FeatureSet = arcpy.GetParameterAsText(0)
     countrySelection = arcpy.GetParameterAsText(1)
     fao_indicator = arcpy.GetParameterAsText(2)
     outputLocation = arcpy.GetParameterAsText(3)

     try:
          #Save the Feature Set to Feature Layer
          arcpy.MakeFeatureLayer_management(src_FeatureSet, out_src_FeatureSet)
          aoi = arcpy.CopyFeatures_management(out_src_FeatureSet, os.path.join(arcpy.env.scratchFolder, "AOI_layer.shp"))
          
          #Obtain the country ISO code to avoid formatting problems with country name
          df_iso = pd.read_csv(country_iso_csv)
          countryDF = df_iso[df_iso["Area"] == countrySelection]
          iso_code = countryDF["ISO_code"].iloc[0]
          arcpy.AddMessage("country: " + countrySelection + ", code: " + iso_code)
          arcpy.AddMessage(iso_code)
          
          #Obtain the Item Code and Item Description selected by the user
          if fao_indicator == "Average dietary energy supply adequacy (%) (3-year average)":
               itemCodeFS = 21010
          if fao_indicator == "Average value of food production (constant I$ per person) (3-year average)":
               itemCodeFS = 21011
          if fao_indicator == "Share of dietary energy supply derived from cereals, roots and tubers (%) (3-year average)":
               itemCodeFS = 21012
          if fao_indicator == "Average protein supply (g/capita/day) (3-year average)":
               itemCodeFS = 21013
          if fao_indicator == "Average supply of protein of animal origin (g/capita/day) (3-year average)":
               itemCodeFS = 21014
          if fao_indicator == "Rail-lines density (per 100 square km of land area)":
               itemCodeFS = 21016
          if fao_indicator == "Gross domestic product per capita, PPP (constant 2011 international $)":
               itemCodeFS = 22013
          if fao_indicator == "Prevalence of undernourishment (%) (3-year average)":
               itemCodeFS = 21004
          if fao_indicator == "Depth of the food deficit (kcal/capita/day) (3-year average)":
               itemCodeFS = 21023
          
          #Select FAO Food Security information of interest
          df = pd.read_csv(fao_clean, dtype = {"Area": "S100", "Area Code": "f8", "Element": "S25", "Element Code": "f8", "Flag": "S25", 
                                                       "Flag Description": "S25", "ISO_code": "S25", "Item": "S150", "Item Code": "i4", "Unit": "S25", 
                                                       "Value": "f8", "Year": "S25", "Year Code": "f8", "YearEnd": "i4"})
          
          plotFAO(df, iso_code, fao_indicator, itemCodeFS, countrySelection, outputLocation)

          #test output layer
          arcpy.SetParameter(4, aoi)
          #arcpy.SetParameter(4, fig)
               
     #except Exception:
     #     e = sys.exc_info()[1]
     #     arcpy.AddError('An error occurred: {}'.format(e.args[0]))
          
     except Exception:
          if exc_info is not None and exec_info is not (None, None, None):
               e = sys.exc_info()[1]
               arcpy.AddError('An error occurred: {}'.format(e.args[0]))
     
if __name__ == '__main__':
     foodSecurity_v1()

There are actually nine food security indicators that I'm plotting, but I've only included the code for the first indicator in the 'plotFAO' function. The other plotting snippets look nearly identical, just with different food security indicators.

I've tried a couple of options in the 'except' statement, but none have been successful. Thanks again for weighing in.

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

Paul... I hate try-except blocks, because it is failing without a useful message

exc_info  I presume it is defined somewhere?

and I am still having trouble with

mod = sys.modules[extant]  

which means 'extant' is a variable which a key yielding a module... 

so what is extant a variable or a key (ie 'extant' not extant)

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

pygal isn't being picked up, but is the 2nd time?

# ---- try changing the order of imports

import sys
import os
import numpy as np
import pandas as pd
import pygal as pg
import arcpy‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

and try moving everything to a folder on your computer without spaces like c:\test

0 Kudos
PaulMcCord2
New Contributor III

Dan,

Thanks again for the suggestions. The try-except statement from my initial script had some careless errors on my part. I've now fixed this. Here is the script again with the corrections made.

#Import modules
import sys
import os
import numpy as np
import pandas as pd
import pygal as pg
import arcpy

arcpy.env.overwriteOutput = True
arcpy.env.outputCoordinateSystem = arcpy.SpatialReference(3857)

def plotFAO(df, iso_code, fao_indicator, itemCodeFS, countrySelection, outputLocation):
     arcpy.AddMessage("IN FAO LOOP")
     #Select the appropriate indicator
     df_selected = df[df["Item Code"] == itemCodeFS]
     #color code for plots
     customColorStyle = pg.style.Style(colors = ['#D35400', '#979A9A', '#797D7F', '#5F6A6A', '#515A5A'], title_font_size = 18)
     arcpy.AddMessage("added Style")
     if itemCodeFS == 21010:
          #Creat the line chart plot
          line_chart = pg.Line(title = u"Average Dietary Energy Supply Adequacy - \n" + countrySelection, x_title = "End Year of '3-Year Average'", y_title = "% - 3-Year Average", style = customColorStyle, show_dots = False, stroke_style = {'width': 3}, show_minor_y_labels = True)
          line_chart.x_labels = map(str, range(2001, 2018))
          #Get just the Value column for the country of interest and the country groupings
          country_series = df_selected.loc[df_selected["ISO_code"] == iso_code, "Value"]
          low_income_series = df_selected.loc[df_selected["ISO_code"] == "lowestIncome", "Value"]
          mid_lower_income_series = df_selected.loc[df_selected["ISO_code"] == "midLowerIncome", "Value"]
          mid_upper_income_series = df_selected.loc[df_selected["ISO_code"] == "midUpperIncome", "Value"]
          high_income_series = df_selected.loc[df_selected["ISO_code"] == "uppermostIncome", "Value"]
          #Add Pd Series to the line chart
          line_chart.add(countrySelection, country_series)
          line_chart.add("Low-Income", low_income_series)
          line_chart.add("Lower-Middle-Income", mid_lower_income_series)
          line_chart.add("Upper-Middle-Income", mid_upper_income_series)
          line_chart.add("High-Income", high_income_series)
          #Output the plot
          line_chart.render_to_file(os.path.join(outputLocation, 'test_pygal3.svg'))

def foodSecurity_v1():

     #FAO Data
     fao_clean = "C:\\test\\foodSecurity\\fao_clean.csv"
     #Country ISO codes
     country_iso_csv = "C:\\test\\foodSecurity\\FAO_Area_Export.csv"

     #Set tool parameters
     #Local variables
     out_src_FeatureSet = r"in_memory\src_feature_set"
     #User specified parameters
     src_FeatureSet = arcpy.GetParameterAsText(0)
     countrySelection = arcpy.GetParameterAsText(1)
     fao_indicator = arcpy.GetParameterAsText(2)
     outputLocation = arcpy.GetParameterAsText(3)

     try:
          #Save the Feature Set to Feature Layer
          arcpy.MakeFeatureLayer_management(src_FeatureSet, out_src_FeatureSet)
          aoi = arcpy.CopyFeatures_management(out_src_FeatureSet, os.path.join(arcpy.env.scratchFolder, "AOI_layer.shp"))
          
          #Obtain the country ISO code to avoid formatting problems with country name
          df_iso = pd.read_csv(country_iso_csv)
          countryDF = df_iso[df_iso["Area"] == countrySelection]
          iso_code = countryDF["ISO_code"].iloc[0]
          arcpy.AddMessage("country: " + countrySelection + ", code: " + iso_code)
          arcpy.AddMessage(iso_code)
          
          #Obtain the Item Code and Item Description selected by the user
          if fao_indicator == "Average dietary energy supply adequacy (%) (3-year average)":
               itemCodeFS = 21010
          if fao_indicator == "Average value of food production (constant I$ per person) (3-year average)":
               itemCodeFS = 21011
          if fao_indicator == "Share of dietary energy supply derived from cereals, roots and tubers (%) (3-year average)":
               itemCodeFS = 21012
          if fao_indicator == "Average protein supply (g/capita/day) (3-year average)":
               itemCodeFS = 21013
          if fao_indicator == "Average supply of protein of animal origin (g/capita/day) (3-year average)":
               itemCodeFS = 21014
          if fao_indicator == "Rail-lines density (per 100 square km of land area)":
               itemCodeFS = 21016
          if fao_indicator == "Gross domestic product per capita, PPP (constant 2011 international $)":
               itemCodeFS = 22013
          if fao_indicator == "Prevalence of undernourishment (%) (3-year average)":
               itemCodeFS = 21004
          if fao_indicator == "Depth of the food deficit (kcal/capita/day) (3-year average)":
               itemCodeFS = 21023
          
          #Select FAO Food Security information of interest
          df = pd.read_csv(fao_clean, dtype = {"Area": "S100", "Area Code": "f8", "Element": "S25", "Element Code": "f8", "Flag": "S25", 
                                                       "Flag Description": "S25", "ISO_code": "S25", "Item": "S150", "Item Code": "i4", "Unit": "S25", 
                                                       "Value": "f8", "Year": "S25", "Year Code": "f8", "YearEnd": "i4"})
          
          plotFAO(df, iso_code, fao_indicator, itemCodeFS, countrySelection, outputLocation)

          #test output layer
          arcpy.SetParameter(4, aoi)
          #arcpy.SetParameter(4, fig)
               
     except Exception:
          e = sys.exc_info()[1]
          arcpy.AddError('An error occurred: {}'.format(e.args[0]))
     
if __name__ == '__main__':
     foodSecurity_v1()

I've tried several ways to correct this issue where the tool fails the second time it is run, including removing the try-except statement, rearranging the order in which the Pygal module is imported, and placing the script and all csv files on which it relies in a folder where all spaces in the path have been removed. 

Unfortunately, the process still fails the second time the tool is run. So, when I run it the first time, I'm able to produce the FAO food security plot, but the second time I run the tool, I get the same 'NoneType' error. When I restart ArcMap, I'm able to run the tool again (but it then fails the second time the tool is run).

I've tried to isolate the problem, and what I've discovered is that when I comment out 'import pygal as pg' from the module imports at the beginning of the script, the tool will run as many times in succession as I like. But as soon as 'import pygal as pg' is included in my script, it fails the second time the tool runs. So, it isn't a particular Pygal function that is causing the tool to fail, it is the importing of Pygal itself. However, as you suggested, rearranging the order in which the module is imported does not seem to fix the issue.

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

I still think it is namespace... so in desperation...

try

import pygal   # dump the pg 

change all references to pg.blah to pygal.blah  ie lines 16, 20...

and for good measure, add

del pygal

in your __name__=='__main__'   section after the func call

0 Kudos
PaulMcCord2
New Contributor III

Ugh, it's a stubborn issue. I'm now just importing pygal as you suggested (rather than 'import pygal as pg'). And I've changed all pg.blah to pygal.blah, as well as adding 'del pygal' to the end of the script. Unfortunately the problem remains in that I'm getting the same error (AttributeError: 'NoneType' object has no attribute 'modules').

0 Kudos
DanPatterson_Retired
MVP Esteemed Contributor

hmmmm switch to Pro  

The Anaconda of python is much better controlled

You might have an eclectic installation of python and something isn't being directed properly.  You could spend days on this... and you are almost at the point where buying a new machine and doing a fresh install would be cheaper.

Sorry

PaulMcCord2
New Contributor III

Thanks Dan, your assistance in guiding me through this issue was much appreciated!

0 Kudos