ArcGIS Pro Crash while increasing the number of input layers

3837
25
11-09-2018 02:15 AM
FengchaoGu1
New Contributor II

I have one feedback about ArcGIS Pro 2.2. It crashes every time when I run a toolbox with increased number of shapefiles (around 56) as input parameters.  The toolbox is created by our team and below is the detailed description.

The function of this toolbox:

Get the input parameters and make attributes selection.

Main arcpy functions:

input = arcpy.GetParameterAsText()

arcpy.SelectLayerByAttribute_management()

Result:

This toolbox can run successfully with flexible numbers of input shapefiles in ArcMap 10.5 or ArcMap 10.6, no matter how many the input layers are.  

G
M
T
Detect language Afrikaans Albanian Amharic Arabic Armenian Azerbaijani Basque Belarusian Bengali Bosnian Bulgarian Catalan Cebuano Chichewa Chinese (Simplified) Chinese (Traditional) Corsican Croatian Czech Danish Dutch English Esperanto Estonian Filipino Finnish French Frisian Galician Georgian German Greek Gujarati Haitian Creole Hausa Hawaiian Hebrew Hindi Hmong Hungarian Icelandic Igbo Indonesian Irish Italian Japanese Javanese Kannada Kazakh Khmer Korean Kurdish Kyrgyz Lao Latin Latvian Lithuanian Luxembourgish Macedonian Malagasy Malay Malayalam Maltese Maori Marathi Mongolian Myanmar (Burmese) Nepali Norwegian Pashto Persian Polish Portuguese Punjabi Romanian Russian Samoan Scots Gaelic Serbian Sesotho Shona Sindhi Sinhala Slovak Slovenian Somali Spanish Sundanese Swahili Swedish Tajik Tamil Telugu Thai Turkish Ukrainian Urdu Uzbek Vietnamese Welsh Xhosa Yiddish Yoruba Zulu
Afrikaans Albanian Amharic Arabic Armenian Azerbaijani Basque Belarusian Bengali Bosnian Bulgarian Catalan Cebuano Chichewa Chinese (Simplified) Chinese (Traditional) Corsican Croatian Czech Danish Dutch English Esperanto Estonian Filipino Finnish French Frisian Galician Georgian German Greek Gujarati Haitian Creole Hausa Hawaiian Hebrew Hindi Hmong Hungarian Icelandic Igbo Indonesian Irish Italian Japanese Javanese Kannada Kazakh Khmer Korean Kurdish Kyrgyz Lao Latin Latvian Lithuanian Luxembourgish Macedonian Malagasy Malay Malayalam Maltese Maori Marathi Mongolian Myanmar (Burmese) Nepali Norwegian Pashto Persian Polish Portuguese Punjabi Romanian Russian Samoan Scots Gaelic Serbian Sesotho Shona Sindhi Sinhala Slovak Slovenian Somali Spanish Sundanese Swahili Swedish Tajik Tamil Telugu Thai Turkish Ukrainian Urdu Uzbek Vietnamese Welsh Xhosa Yiddish Yoruba Zulu
Text-to-speech function is limited to 200 characters

The toolbox can run successfully with less flexible number of input shapefiles (from 1 to 20) in ArcGIS Pro 2.2.

However, when the number of input shapefiles is increased to 56, it crashes and I get the error “ArcGIS Pro has encountered a serious application error and is unable to continue”.

0 Kudos
25 Replies
ThomasColson
MVP Frequent Contributor

Can you share your Python script and toolbox? Not a lot to troubleshoot using just screen shots in a Python error. 

0 Kudos
FengchaoGu1
New Contributor II

yes, sure. I will upload two different scripts, one is for ArcMap, and another one is for ArcGIS Pro. 

Below is the ArcMap version.

import arcpy
from arcpy import env
import sys
import os
import openpyxl
from subprocess import Popen

env.overwriteOutput = True

# User imports
input = arcpy.GetParameterAsText(0)
fc1234_boolean = arcpy.GetParameterAsText(1)
xlsx_boolean = arcpy.GetParameterAsText(2)

# Other variables
inputlist = input.split(";")
inputfile_first = inputlist[0]
inputlyr = arcpy.mapping.Layer(inputfile_first)  # lyr
inputpath = inputlyr.dataSource  # path
inputpath0 = os.path.split(inputpath)[0]

mxd = arcpy.mapping.MapDocument("CURRENT")
df = arcpy.mapping.ListDataFrames(mxd, "Layers")[0]
home_f = os.path.dirname(mxd.filePath)
testcode = os.path.basename(home_f)
year = testcode.split("_")[0]
country = testcode.split("_")[1]
city_id = testcode.split("_")[2]
cityshort = testcode.split("_")[3]
city_index = testcode.split("_")[4]
testcodeshort = year + '_' + cityshort
working_f = home_f + '\\Working_folder\\'
working_space = arcpy.env.workspace  # Default.gdb
dr_file = home_f + '\\NAV\\Drive_route\\' + testcode + "_driveroute.shp"

streets_nav = testcodeshort + "_Streets_NAV"
streets_nav_fc1234 = testcodeshort + "_Streets_NAV_FC1234"
streets_nav_fc1234_long = testcode + "_Streets_NAV_FC1234"
rdms = testcodeshort + "_RDM_line"
rdms_fc12 = testcodeshort + "_RDM_line_FC12"
rdms_fc34 = testcodeshort + "_RDM_line_FC34"
ows = testcodeshort + "_OW"
ows_fc12 = testcodeshort + "_OW_FC12"
ows_fc34 = testcodeshort + "_OW_FC34"
Geo_Stats = working_space + "\\Geo_Stats"
OW_Stats = working_space + "\\OW_Stats"
RDM_Stats = working_space + "\\RDM_Stats"
linkidList = []

# Preset/predefine calculation variables to '0' to prevent errors later in script with empty values
FC1len = 0
FC2len = 0
FC3len = 0
FC4len = 0
FC5len = 0
owFC1len = 0
owFC2len = 0
owFC3len = 0
owFC4len = 0
owFC5len = 0
rdmFC1cnt = 0
rdmFC2cnt = 0
rdmFC3cnt = 0
rdmFC4cnt = 0
rdmFC5cnt = 0

# Selecting all links in streets_nav or streets_nav_fc1234
if fc1234_boolean.lower() != 'true':
    arcpy.AddMessage("Making selection")
    for fc in inputlist:
        with arcpy.da.SearchCursor(fc, ["LINK_ID"]) as cursor:
            for row in cursor:
                linkidList.append(row[0])
    where = '"LINK_ID" = {0}'.format(int(linkidList[0]))
    del linkidList[0]
    for linkid in linkidList:
        where += ' OR "LINK_ID" = {0}'.format(int(linkid))
    arcpy.SelectLayerByAttribute_management(streets_nav, "NEW_SELECTION", where)
    arcpy.SelectLayerByAttribute_management(ows, "NEW_SELECTION", where)
    arcpy.SelectLayerByAttribute_management(rdms, "NEW_SELECTION", where)
else:
    arcpy.AddMessage("Making selection FC1234")
    layerList = []
    for lyr in arcpy.mapping.ListLayers(mxd, "", df):
        layerList.append(lyr.name)
    for fc in inputlist:
        with arcpy.da.SearchCursor(fc, ["LINK_ID"]) as cursor:
            for row in cursor:
                linkidList.append(row[0])
    where = '"LINK_ID" = {0}'.format(int(linkidList[0]))
    del linkidList[0]
    for linkid in linkidList:
        where += ' OR "LINK_ID" = {0}'.format(int(linkid))
    if streets_nav_fc1234 in layerList:
        streets_layer = streets_nav_fc1234
    elif streets_nav_fc1234_long in layerList:
        streets_layer = streets_nav_fc1234_long
    arcpy.SelectLayerByAttribute_management(streets_layer, "NEW_SELECTION", where)
    if ows_fc12 in layerList:
        arcpy.SelectLayerByAttribute_management(ows_fc12, "NEW_SELECTION", where)
    if ows_fc34 in layerList:
        arcpy.SelectLayerByAttribute_management(ows_fc34, "NEW_SELECTION", where)
    if rdms_fc12 in layerList:
        arcpy.SelectLayerByAttribute_management(rdms_fc12, "NEW_SELECTION", where)
    if rdms_fc34 in layerList:
        arcpy.SelectLayerByAttribute_management(rdms_fc34, "NEW_SELECTION", where)

# Zoom to selection
df.zoomToSelectedFeatures()
df.scale = df.scale * 1.1

# Make .xlsx if optional output is given
if xlsx_boolean.lower() == 'true':
    if fc1234_boolean.lower() == 'true':
        arcpy.AddError(
            "Not possible to write only FC1234 attributes to an .xlsx file. Run tool again without this option")
        sys.exit()

    arcpy.AddMessage("Calculating counts for .xlsx")
    # Geo statistics per fc
    arcpy.Statistics_analysis(streets_nav, Geo_Stats, "LEN SUM", "FUNC_CLASS")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '1'", "", "", "")
    for row in rows:
        FC1len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '2'", "", "", "")
    for row in rows:
        FC2len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '3'", "", "", "")
    for row in rows:
        FC3len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '4'", "", "", "")
    for row in rows:
        FC4len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '5'", "", "", "")
    for row in rows:
        FC5len = row.getValue("SUM_LEN")

    geoLen12 = FC1len + FC2len
    geoLen34 = FC3len + FC4len
    geoLen5 = FC5len
    # OW statistics per fc
    arcpy.Statistics_analysis(ows, OW_Stats, "LEN SUM", "FUNC_CLASS")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '1'", "", "", "")
    for row in rows:
        owFC1len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '2'", "", "", "")
    for row in rows:
        owFC2len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '3'", "", "", "")
    for row in rows:
        owFC3len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '4'", "", "", "")
    for row in rows:
        owFC4len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '5'", "", "", "")
    for row in rows:
        owFC5len = row.getValue("SUM_LEN")

    owLen12 = owFC1len + owFC2len
    owLen34 = owFC3len + owFC4len
    owLen5 = owFC5len
    # RDM statistics per fc
    arcpy.Statistics_analysis(rdms, RDM_Stats, "LEN SUM", "FUNC_CLASS")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '1'", "", "", "")
    for row in rows:
        rdmFC1cnt = row.getValue("FREQUENCY")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '2'", "", "", "")
    for row in rows:
        rdmFC2cnt = row.getValue("FREQUENCY")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '3'", "", "", "")
    for row in rows:
        rdmFC3cnt = row.getValue("FREQUENCY")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '4'", "", "", "")
    for row in rows:
        rdmFC4cnt = row.getValue("FREQUENCY")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '5'", "", "", "")
    for row in rows:
        rdmFC5cnt = row.getValue("FREQUENCY")

    rdmCnt12 = rdmFC1cnt + rdmFC2cnt
    rdmCnt34 = rdmFC3cnt + rdmFC4cnt
    rdmCnt5 = rdmFC5cnt
    # Drive_route number of links
    count_dr = arcpy.GetCount_management(dr_file)
    count_dr_int = int(count_dr.getOutput(0))

    # Write outputs to .xlsx
    # Using excel template or create new one if not there
    if os.path.isfile(
            r'C:\Program Files (x86)\ArcGIS\Desktop10.5\Styles\Prep_layer_definition\Calc_templates\calc_dr_selection_temp.xlsx'):
        dr_selection_temp = openpyxl.load_workbook(
            r'C:\Program Files (x86)\ArcGIS\Desktop10.5\Styles\Prep_layer_definition\Calc_templates\calc_dr_selection_temp.xlsx')
    else:
        dr_selection_temp = openpyxl.Workbook()
    sheet = dr_selection_temp.get_active_sheet()

    # Print numbers
    sheet['D3'] = geoLen12
    sheet['F3'] = geoLen34
    sheet['H3'] = geoLen5

    sheet['D4'] = owLen12
    sheet['F4'] = owLen34
    sheet['H4'] = owLen5

    sheet['D5'] = rdmCnt12
    sheet['F5'] = rdmCnt34
    sheet['H5'] = rdmCnt5

    sheet['D6'] = geoLen12
    sheet['F6'] = geoLen34
    sheet['H6'] = geoLen5

    sheet['B13'] = count_dr_int

    # Saving file to Working_folder
    try:
        arcpy.AddMessage("Selection saved in {0}_calc_dr_selection.xlsx (in Working_folder)".format(testcode))
        dr_selection_temp.save(working_f + '\\' + testcode + '_calc_dr_selection.xlsx')
        p = Popen(working_f + '\\' + testcode + '_calc_dr_selection.xlsx', shell=True)
    except:
        arcpy.AddError(testcode + "_calc_dr_selection.xlsx is already open. Close it first before running the tool")
        sys.exit()
0 Kudos
FengchaoGu1
New Contributor II

Here is the ArcGIS Pro version.

import arcpy
from arcpy import env
import sys
import os
import openpyxl
from subprocess import Popen

env.overwriteOutput = True

# User imports
input = arcpy.GetParameterAsText(0)
fc1234_boolean = arcpy.GetParameterAsText(1)
xlsx_boolean = arcpy.GetParameterAsText(2)

# Other variables
inputlist = input.split(";")
inputfile_first = inputlist[0]
#inputlyr = arcpy.mapping.Layer(inputfile_first)  # lyr
desc = arcpy.Describe(inputfile_first)
inputpath = desc.path
#inputpath = inputlyr.dataSource  # path
inputpath0 = os.path.split(inputpath)[0]

aprx = arcpy.mp.ArcGISProject("CURRENT")
aprxMap = aprx.listMaps("Map")[0]
home_f = os.path.dirname(aprx.filePath)
testcode = os.path.basename(home_f)
year = testcode.split("_")[0]
country = testcode.split("_")[1]
city_id = testcode.split("_")[2]
cityshort = testcode.split("_")[3]
city_index = testcode.split("_")[4]
testcodeshort = year + '_' + cityshort
working_f = home_f + '\\Working_folder\\'
working_space = arcpy.env.workspace  # Default.gdb
dr_file = home_f + '\\NAV\\Drive_route\\' + testcode + "_driveroute.shp"

streets_nav = testcodeshort + "_Streets_NAV"
streets_nav_fc1234 = testcodeshort + "_Streets_NAV_FC1234"
streets_nav_fc1234_long = testcode + "_Streets_NAV_FC1234"
rdms = testcodeshort + "_RDM_line"
rdms_fc12 = testcodeshort + "_RDM_line_FC12"
rdms_fc34 = testcodeshort + "_RDM_line_FC34"
ows = testcodeshort + "_OW"
ows_fc12 = testcodeshort + "_OW_FC12"
ows_fc34 = testcodeshort + "_OW_FC34"
Geo_Stats = working_space + "\\Geo_Stats"
OW_Stats = working_space + "\\OW_Stats"
RDM_Stats = working_space + "\\RDM_Stats"
linkidList = []

# Preset/predefine calculation variables to '0' to prevent errors later in script with empty values
FC1len = 0
FC2len = 0
FC3len = 0
FC4len = 0
FC5len = 0
owFC1len = 0
owFC2len = 0
owFC3len = 0
owFC4len = 0
owFC5len = 0
rdmFC1cnt = 0
rdmFC2cnt = 0
rdmFC3cnt = 0
rdmFC4cnt = 0
rdmFC5cnt = 0

# Selecting all links in streets_nav or streets_nav_fc1234
if fc1234_boolean.lower() != 'true':
    arcpy.AddMessage("Making selection")
    for fc in inputlist:
        with arcpy.da.SearchCursor(fc, ["LINK_ID"]) as cursor:
            for row in cursor:
                linkidList.append(row[0])
    where = '"LINK_ID" = {0}'.format(int(linkidList[0]))
    del linkidList[0]
    for linkid in linkidList:
        where += ' OR "LINK_ID" = {0}'.format(int(linkid))
    arcpy.SelectLayerByAttribute_management(streets_nav, "NEW_SELECTION", where)
    arcpy.SelectLayerByAttribute_management(ows, "NEW_SELECTION", where)
    arcpy.SelectLayerByAttribute_management(rdms, "NEW_SELECTION", where)
else:
    arcpy.AddMessage("Making selection FC1234")
    layerList = []
    for lyr in arcpy.mapping.ListLayers(mxd, "", df):
        layerList.append(lyr.name)
    for fc in inputlist:
        with arcpy.da.SearchCursor(fc, ["LINK_ID"]) as cursor:
            for row in cursor:
                linkidList.append(row[0])
    where = '"LINK_ID" = {0}'.format(int(linkidList[0]))
    del linkidList[0]
    for linkid in linkidList:
        where += ' OR "LINK_ID" = {0}'.format(int(linkid))
    if streets_nav_fc1234 in layerList:
        streets_layer = streets_nav_fc1234
    elif streets_nav_fc1234_long in layerList:
        streets_layer = streets_nav_fc1234_long
    arcpy.SelectLayerByAttribute_management(streets_layer, "NEW_SELECTION", where)
    if ows_fc12 in layerList:
        arcpy.SelectLayerByAttribute_management(ows_fc12, "NEW_SELECTION", where)
    if ows_fc34 in layerList:
        arcpy.SelectLayerByAttribute_management(ows_fc34, "NEW_SELECTION", where)
    if rdms_fc12 in layerList:
        arcpy.SelectLayerByAttribute_management(rdms_fc12, "NEW_SELECTION", where)
    if rdms_fc34 in layerList:
        arcpy.SelectLayerByAttribute_management(rdms_fc34, "NEW_SELECTION", where)

# # Zoom to selection
# df.zoomToSelectedFeatures()
# df.scale = df.scale * 1.1

# Make .xlsx if optional output is given
if xlsx_boolean.lower() == 'true':
    if fc1234_boolean.lower() == 'true':
        arcpy.AddError(
            "Not possible to write only FC1234 attributes to an .xlsx file. Run tool again without this option")
        sys.exit()

    arcpy.AddMessage("Calculating counts for .xlsx")
    # Geo statistics per fc
    arcpy.Statistics_analysis(streets_nav, Geo_Stats, "LEN SUM", "FUNC_CLASS")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '1'", "", "", "")
    for row in rows:
        FC1len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '2'", "", "", "")
    for row in rows:
        FC2len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '3'", "", "", "")
    for row in rows:
        FC3len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '4'", "", "", "")
    for row in rows:
        FC4len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(Geo_Stats, "FUNC_CLASS = '5'", "", "", "")
    for row in rows:
        FC5len = row.getValue("SUM_LEN")

    geoLen12 = FC1len + FC2len
    geoLen34 = FC3len + FC4len
    geoLen5 = FC5len
    # OW statistics per fc
    arcpy.Statistics_analysis(ows, OW_Stats, "LEN SUM", "FUNC_CLASS")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '1'", "", "", "")
    for row in rows:
        owFC1len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '2'", "", "", "")
    for row in rows:
        owFC2len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '3'", "", "", "")
    for row in rows:
        owFC3len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '4'", "", "", "")
    for row in rows:
        owFC4len = row.getValue("SUM_LEN")
    rows = arcpy.SearchCursor(OW_Stats, "FUNC_CLASS = '5'", "", "", "")
    for row in rows:
        owFC5len = row.getValue("SUM_LEN")

    owLen12 = owFC1len + owFC2len
    owLen34 = owFC3len + owFC4len
    owLen5 = owFC5len
    # RDM statistics per fc
    arcpy.Statistics_analysis(rdms, RDM_Stats, "LEN SUM", "FUNC_CLASS")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '1'", "", "", "")
    for row in rows:
        rdmFC1cnt = row.getValue("FREQUENCY")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '2'", "", "", "")
    for row in rows:
        rdmFC2cnt = row.getValue("FREQUENCY")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '3'", "", "", "")
    for row in rows:
        rdmFC3cnt = row.getValue("FREQUENCY")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '4'", "", "", "")
    for row in rows:
        rdmFC4cnt = row.getValue("FREQUENCY")
    rows = arcpy.SearchCursor(RDM_Stats, "FUNC_CLASS = '5'", "", "", "")
    for row in rows:
        rdmFC5cnt = row.getValue("FREQUENCY")

    rdmCnt12 = rdmFC1cnt + rdmFC2cnt
    rdmCnt34 = rdmFC3cnt + rdmFC4cnt
    rdmCnt5 = rdmFC5cnt
    # Drive_route number of links
    count_dr = arcpy.GetCount_management(dr_file)
    count_dr_int = int(count_dr.getOutput(0))

    # Write outputs to .xlsx
    # Using excel template or create new one if not there
    if os.path.isfile(
            r'C:\Program Files (x86)\ArcGIS\Desktop10.5\Styles\Prep_layer_definition\Calc_templates\calc_dr_selection_temp.xlsx'):
        dr_selection_temp = openpyxl.load_workbook(
            r'C:\Program Files (x86)\ArcGIS\Desktop10.5\Styles\Prep_layer_definition\Calc_templates\calc_dr_selection_temp.xlsx')
    else:
        dr_selection_temp = openpyxl.Workbook()
    sheet = dr_selection_temp.get_active_sheet()

    # Print numbers
    sheet['D3'] = geoLen12
    sheet['F3'] = geoLen34
    sheet['H3'] = geoLen5

    sheet['D4'] = owLen12
    sheet['F4'] = owLen34
    sheet['H4'] = owLen5

    sheet['D5'] = rdmCnt12
    sheet['F5'] = rdmCnt34
    sheet['H5'] = rdmCnt5

    sheet['D6'] = geoLen12
    sheet['F6'] = geoLen34
    sheet['H6'] = geoLen5

    sheet['B13'] = count_dr_int

    # Saving file to Working_folder
    try:
        arcpy.AddMessage("Selection saved in {0}_calc_dr_selection.xlsx (in Working_folder)".format(testcode))
        dr_selection_temp.save(working_f + '\\' + testcode + '_calc_dr_selection.xlsx')
        p = Popen(working_f + '\\' + testcode + '_calc_dr_selection.xlsx', shell=True)
    except:
        arcpy.AddError(testcode + "_calc_dr_selection.xlsx is already open. Close it first before running the tool")
        sys.exit()
0 Kudos
ThomasColson
MVP Frequent Contributor

Wow....that's a tech support case for sure.In the Pro version, what happens when you move 

calc_dr_selection_temp.xlsx

to another directory with no spaces or special characters and where your computer login has "Full Control" to the directory?

0 Kudos
FengchaoGu1
New Contributor II

Hi I tried to move it to another directory with no spaces or special characters but it does not work. I got the same crash. And since it works in ArcGIS Pro with less number of input, and it only crash when there are more than 50 input. So I do not really think it is about this .xlsx template.

0 Kudos
ColinZwicker
Esri Contributor

Fengchao,

Instead of building a large OR list for your linkID's 

where += ' OR "LINK_ID" = {0}'.format(int(linkid))

Can you try making the whereclause in the form of an in list?, e.g., IN (ID1, ID2, ID3, N)?  Ideally this IN clause would contain no duplicated ID's (so perhaps collect them in a list and then only add the unique values).

re: "I have tried to increase the size of virtual memory to 100GB. But unfortunately, I still get the same crash error. So it seems that because of the script itself. "

As the thread has somewhat devised this issue isnt about heap memory but about stack memory and the usage of that stack while attempting to parse a whereclause with a growing number of logical operators.

Thanks,

Colin

ColinZwicker
Esri Contributor

I have been told that making a list unique is relatively painless in python

ids = [1, 2, 3, 4, 1, 5, 6, 1, 2, 3]
>>> sorted(set(ids))
[1, 2, 3, 4, 5, 6]

FengchaoGu1
New Contributor II

@ Colin Zwicker  Thanks a lot!  I will try this way to adjust my scripts a little bit.  I will post the result later.  

0 Kudos
FengchaoGu1
New Contributor II

@Colin Zwicker and Marco Boeringa      I tried to split the list into two and it works (See the scripts below ). This is a workaround. 

if fc1234_boolean.lower() != 'true':
    arcpy.AddMessage("Making selection")
    for fc in inputlist:
        with arcpy.da.SearchCursor(fc, ["LINK_ID"]) as cursor:
            for row in cursor:
                linkidList.append(row[0])
    linkidList1 = []
    linkidList2 = []
    linkidList1 = linkidList[:len(linkidList)//2]
    linkidList2 = linkidList[len(linkidList)//2:]
    arcpy.AddMessage(linkidList1)
    arcpy.AddMessage(linkidList2)
    where1 = '"LINK_ID" = {0}'.format(int(linkidList1[0]))
    del linkidList1[0]
    for linkid in linkidList1:
        where1 += ' OR "LINK_ID" = {0}'.format(int(linkid))
    where2 = '"LINK_ID" = {0}'.format(int(linkidList2[0]))
    del linkidList2[0]
    for linkid in linkidList2:
        where2 += ' OR "LINK_ID" = {0}'.format(int(linkid))
    arcpy.AddMessage(where1)
    arcpy.AddMessage(where2)
    arcpy.SelectLayerByAttribute_management(streets_nav, "NEW_SELECTION", where1)
    arcpy.SelectLayerByAttribute_management(streets_nav, "NEW_SELECTION", where2)
    arcpy.SelectLayerByAttribute_management(ows, "NEW_SELECTION", where1)
    arcpy.SelectLayerByAttribute_management(ows, "NEW_SELECTION", where2)
    arcpy.SelectLayerByAttribute_management(rdms, "NEW_SELECTION", where1)
    arcpy.SelectLayerByAttribute_management(rdms, "NEW_SELECTION", where2)

Therefore, the feedback is:

1. With this SQL

where += ' OR "LINK_ID" = {0}'.format(int(linkid))

If the record is quite large, for example in this case 6090, then the arcpy function

"SelectLayerByAttribute_management" can not handle it.

arcpy.SelectLayerByAttribute_management(streets_nav, "NEW_SELECTION", where)

However, if the record has been splitted, then it can handle it.

It is a workaround to make the tool working but I did not figure it out the reason behind it.

Regards,

Fengchao