Arcpy - adding temporary files to display when running custom script

1492
6
Jump to solution
09-03-2021 04:12 AM
JakubWabinski
New Contributor III

Hey,

I was recently working on a pipeline for semi-automatic generalization of data. I was able to solve most of the problems using built-in tools and Model Builder but when it comes to selection and displacement of point features I haven't found an appropriate tool and decided to write my own Python script.

So the script is not optimized but at least it works. The problem is that when I create a script tool based on this code with specific parameters, the intermediate files are created within workspace but are not added to display on the current map (while running within notebook, all the intermediate files are added to display) and thus, functions that base on layers (e.g. DetectGraphicConflict) do not work properly. 

What can I do to force the script to add the intermediate files from tools like CopyFeatures or Erase into the active map view so that then tools like DetectGraphicConflict could work?

def point_generalization(resource_type, 
                         sort_a = 'FREQUENCY', 
                         sort_b = 'sort_b', 
                         sort_c = 'sort_c', 
                         symbol_size = 7, 
                         min_distance = '2 Millimeters',
                         reference_scale = '2500000'):
    
    def clear_selection(layer_name):
        arcpy.management.SelectLayerByAttribute(layer_name, "CLEAR_SELECTION")
    
    try:
        arcpy.env.referenceScale = reference_scale
        search_distance = str(int(symbol_size) * arcpy.env.referenceScale / 1000) # TO BE PASSED AS FUNCTION ARGUMENTS
        displacement_value = str(int(symbol_size) * arcpy.env.referenceScale / 1000 / 2) # TO BE PASSED AS FUNCTION ARGUMENTS
        clear_selection(resource_type)

        # ----- First field for sorting ----- the number of features of specific type > those with the least occurences will be considered first in generalization    

        arcpy.analysis.Frequency(resource_type, 
                                 'ResourceCount', 
                                 'Type') # Create a table that will hold the frequency of particular resource types
        arcpy.management.JoinField(resource_type, 
                                   'Type', 
                                   'ResourceCount', 
                                   'Type', 
                                   'FREQUENCY')

        # ----- Second field for sorting ----- the number of conflicting objects > those with the most conflicts will be considered first in generalization

        arcpy.management.AddField(resource_type, 
                                  sort_b, 
                                  'LONG') # add field to hold the number of proximate points (function's argument)
        with arcpy.da.UpdateCursor(resource_type, ('OBJECTID', sort_b)) as cursor: # iterate over feature class rows to update values    
            for row in cursor:  
                selection = arcpy.management.SelectLayerByAttribute(resource_type, 
                                                                    'NEW_SELECTION', 
                                                                    'OBJECTID = {}'.format(row[0])) # select by attribute - iterate over every element in feature class
                arcpy.management.CopyFeatures(selection, 
                                              'temporary') # create a temporary layer based on the previous selection
                clear_selection(resource_type) # Clear the current selection - just in case
                arcpy.analysis.Erase(resource_type, 
                                     'temporary', 
                                     'erase') # Perform Erase operation to get a subtraction of original layer and the selected feature

                arcpy.cartography.DetectGraphicConflict('temporary',
                                                       'erase',
                                                       'conflict'
                                                       ) # Detect conflicting area between the selected feature (iterated) and the rest of the features in the original feature class
                conflicting_features = arcpy.management.SelectLayerByLocation(resource_type,
                                                      'INTERSECT',
                                                      'conflict',
                                                      search_distance = displacement_value + ' Meters' 
                                                      ) # Select objects that cause the conflict to determine their Mean Center
                row[1] = int(arcpy.management.GetCount(conflicting_features)[0]) # Get the number of featuers selected and assign the function's outpoot as a new column in the row
                cursor.updateRow(row) # update the current iteration row with the new value 

        # ----- Third field for sorting ----- the number of features in general within a specified threshold. Those with lower value will be considered first in generalization

        clear_selection(resource_type) # Clear selections just in case            
        arcpy.management.AddField(resource_type, 
                                  sort_c, 
                                  'LONG') # add field to hold the number of proximate points
        with arcpy.da.UpdateCursor(resource_type, ('OBJECTID', sort_c)) as cursor2: # other type of cursor for updating fields    
            for row in cursor2:                
                selection2 = arcpy.SelectLayerByAttribute_management(resource_type, 
                                                                     'NEW_SELECTION', 
                                                                     'OBJECTID = {}'.format(row[0]))
                proximity_count2 = arcpy.management.GetCount(
                    arcpy.management.SelectLayerByLocation(selection2, 
                                                           'WITHIN_A_DISTANCE', 
                                                           resource_type, 
                                                           str(float(search_distance)*2)
                                                          )
                                                        )
                row[1] = int(proximity_count2.getOutput(0)) # convert 'Result' type object from .GetCount() into integer by using .getOutput()
                cursor2.updateRow(row) # update the current iteration row with the new value 

        clear_selection(resource_type) # Clear selections just in case
        resource_type_sorted = resource_type + '_sorted'
        arcpy.management.Sort(resource_type, 
                              resource_type_sorted, 
                              [[sort_a, 'ASCENDING'],[sort_b, 'DESCENDING'],  [sort_c, 'ASCENDING']]
                             )

        # ------------------------------------------- Generalization -------------------------------------------

        # Deleting the newly generated fields
        fclasses_list = [resource_type, resource_type_sorted]
        exclude_list = ['OBJECTID', 'Shape', 'POINT_X', 'POINT_Y', 'POINT_Z', 'Type', 'sort_b']

        for fclass in fclasses_list:
            for field in arcpy.ListFields(fclass):
                if field.name in exclude_list:
                    continue
                else: 
                    arcpy.management.DeleteField(fclass, 
                                                 field.name)

        resource_type_generalized = resource_type + '_generalized'
        arcpy.management.CreateFeatureclass(arcpy.env.workspace,  
                                            resource_type_generalized, 
                                            'POINT', 
                                            template = resource_type_sorted, 
                                            spatial_reference = arcpy.Describe(resource_type_sorted).name
                                           ) # Create a feature class to hold generalized outputs (mean_centers)

        with arcpy.da.SearchCursor(resource_type_sorted, ('OBJECTID', 'sort_b')) as cursor:     
            for row in cursor:
                arcpy.management.ClearWorkspaceCache() # For optimization
                selection = arcpy.SelectLayerByAttribute_management(resource_type_sorted, 
                                                                    'NEW_SELECTION', 
                                                                    'OBJECTID = {}'.format(row[0])) # select by attribute - iterate over every element in feature class
                arcpy.management.CopyFeatures(selection, 
                                              'temporary') # create a temporary layer based on the previous selection
                clear_selection(resource_type_sorted) # Clear the current selection - just in case
                arcpy.analysis.Erase(resource_type_sorted, 
                                     'temporary', 
                                     'erase') # Perform Erase operation to get a subtraction of original layer and the selected feature

                arcpy.cartography.DetectGraphicConflict('temporary',
                                                       'erase',
                                                       'conflict'
                                                       ) # Detect conflicting area between the selected feature (iterated) and the rest of the features in the original feature class

                conflicting_features = arcpy.management.SelectLayerByLocation(resource_type_sorted,
                                                                              'INTERSECT',
                                                                              'conflict',
                                                                              search_distance = displacement_value + ' Meters' 
                                                                             ) # Select objects that cause the conflict to determine their Mean Center
                matchcount = int(arcpy.GetCount_management(conflicting_features)[0]) # Get the number of featuers selected

                # Control measures:
                print('row number: ' + str(row[0]))
                print('matched objects: ' + str(matchcount)) 
                print('sort_b value: ' + str(row[1]) +'\n' + str('-'*10))
                # ----------------
                if matchcount > 2:
                    arcpy.management.Append('temporary',
                                           resource_type_generalized
                                           )
                    arcpy.management.DeleteFeatures(conflicting_features) # Delete the original selected features
                if matchcount == 2:
                    MeanCenterTemp = arcpy.stats.MeanCenter(conflicting_features, 
                                                            'MeanCenter'
                                                           )
                    arcpy.management.DeleteField('MeanCenter', 
                                                 ['XCoord','YCoord','ZCoord']
                                                ) # Delete fields breaking the schema # !!!!!!!!!!!!!!!!!!!!!!!!!!!
                    # --------------------------------               
                    arcpy.management.AddField('MeanCenter', 
                                              'Type',
                                             'TEXT'
                                             )
                    arcpy.management.AddField('MeanCenter', 
                                              'sort_b',
                                             'LONG'
                                             )
                    with arcpy.da.UpdateCursor('MeanCenter', 'Type') as cursor1:
                        with arcpy.da.SearchCursor('temporary', 'Type') as cursor2:
                            for row1, row2 in zip(cursor1,cursor2):
                                row1[0]=row2[0]
                                cursor1.updateRow(row1)
                    # -------------------------------- 
                    arcpy.management.Append(MeanCenterTemp, 
                                            resource_type_generalized
                                           ) # Append generalization results to the newly created feature class
                    arcpy.management.Delete(MeanCenterTemp) # Delete the temporary MeanCenter with unwanted symbology
                    arcpy.management.DeleteFeatures(conflicting_features) # Delete the original selected features
                    clear_selection(resource_type_sorted) # Clear the current selection - just in case                                       
                else: 
                    arcpy.management.Append('temporary', 
                                            resource_type_generalized
                                           ) # Append generalization results to the newly created feature class 

        arcpy.management.DeleteIdentical(resource_type_generalized, 'Shape')
        arcpy.cartography.DisperseMarkers(resource_type_generalized, min_distance, 'EXPANDED') # get the layer name by using string object 
        
        print('SUCCESS!')
        
    finally:
        # --------------------------- HOW TO INCLUDE VARIABLES???
        arcpy.management.Delete(['temporary', 'erase', 'conflict', 'ResourceCount']) # , arcpy.Describe(resource_type_sorted).name
        
point_generalization(resource_type = 'Copper_ore' , symbol_size = 7)

 The script from the block above works fine when run from notebook within ArcGIS Pro but returns ERRORs 000732 in line 48 when run from custom script in toolbox: 

ERROR 000732: Input Layer: Dataset temporary does not exist or is not supported
ERROR 000732: Conflict Layer: Dataset erase does not exist or is not supported
Failed to execute (DetectGraphicConflict).
Failed to execute (ParameterTest).

I would be glad for any suggestions on how to fix this. Any way to make the custom script act as the notebook cell? Perhaps a solution is within some temporary memory workspace?

I know the script itself is messy and runs slow so maybe you could suggest some performance improvements?

Thank you in advance,

Jakub

0 Kudos
1 Solution

Accepted Solutions
JakubWabinski
New Contributor III

I will reply to myself cause perhaps somebody will find it useful. Well, I find this approach weird but in order to view stuff in ToC one has to create layers and (derived) parameters that should be referred using SetParameterAsText function. I have managed to fix my code based on these 2 discussions:

https://community.esri.com/t5/python-questions/script-tool-geoprocessing-output-not-displayed-to/td-...

https://community.esri.com/t5/python-questions/explain-arcpy-setparameterastext-to-me-like-a-5th/td-...

View solution in original post

0 Kudos
6 Replies
DanPatterson
MVP Esteemed Contributor

on a quick scan I didn't see any

Make Feature Layer (Data Management)—ArcGIS Pro | Documentation

instances to make featurelayers of what you want.  These won't persist after the script is run unless copy or save is run to keep them


... sort of retired...
0 Kudos
JakubWabinski
New Contributor III

Thanks Dan, that is correct. I was not using them.

I was thinking about this solution but this requires adding many additional operations and probably affect the performance? Isn't there an easier and more optimal way?

What is actually going on behind the scenes that in case of notebook all the features created are automatically added to ToC but not when running a script tool?

0 Kudos
JakubWabinski
New Contributor III

Hello again. I tried using Make Feature Layer. Well the difference is that the tool is able to proceed now but the layers are not showing in ToC anyway. Something's wrong but it looks more like a problem with workspace/environment...?

0 Kudos
DanPatterson
MVP Esteemed Contributor

I never have any issues with outputs being added to a map... Check this settingadd_results_to_map2.png


... sort of retired...
0 Kudos
JakubWabinski
New Contributor III

I have this setting checked. Please see the screenshot. Map is active, parameters are set, feature classes were created in the database but not added to display.

 

JakubWabinski_0-1630681663785.png

 

0 Kudos
JakubWabinski
New Contributor III

I will reply to myself cause perhaps somebody will find it useful. Well, I find this approach weird but in order to view stuff in ToC one has to create layers and (derived) parameters that should be referred using SetParameterAsText function. I have managed to fix my code based on these 2 discussions:

https://community.esri.com/t5/python-questions/script-tool-geoprocessing-output-not-displayed-to/td-...

https://community.esri.com/t5/python-questions/explain-arcpy-setparameterastext-to-me-like-a-5th/td-...

0 Kudos