Skip navigation
All People > rfairhur24 > Richard Fairhurst's Blog > 2015 > July
2015

Google has added 25 million buildings in the United States this year to their Map view.  These building outlines can be converted to polygons in ArcGIS if you have Photoshop and do the following steps:

1.      Locate an area with buildings in Google Maps with the Map view on.  Maximize the browser window.  Zoom until you are at least at the second level zoom above the minimum level to make the buildings appear to get enough resolution. 
2.      Press the Alt+Prt Scr buttons on the keyboard to get a screen shot (or use your favorite screen shot software).

Google1.png
3.      Open Photshop and create a new project (on the menu use File -> New or press Ctrl+N).
4.      Paste the screenshot into the Photoshop project (on the menu use Edit -> Paste or Ctrl+V).


5.      From the Menu choose Select -> Color Range… (I added a custom keyboard shortcut of Alt+Shift+Ctrl+R)
6.      With the Select option set to Sampled Color click the eyedropper cursor on the center of a building.  Check the inverse option.  Press the OK button.


7.      With the selection still active create a new layer (on the menu use Layer -> New -> Layer… or press Shift+Ctrl+N).  Name the layer Building Background and press the OK button.


8.      Add a fill to the layer (from the menu use Edit -> Fill… or press Shift+F5). Choose the Mode “Black” and press the OK button.


9.      Deselect everything by pressing Ctrl+D.  From the menu choose Select -> Color Range... (I added a custom keyboard shortcut of Alt+Shift+Ctrl+R).  With the Select option set to Sampled Color click the eyedropper cursor on the center of a building.  Press the OK button.


10.  With the selection still active create a new layer (on the menu use Layer -> New -> Layer… or press Shift+Ctrl+N).  Name the layer Building Foreground and press the OK button.


11.  Add a fill to the layer (from the menu use Edit -> Fill… or press Shift+F5). Choose the Mode “White” and press the OK button.


12.  You should now have a Black and White image.  The outlines of the roads will have areas with white pixels that need to be painted out.  Set the palette color to Black.  Right click on the map and choose a paint brush size that fits comfortably within the roadway (about 40 pixels).  Paint out the small white pixels.  Also paint out partial buildings on the edges of the map.
13.  On the menu choose Image -> Mode -> Grayscale… (I added a custom keyboard shortcut to the Grayscale option of Alt+Shift+Ctrl+G) and then choose to not Flatten the image and to Discard the Color information.


14.  Save the photoshop project as a BMP.  Use 8 Bit depth (if it says higher depths you have a problem).
15.  Open a new ArcMap session and just add aerials or a satellite basemap.
16.  Right click the Date Frame and chose Date Frame Properties.  On the Freme tab choose the background to be black.
17.  Locate the same set of buildings on the aerial.
18.  Add the BMP you saved in step 14.  Say OK to the warning about the image not being spatially referenced.
19.  Change the BMP layer transparency to 50%.
20.  Right click the BMP layer and Zoom to Layer.
21.  Open the Georeferencing toolbar.


22.  Press the Add Control Points button and choose a corner of a building that you can match to the aerial.


23.  Press the Back button to return to the aerial and set the control point at the corner of the same building on the aerial.
24.  Set another control point that is vertical or horizontal to the previous point, not diagonal.
25.  Set additional control points if necessary, but not more than 4 total points.
26.  When you are satisfied with the georeferencing position on the toolbar choose Georeferencing -> Update Georeferencing.


27.  Set the BMP transparency to 0% and turn off the aerials.


28.  Open the Toolboxes -> System Toolboxes -> Conversion Tools Toolbox -> From Raster Toolset -> Raster to Polygon tool.
29.  Choose the bmp as the input and set the output location and name to a geodatabase feature class and press OK.
30.  Start an editor session on the polygon layer you just created and delete the background polygon.
31.  Open the Table View for the new polygon feature class and sort by shape area. Choose any very small polygons that are not buildings that you did not paint out and delete them.

32.  Set up a permanent feature class that you will append all new buildings to or append this set of buildings to the feature class you set up previously.  The result shown below created 115 building outline polygons (the polygons are set to hollow with a red outline that is 3 points thick).

I have wanted tools in ArcMap that let me do more things with Standalone Tables.   For example, there is no tool that will let me duplicate records in a Standalone Table or insert a row directly into an existing Standalone Table that came from the tabular portion of a feature.  I cannot Copy/Paste rows or features into a Standalone Table at all like I can with features in a Feature Class.  Even for features using Copy/Paste is slow and can only be done in an edit session.  The Append tool works for table to table inserts or feature class to feature inserts, but only if the input and output are different from each other.  The Append tool also will not let me take the tabular data of a feature and append it into an existing Standalone Table.

 

I wanted a tool that would quickly do inserts and work for Feature Class to Feature Class, Feature Class to Table, or Table to Table inserts.  I also wanted the selection behavior to be similar to the Copy/Paste behavior.  I wanted the tool to only work when records are first selected, and I wanted the selected records of the input to be cleared and the newly inserted records in the target to be selected when the tool finishes.  That is especially useful when the input and the target are the same table or feature class.  I also wanted the tool to automatically match fields like Copy and Paste does.

 

The field matching behavior is similar to using the default field map that appears in the Append tool when the NO TEST option is chosen, so the schemas don't have to match exactly.  However, unlike the Append tool, the Insert Selected Features or Rows tool does not support multiple inputs and does not support customized field mapping.

 

An edit session should not be active if the data is versionsed, since arcpy cannot connect to an active edit session.  The tool will internally start and stop an edit session when versioned data is involved.  If data in not versioned an edit session may be active, but is not necessary.  When an edit session is active for unversioned data, if the edit session is stopped before saving the inserts then the inserts will disappear without warning and will not be saved.

 

The tool lets you specify the number of copies of each feature or row you want to insert into the target layer or table view.  The minimum number of copies is 1, and 1 is the default value.

 

NOTE:  If the table view of the target is open when the tool is run, then the Append tool and my tool are affected by an Esri bug.  The records are inserted, but the total record count of the table view does not update and the newly inserted records cannot be accessed or seen when Show all records view is active.  However, since the inserted records are selected by my tool, after the tool completes the selected records count of the table view is updated and the new records do appear when the Show selected records view is active.  To fully refresh the table view to show the new records in the Show all records view mode either the table view has to be closed and reopened or a field has to have its values sorted.

 

The attached python toolbox includes the code for the tool and the tool help files.  I recommend that you save the attached toolbox in your "%APPDATA%\ESRI\Desktop10.2\ArcToolbox\My Toolboxes" directory. The toolbox also Includes my Multiple Field Key to Single Field Key tool for 10.2.  For more on that tool see my blog here.

 

The code for the tool is shown below:

 

# Author: Richard Fairhurst
# Date:   July 2015


import arcpy

class Toolbox(object):
    def __init__(self):
        """Define the toolbox (the name of the toolbox is the name of the
        .pyt file)."""
        self.label = "Field Match Tools"
        self.alias = ""

        # List of tool classes associated with this toolbox
        self.tools = [InsertSelectedFeaturesOrRows]

class InsertSelectedFeaturesOrRows(object):
    def __init__(self):
        """Define the tool (tool name is the name of the class)."""
        self.label = "Insert Selected Features or Rows"
        self.description = ""
        self.canRunInBackground = False


    def getParameterInfo(self):
        """Define parameter definitions"""
        # First parameter
        param0 = arcpy.Parameter(
            displayName="Source Layer or Table View",
            name="source_layer_or_table_view",
            datatype="GPTableView",
            parameterType="Required",
            direction="Input")


        # Second parameter
        param1 = arcpy.Parameter(
            displayName="Target Layer or Table View",
            name="target_layer_or_table_view",
            datatype="GPTableView",
            parameterType="Required",
            direction="Input")


        # Third parameter
        param2 = arcpy.Parameter(
            displayName="Number of Copies to Insert",
            name="number_of_copies_to_insert",
            datatype="GPLong",
            parameterType="Required",
            direction="Input")


        param2.value = 1        


        # Fourth parameter
        param3 = arcpy.Parameter(
            displayName="Derived Layer or Table View",
            name="derived_table",
            datatype="GPTableView",
            parameterType="Derived",
            direction="Output")


        param3.parameterDependencies = [param1.name]
        param3.schema.clone = True


        params = [param0, param1, param2, param3]


        return params


    def isLicensed(self):
        """Set whether tool is licensed to execute."""
        return True


    def updateParameters(self, parameters):
        """Modify the values and properties of parameters before internal
        validation is performed.  This method is called whenever a parameter
        has been changed."""
        return


    def updateMessages(self, parameters):
        """Modify the messages created by internal validation for each tool
        parameter.  This method is called after internal validation."""
        if parameters[1].value:
            insertFC = parameters[1].value
            strInsertFC = str(insertFC)
            if parameters[0].value and '<geoprocessing Layer object' in strInsertFC:
                FC = parameters[0].value
                strFC = str(FC)
                if not '<geoprocessing Layer object' in strFC:
                    print("Input FC must be a layer if output is a layer")
                    parameters[0].setErrorMessage("Input must be a feature layer if the Output is a feature layer!")
                else:
                    dscFCLyr = arcpy.Describe(FC)
                    dscinsertFCLyr = arcpy.Describe(insertFC)
                    # add the SHAPE@ field if the shapetypes match
                    if dscFCLyr.featureclass.shapetype != dscinsertFCLyr.featureclass.shapetype:
                        print("Input and Output have different geometry types!  Geometry must match!")
                        parameters[0].setErrorMessage("Input and Output do not have the same geometry")
                    
                    if dscFCLyr.featureclass.spatialReference.name != dscinsertFCLyr.featureclass.spatialReference.name:
                        print("Input and Output have different Spatial References!  Spatial References must match!")
                        parameters[0].setErrorMessage("Input and Output do not have the same Spatial References!  Spatial References must match!")
        if parameters[2].value <= 0:
            parameters[2].setErrorMessage("The Number of Row Copies must be 1 or greater")
        return


    def execute(self, parameters, messages):
        """The source code of the tool."""
        try:
            mxd = arcpy.mapping.MapDocument(r"CURRENT")
            df = arcpy.mapping.ListDataFrames(mxd)[0]


            FC = parameters[0].value
            insertFC = parameters[1].value


            strFC = str(FC)
            strInsertFC = str(insertFC)


            FCLyr = None
            insertFCLyr = None


            for lyr in arcpy.mapping.ListLayers(mxd, "", df):
                # Try to match to Layer
                if '<geoprocessing Layer object' in strFC:
                    if lyr.name.upper() == FC.name.upper():
                        FCLyr = lyr
                if '<geoprocessing Layer object' in strInsertFC:
                    if lyr.name.upper() == insertFC.name.upper():
                        insertFCLyr = lyr
            if FCLyr == None or insertFCLyr == None:
                # Try to match to table if no layer found
                if FCLyr == None:
                    tables = arcpy.mapping.ListTableViews(mxd, "", df)
                    for table in tables:
                        if table.name.upper() == strFC.upper():
                            FCLyr = table
                            break
                if insertFCLyr == None:
                    tables = arcpy.mapping.ListTableViews(mxd, "", df)
                    for table in tables:
                        if table.name.upper() == strInsertFC.upper():
                            insertFCLyr = table
                            break


            # If both layers/tables are found then process fields and insert cursor
            if FCLyr != None and insertFCLyr != None:
                dsc = arcpy.Describe(FCLyr)         
                       
                selection_set = dsc.FIDSet


                # only process layers/tables if there is a selection in the FCLyr
                if len(selection_set) > 0:
                    print("{} has {} {}{} selected".format(FCLyr.name, len(selection_set.split(';')), 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row', '' if len(selection_set.split(';')) == 1 else 's'))
                    arcpy.AddMessage("{} has {} {}{} selected".format(FCLyr.name, len(selection_set.split(';')), 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row', '' if len(selection_set.split(';')) == 1 else 's'))
                    
                    FCfields = arcpy.ListFields(FCLyr)
                    insertFCfields = arcpy.ListFields(insertFCLyr)


                    # Create a field list of fields you want to manipulate and not just copy    
                    # All of these fields must be in the insertFC    
                    manualFields =  []
                    matchedFields = []
                    for manualField in manualFields:
                        matchedFields.append(manualField.upper())
                    for FCfield in FCfields:
                        for insertFCfield in insertFCfields:
                            if (FCfield.name.upper() == insertFCfield.name.upper() and
                                FCfield.type == insertFCfield.type and
                                FCfield.type <> 'Geometry' and
                                insertFCfield.editable == True and
                                not (FCfield.name.upper() in matchedFields)):    


                                matchedFields.append(FCfield.name)    
                                break
                            elif (FCfield.type == 'Geometry' and
                                  FCfield.type == insertFCfield.type):


                                matchedFields.append("SHAPE@")
                                break
                            elif insertFCfield.type == "OID":
                                oid_name = insertFCfield.name
                   
                    if len(matchedFields) > 0:
                        # Print the matched fields list
                        print("The matched fields are: {}".format(matchedFields))
                        arcpy.AddMessage("The matched fields are: {}".format(matchedFields))


                        copies = parameters[2].value
                        print("Making {} {} of each {}".format(copies, 'copy' if copies == 1 else 'copies', 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row'))
                        arcpy.AddMessage("Making {} {} of each {}".format(copies, 'copy' if copies == 1 else 'copies', 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row'))


                        oid_list = []
                        # arcpy.AddMessage(oid_name)
                        dscInsert = arcpy.Describe(insertFCLyr)
                        if '<geoprocessing Layer object' in strInsertFC:
                            oid_name = arcpy.AddFieldDelimiters(dscInsert.dataElement, oid_name)
                        else:
                            oid_name = arcpy.AddFieldDelimiters(dscInsert, oid_name)
                        rowInserter = arcpy.da.InsertCursor(insertFCLyr, matchedFields)
                        print("The output workspace is {}".format(insertFCLyr.workspacePath))
                        arcpy.AddMessage("The output workspace is {}".format(insertFCLyr.workspacePath))
                        if '<geoprocessing Layer object' in strInsertFC:
                            versioned = dscInsert.featureclass.isVersioned
                        else:
                            versioned = dscInsert.table.isVersioned
                        
                        if versioned:
                            print("The output workspace is versioned")
                            arcpy.AddMessage("The output workspace is versioned")
                            with arcpy.da.Editor(insertFCLyr.workspacePath) as edit:
                                with arcpy.da.SearchCursor(FCLyr, matchedFields) as rows:       
                                    for row in rows:       
                                        for i in range(copies):
                                            oid_list.append(rowInserter.insertRow(row))
                        else:
                            print("The output workspace is not versioned")
                            arcpy.AddMessage("The output workspace is not versioned")
                            with arcpy.da.SearchCursor(FCLyr, matchedFields) as rows:       
                                for row in rows:       
                                    for i in range(copies):
                                        oid_list.append(rowInserter.insertRow(row))
                        del row       
                        del rows       
                        del rowInserter  
                        if len(oid_list) == 1:
                            whereclause = oid_name + ' = ' + str(oid_list[0])
                        elif len(oid_list) > 1:
                            whereclause = oid_name + ' IN (' + ','.join(map(str, oid_list)) + ')'
                        if len(oid_list) > 0:
                            # arcpy.AddMessage(whereclause)
                            # Switch feature selection
                            arcpy.SelectLayerByAttribute_management(FCLyr, "CLEAR_SELECTION", "")
                            arcpy.SelectLayerByAttribute_management(insertFCLyr, "NEW_SELECTION", whereclause)
                            print("Successfully inserted {} {}{} into {}".format(len(oid_list), 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row', '' if len(selection_set.split(';')) == 1 else 's', insertFCLyr.name))
                            arcpy.AddMessage("Successfully inserted {} {}{} into {}".format(len(oid_list), 'feature' if '<geoprocessing Layer object' in strInsertFC else 'table row', '' if len(selection_set.split(';')) == 1 else 's', insertFCLyr.name))
                    else:
                        print("Input and Output have no matching fields")
                        arcpy.AddMessage("Input and Output have no matching fields")
                else:
                    print("There are no features selected")
                    arcpy.AddMessage("There are no features selected")


                     
            # report if a layer/table cannot be found
            if FCLyr == None:
                print("There is no layer or table named '{}' in the map".format(FC))
                arcpy.AddMessage("There is no layer or table named '" + FC + "'")
            if insertFCLyr == None:
                print("There is no layer or table named '{}' in the map".format(insertFC))
                arcpy.AddMessage("There is no layer or table named '{}' in the map".format(insertFC))


            arcpy.RefreshActiveView()                             
            return
        except Exception as e:     
            # If an error occurred, print line number and error message     
            import traceback, sys     
            tb = sys.exc_info()[2]     
            print("Line %i" % tb.tb_lineno)
            arcpy.AddMessage("Line %i" % tb.tb_lineno)
            print(e.message)
            arcpy.AddMessage(e.message)