Help using the land_cover_classification_using_unet jupyter notebook sample
I am really struggling to understand how to use building polygons to create an output that will work with the Export Training Data for Deep Learning tool. I am making an assumption that I need to create classified tiles for use with the Classify Pixels Deep Learning Model, but that assumption could be wrong. I cannot figure out how to make it create classified tiles that make any sense to me. When I run the Polygon to Raster tool with just the building footprint shapes it creates a raster with no background. When I use that raster with the Export Training Data tool it says it worked but no image chips are created.
(Aerial with Polygon to Raster output on top. No training image chips are created when these two raster are used as inputs to the Export Training Data for Deep Learning tool)
I tried running the Segment Mean Shift tool. It creates a raster with an added background surrounding the buildings that seems to work with the Export tool. However, that raster seems to treat the buildings as the no data values and the background as the raster feature. When I run the Export Training Data tool it only creates classified tile chips of the portions of my aerial that contain no buildings. It doesn't seem like that output will do me any good for classifying pixels as buildings.
(Segment Mean Shift Tool output based on Polygon to Raster input)
(Training Image Chips are created by the Export Training Data for Deep Learning Tool when I use the Segment Mean Shift raster as the classified raster, but none of the chips have buildings in them. They only cover portions of the aerial that had no buildings at all within the chip. The Output No Feature Tiles option made no difference)
I have no idea what I should expect from these tools, because the documentation and examples are no help. I have tried at least 50 workflow variants to try to get classified tile chips that show buildings and nothing has worked. Outputs have ranged from the Export tool creating nothing, to creating only chips without buildings, to errors stating that the raster is incompatible with the Export Training Data tool. The Segment Mean Shift tool output shown above is the best result I have had so far.
My trial and error definitely seems to just be error at this point. I am sure Esri support will be useless for this task, since at this point it is clear to me that they know less than I do. Sandeep I really, really need someone to provide more details about your building footprints workflow than what your blog says
Sandeep:
Your blog makes it seem like it is a simple task to create a classified raster from building footprint polygons that the Export Training Data tool will accept.
Extracting building footprints
We used the existing building footprints as training data to train another deep learning model for extracting building footprints. This time, the model that we had to train was a pixel classification model, wherein each pixel belonging to a building needed to be labelled as such.
Pixel classification for extracting buildings
Exporting training data
ArcGIS Pro has support for exporting training samples in the Classified Tiles format for training such models. All we had to do was convert the existing building footprint layer to a classified raster and feed it into the tool.
So it is clear that I need to create a classified raster and use the Classified Tiles output from the Export Training Data tool to Extract Building Footprints. Your blog makes it sound like you and your team think it should be obvious to everyone what steps they need to follow to create a classified raster from a building footprint layer, but I assure you that it is not. There is nothing obvious about it and I have wasted days searching the ArcGIS Pro and ArcMap help documents for anything that creates an identical output that works with the Export Training Data tool. All of my outputs that resemble yours are rejected by the tool as not being Thematic. Is there any user out there who has a clue what actually needs to be done to complete this step?
Sandeep:
I tried to follow your instructions. I am using ArcGIS PRo 2.4.1. First I used the Identity tool to create a background polygon surrounding my building footprint polygons so that I could classify the entire feature extent. I assigned a classValue of 1 to the building footprint polygons and a classValue of 2 to the background polygon. I used the Polygon to Raster tool to output a raster of the buildings and background. I changed the raster format from Generic to Thematic in the output and the NODATA value from 3 to 0, but noticed the output was a 2-bit pixel-depth output, not an 8-bit unsigned pixel depth output. So I used the Copy Raster tool to make a copy that was 8-bit unsigned, and verified that the layer format was Thematic and the NODATA value was 0. I verified that both my aerial image and my label raster have identical Extents and Spatial References and that all of the pixels are assigned a value in the label raster.
Here is a picture of how the aerial image and the label image appear in the map Table of Contents legend of ArcGIS Pro 2.4.1:
Here is a picture of the label raster and the raster properties of the label raster in Catalog:
Here is a picture of the aerial and its Catalog properties:
Here are the Export Training Data for Deep Learning tool settings:
The output of the tool has no image chips with buildings, only image chips without any buildings:
Here are the label tiles:
The output is identical when I check the Output No Feature Tiles option on the Export tool. The output is also identical if I copy the aerial and label rasters into the ArcGIS Pro project File Geodatabase and use those rasters as inputs for the tool.
Why is the Export tool failing to output any image chips with buildings in them? Is this output what I should expect? What do the output tiles look like for the rasters shown in your notebook?
I tried again to follow your notebook example more exactly. I used ObjectID rather than the classValue field as input to the Polygon To Raster tool:
I then used the Reclassify tool to assign all buildings to 1 and NODATA to 0 (I had deleted the background feature so my ObjectIDs started at 2 rather than 1).
I ran the Export Data tool based on the settings you showed.
I changed the layer to Thematic.
The Raster and its properties look like this:
When I ran the Export Training Data using that label raster I get the same set of tiles that have no buildings in them like I did in the previous post:
If I change the NODATA property of the label raster to 0 in Catalog then the background disappears when I bring the raster into ArcGIS Pro, but when I use that as my input the Export Training Data tool doesn't output any classified tiles at all.
So is this the best output I am going to get with the Export Training Data tool and will these tiles work if I tried to do the other steps in the notebook?
I have started investigating developing my own tool.to replace the Export Training Data for Deep Learning. Example 2 in the NumpyArrayToRaster help is very close to what I want to do and it works well for my classified raster. In fact I can modify the code to only output chips that contain a code of 1 for buildings to get only the set of chips that contain a building, which is the exact opposite output from the one I am getting from the Esri tool and what I really want. The performance is also great and can extract 900 tiles with 500 x 500 pixels from a 15,000 x 15,000 pixel classified raster in under 2 minutes. I am also confident I can create the json, emd, map.txt and stats.txt files that the Esri tool also creates using the numpy array to generate the necessary statistics.
import arcpy
import numpy
filein = r"C:\Users\rfairhur\Documents\Jupyter Notebooks\arcgis-python-api-master\Imagery\Riverside\Reclass_BUILDING_LABEL.tif"
fileout = r"C:\Users\rfairhur\Documents\Jupyter Notebooks\arcgis-python-api-master\Imagery\Riverside\output\labels\BUILDING_AREA_1080.tif"
blocksize = 500
arcpy.env.outputCoordinateSystem = filein
arcpy.env.cellSize = filein
myRaster = arcpy.Raster(filein)
blockno = 0
for x in range(0, myRaster.width, blocksize):
for y in range(0, myRaster.height, blocksize):
# Lower left coordinate of block (in map units)
mx = myRaster.extent.XMin + x * myRaster.meanCellWidth
my = myRaster.extent.YMin + y * myRaster.meanCellHeight
# Upper right coordinate of block (in cells)
lx = min([x + blocksize, myRaster.width])
ly = min([y + blocksize, myRaster.height])
# noting that (x, y) is the lower left coordinate (in cells)
# Extract data block
myData = arcpy.RasterToNumPyArray(myRaster, arcpy.Point(mx, my),
lx-x, ly-y)
if 1 in myData:
# Convert data block back to raster
myRasterBlock = arcpy.NumPyArrayToRaster(myData, arcpy.Point(mx, my),
myRaster.meanCellWidth,
myRaster.meanCellHeight)
# Save on disk as 'filename_#.ext'
filetemp = ('_%i.' % blockno).join(fileout.rsplit('.',1))
myRasterBlock.save(filetemp)
# Maintain a list of saved temporary files
blockno += 1
print("Made " + str(blockno) + " images")
# Release raster objects from memory
del myRasterBlock
del myRaster
print("done")The output of the classified raster chip is shown below overlaying the original aerial using 30% transparency and it looks great (the raster matches the original building footprint polygon outlines):
I would like to use the same code with my aerials, since the performance is great. However, the same code does not work well for processing my aerial image. Even though the code runs and is very fast, the output of the NumpyArrayToRaster looks bad. Even though it is in the correct location and has the correct cell size, it looks nothing like a photo anymore. I made sure the input and output format were both tif, but the output looks bad as shown below where the output chip is overlaying the original aerial at 20% transparency:
The output raster retains just a suggestion of outlines of the features contained in the original aerial that are aligned correctly with the original aerial, but clearly this output is no longer formatted as a aerial. I could use the Clip Raster tool as an alternative to do this step in the process and that would allow me to create a tool that could replace the Esri tool, but I believe the performance of that tool would definitely suffer relative to the performance of the Esri tool.
Dan, do you have any ideas on processes that would quickly clip aerials to generate a set of aerial image chip files and their world files at a minimum? Are there any alternative python modules I should look into, or is there some way to use RasterToNumpyArray and NumpyArrayToRaster to input and output an aerial that ends up with a usable aerial image? I didn't see any Google search results or discussions on GeoNet or stackExchange about using these numpy methods with aerials or satellite images without transforming them. Any ideas on alternate topics I could search? Thanks..
I went ahead and tried using the Clip tool in my script and the performance actually pretty good. To create a set of 900 image tiles and 900 label tiles with 500 x 500 pixels per tile from two original 15,000 x 15,000 pixel rasters took 5 minutes and 11 seconds. That is a rate of about 5.79 tiles per second. That seems acceptable. If I limit the output to only tiles that contain buildings that can speed up the process even more. For example, for the sample image I have been using I would only need 582 image tiles and 582 label tiles to cover all of the tiles that contain buildings, which is only about 64.7% of all of the possible images. I can generate that set of tiles in 3 minutes and 16 seconds (a rate of about 5.94 tiles per second).
So, here is the code I currently have that will generate the set of image and label tiles that contain buildings from my sample rasters:
from time import strftime
print( "Start script: " + strftime("%Y-%m-%d %H:%M:%S"))
import arcpy
import numpy
filein = r"C:\Users\rfairhur\Documents\Jupyter Notebooks\arcgis-python-api-master\Imagery\Riverside\Reclass_BUILDING_LABEL.tif"
imagein = r"C:\Users\rfairhur\Documents\Jupyter Notebooks\arcgis-python-api-master\Imagery\Riverside\BUILDING_AREA_1080.tif"
fileout = r"C:\Users\rfairhur\Documents\Jupyter Notebooks\arcgis-python-api-master\Imagery\Riverside\output\labels\BUILDING_AREA_1080.tif"
imageout = r"C:\Users\rfairhur\Documents\Jupyter Notebooks\arcgis-python-api-master\Imagery\Riverside\output\images\BUILDING_AREA_1080.tif"
blocksize = 500
arcpy.env.outputCoordinateSystem = filein
arcpy.env.cellSize = filein
myRaster = arcpy.Raster(filein)
blockno = 0
for x in range(0, myRaster.width, blocksize):
for y in range(0, myRaster.height, blocksize):
# Lower left coordinate of block (in map units)
mx = myRaster.extent.XMin + x * myRaster.meanCellWidth
my = myRaster.extent.YMin + y * myRaster.meanCellHeight
# Upper right coordinate of block (in cells)
lx = min([x + blocksize, myRaster.width])
ly = min([y + blocksize, myRaster.height])
# noting that (x, y) is the lower left coordinate (in cells)
# Extract data block
myData = arcpy.RasterToNumPyArray(myRaster, arcpy.Point(mx, my),
lx-x, ly-y)
if 1 in myData:
# Convert data block back to raster
myRasterBlock = arcpy.NumPyArrayToRaster(myData, arcpy.Point(mx, my),
myRaster.meanCellWidth,
myRaster.meanCellHeight)
# Save on disk temporarily as 'filename_#.ext'
filetemp = ('_%i.' % blockno).join(fileout.rsplit('.',1))
imagetemp = ('_%i.' % blockno).join(imageout.rsplit('.',1))
myRasterBlock.save(filetemp)
myextent = '{} {} {} {}'.format(mx, my, mx + blocksize * myRaster.meanCellWidth, my + blocksize * myRaster.meanCellHeight)
arcpy.Clip_management(imagein, myextent, imagetemp)
# Maintain a list of saved temporary files
blockno += 1
print("Made " + str(blockno) + " images")
# Release raster objects from memory
del myRasterBlock
del myRaster
print( "Finished script: " + strftime("%Y-%m-%d %H:%M:%S"))
All that remains is to write code for tracking the statistics I need to generate the json, emd, map.txt and stats.txt files and make revisions that would replace all of the hard-coded values with parameters that could be set at run time.
Below is a comparison of the tiles that the Esri Export Training Data for Deep Learning tool was producing vs. the tiles my tool is producing:
The Esri Export Training Data for Deep Learning Tool output. None of the buildings in the original image are contained in any of the tiles. Also, the colors in the tiles have changed slightly compared to the original image.
.
My version of the Export Training Data for Deep Learning Tool output. The image colors match the original and all buildings or portions of buildings in the original image are covered by a tile.
