Select to view content in your preferred language

Numpy Snippets # 3 ... Phish_Nyet ... creating sampling grids using numpy and arcpy

5213
11
09-09-2016 08:46 AM
Labels (1)
DanPatterson_Retired
MVP Emeritus
4 11 5,213

Numpy Snippets

Updated: 2016-09-09

The purpose of this post is to show how numpy can play nicely with arcpy to produce geometry and perform tasks like shape translation, rotation and scaling which are not readily available in arcpy's available functions.

To pay homage to ArcMap's ... Fish Net ... I propose a very scaled down version aptly named Phish_Nyet.  A fuller version will appear when ArcScript 2.0 opens.  This is a demo script.  All you do is vary the parameters in the demo() function within the script.

Sampling_grids.png

Only the rectangular option will be presented here, however, it is possible to make all kinds of sampling grids, including hexagonal ones as shown above.

The following points apply:

  • Output is to a shapefile.
  • The output file must have a projected coordinate system.   To create grids in Geographic coordinates, use Fishnet in ArcMap.  Why?  because invariably people create grids in Geographic coordinates, then project them without densifying the grid blocks.  This results in shapes which are in error because the curvature associated with lines of latitude are not accounted for.
  • A corner point is specified as the origin of the net.  One can determine which corner to use by specifying a dX and dY with positive and/or negative values.  In this fashion, you can get your corner to be the top or bottom, left or right of the output.  If you want the middle to be the origin...do the math and get a corner.
  • The output is controlled by the cell widths (dX and dY), the number of columns and rows an (i.e.  the X and Y directions) and a rotation angle, which is positive for clockwise rotation.

Notes:

  - grid_array - does the numpy magic of generating the grid shapes.   I have tried to be as verbose as possible.  In short, I generate a

    seed shape and propagate it.  I have intentionally kept rotate and  output_polygons as visible function so you can see how they work.

A fuller version will surface as I stated when ArcScripts 2.0 appears.  Change the parameters in the demo() function and run it.

"""
Phish_Nyet.py
Author:  Dan.Patterson@carleton.ca

Purpose: Produce a sampling grid with user defined parameters.
"""
import arcpy
import numpy as np

def demo():
    """Generate the grid using the following parameter"""
    output_shp = r'C:\temp\Phish_Nyet.shp'
    SR =  arcpy.SpatialReference(2951) # u'NAD_1983_CSRS_MTM_9' YOU NEED ONE!!!
    corner = [340000.0, 5022000.0]     # corner of grid 
    dX = 1000.0;  dY = 1000.0          # X and Y cell widths
    cols = 3;  rows = 3                # columns/rows...grids in X and Y direction
    angle = 0                          # rotation angle, clockwise +ve
    # create the grid 
    pnts = grid_array(corner,dX,dY,cols,rows,angle)
    output_polygons(output_shp,SR,pnts)
    print('\nPhish_Nyet has created... {}'.format(output_shp))

def grid_array(corner=[0,0],dX=1,dY=1,cols=1,rows=1,angle=0):
    """create the array of pnts to pass on to arcpy using numpy magic"""
    X = [0.0,0.0,1.0,1.0,0.0]                  # X,Y values for a unit square
    Y = [0.0,1.0,1.0,0.0,0.0]                  # 
    seed = np.column_stack((X,Y)) * [dX,dY]    # first array corner values scaled
    u = [seed + [j*dX,i*dY] for i in range(0,rows) for j in range(0,cols)]
    pnts = np.array(u)                         # 
    x = [rotate(p,angle) for p in pnts]        # rotate the scaled points 
    pnts = [ p + corner for p in x]            # translate them
    return pnts

def rotate(pnts,angle=0):
    """rotate points about the origin in degrees, (+ve for clockwise) """
    angle = np.deg2rad(angle)               # convert to radians
    s = np.sin(angle);  c = np.cos(angle)   # rotation terms
    aff_matrix = np.array([[c, -s],[s, c]]) # rotation matrix
    XY_r = np.dot(pnts, aff_matrix)         # numpy magic to rotate pnts
    return XY_r

def output_polygons(output_shp,SR,pnts):
    """produce the output polygon shapefile"""
    msg = '\nRead the script header... A projected coordinate system required'
    assert (SR != None) and (SR.type=='Projected'), msg
    polygons = []
    for pnt in pnts:                           # create the polygon geometry
        polygons.append(arcpy.Polygon(arcpy.Array([arcpy.Point(*xy) for xy in pnt]),SR))
    if arcpy.Exists(output_shp):               # overwrite any existing versions
        arcpy.Delete_management(output_shp)
    arcpy.CopyFeatures_management(polygons, output_shp)

if __name__ == '__main__':
    """Generate the grid using the listed parameters"""
    demo()  # modify the parameters in demo to run
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Of course other regular geometric shapes can be generated in a similar fashion, but not all may pack like rectangles and others do.

11 Comments
JeffWard
Honored Contributor

I've had fun messing around with this.  I have successfully created a grid of hexagons.  Thanks for indirectly teaching me about list comprehensions.

DanPatterson_Retired
MVP Emeritus

​You're welcome Jeff...I am in the process of organizing 'NumPy Repository' group with more stuff in a centralized location for all things pythonic.  Keep your eyes open for it...

DuncanHornby
MVP Notable Contributor

I'm not a mathematician so find the matrices bit a bit bamboozling but this is nifty bit of code and I can see how one could incorporate this into a larger workflow, nice!

BrianWilson
Frequent Contributor

Matrix methods are very efficient because you can roll all the operations into one step: translate, rotate, scale.

If you are doing millions of transforms it will help. If you are only doing a few thousand and it's easier for you to understand, you can just code up the transform one step at a time. I did that to generate a small grid but now that I know about numpy I have to look at it again.

I wrote a function like this to do the transform:

def affine(p, scale, theta, offset):
    """ Scale, rotate and translate point """
    return arcpy.Point(

      (p.X * math.cos(theta) - p.Y * math.sin(theta)) * scale.X + offset.X,
      (p.X * math.sin(theta) + p.Y * math.cos(theta)) * scale.Y + offset.Y)

(from GitHub - Geo-CEG/shoveltest: Create grids to facilitate shovel tests for archaeology site work.)

p is an arcpy.Point to be transformed, scale is a point defining how much to scale the point, theta is the rotation angle in radians, and offset says how far to move the point.

In practice, I use a "for" loop to march through the points in a feature and call this function each time to move a point from one place to another.I do the same calculations (sine and cosine) each time but I am only moving about 100 points so I don't care, it takes longer for the "arcpy" module to load than to run my program.

Dan's code relies on numpy to do the looping-- in the "rotate" function above, he builds a transformation matrix and then hands an array of points and the matrix to the "dot" method, and all the work takes place there. The numpy library is written in 'c' so it is probably more than 1000 times faster than what I did. Getting the work done is often more important than finding the most efficient code.

In Dan's example he only rotates a point with the matrix. He could have added code to build a matrix that would also scale and translate the points at the same time. Hmm I guess that will have to go into my rewrite...

DanPatterson_Retired
MVP Emeritus

In a pure numpy solution, affine has been changed so that the matrix for translation, rotation, scale and skew can be applied to a numpy array of 'points' in one foul swoop which does a polygon all at once rather than by point by point.  This vectorization relies on the vectorization properties.  I should update the code when I get a chance

AbdulMajid
Emerging Contributor

i am trying to make a hex grid with desired number of rows and columns, your post is helpful in generating a shape file in the end. but i cant figure out how to do the hexagon thing. can you let me know how to do that?

DanPatterson_Retired
MVP Emeritus
AbdulMajid
Emerging Contributor

thankyou for your reply, 

i tried using your sampling tool, i want the grid to be starting from a specified lat/lon with specified number of rows and columns. i really tried hard in figuring it out.

DanPatterson_Retired
MVP Emeritus

Projected coordinates only... no support for decimal degree data, use esri's tools instead

AbdulMajid
Emerging Contributor

sorry for bothering you so much, i just wanted to ask from where i can control the size of hex either in side length or area?

arcmap windowcode

DanPatterson_Retired
MVP Emeritus

deltax and deltay

About the Author
Retired Geomatics Instructor at Carleton University. I am a forum MVP and Moderator. Current interests focus on python-based integration in GIS. See... Py... blog, my GeoNet blog...
Labels