tpcolson

On-Demand GPS Provisioning Widget Thingy: Step 1

Blog Post created by tpcolson Champion on May 19, 2016

Not sure if this rises to the lofty level of qualifying as a "Custom Widget", but I did change one line of js in the GP Widget thanks to Roberts help, so I'll pat myself on the back for my first Widget customization.

 

In the Natural Resource field, GIS'ers spend most of the year dealing with data collected during the "Summer Field Season", then spend the summer helping folks figure out how to make maps and use their GPS. There's never really a good workflow (or time) to synchronize last winters QA'ed GIS data with what field folks need to do their jobs today(now please!). To mitigate demands on time (I spend all my time in the virtual server closet these days) and the need for a large workforce to have the latest and greatest data to put on their GPS so they can get to last years sample sites, AND, not require a large workforce to take extensive training to use finicky mobile GIS software (not a big fan of Collector-yet)....I've been struggling with an "Easy Button" way for, say, a college-junior intern to come into the office in the morning, push a button, and get all of the latest Sasquatch Observation Points, including yesterdays, onto their Garmin (the cheapest we can buy) GPS. Here's how I did it.

 

I had originally developed a runs-every-night Python doo-hicky using Kevin Hibmas Features to GPS Toolbox by hard-coding paths, field name mappings, and dumping a GPX file onto a file server so staff could get the latest GPS data and dump it on their GPS. Problem is, I can only do this for a few projects, and IT not be thrilled with me doing this for the dozens of projects that GIS supports. Why not allow people to go to a web map with a button in it that allowed them to download the GPX file "on demand"?

 

Step 1

 

First we need to pull a feature class out of an SDE database and pre-convert some attribute field names so they "fit nicely" with the convert-to-GPX tool, so we can see some expected text when looking at these points on the Garmin. So using the ArcPy Field Mappings object we can pull that off, at the same time flipping the result into a shapefile. Finally, we use Kevin Hibmas Features to GPS Toolbox FeaturesToGPX Python code to handle the conversion to GPX format.

 

A couple of things to note in the following script. In order for the server-version of this tool to fire the output to the virtual scratch directory, you'll need

OutName = arcpy.GetParameterAsText(0)

somewhere at the top and then

outGPX = os.path.join(arcpy.env.scratchFolder, OutName)

as your last definition. Finally, note how we're handling the "middle-put" shapefile. We want it deleted after the GPX conversion, so the next person that runs this tool will always get the latest data. Also note that I'm using a Data Store-registered folder for the location of the SDE connection file-just my personal preference.

try:
    from xml.etree import cElementTree as ET
except:
    from xml.etree import ElementTree as ET
import arcpy, sys, traceback
from arcpy import env
import time, os
from subprocess import call
unicode = str


OutName = arcpy.GetParameterAsText(0)


    ''' We want to have some fields from the database appear on the Garmin Wypt screen
    '''
fms = arcpy.FieldMappings()
fms.addTable("\\\\gisserver\DATASTORE\GP\\BIGFOOT SASQUATCH.sde\\SASQUATCH.DBO.SASQUATCH_POINTS")
FLD_LOC_NAME = arcpy.FieldMap()
FLD_TRAIL = arcpy.FieldMap()


    ''' Convert location name to "name" for Garmin
    '''
FLD_LOC_NAME.addInputField("\\\\gisserver\DATASTORE\GP\\BIGFOOT SASQUATCH.sde\\SASQUATCH.DBO.SASQUATCH_POINTS", "LOC_NAME")  
LOC_NAME_MAP = FLD_LOC_NAME.outputField  
LOC_NAME_MAP.name = "Name"  
FLD_LOC_NAME.outputField = LOC_NAME_MAP  
fms.addFieldMap(FLD_LOC_NAME)
    ''' Convert trail name to "desc" for Garmin
    '''
FLD_TRAIL.addInputField("\\\\gisserver\DATASTORE\GP\\BIGFOOT SASQUATCH.sde\\SASQUATCH.DBO.SASQUATCH_POINTS", "TRAIL")  
TRAIL_MAP = FLD_TRAIL.outputField  
TRAIL_MAP.name = "Descript"  
FLD_TRAIL.outputField = TRAIL_MAP  
fms.addFieldMap(FLD_TRAIL)




    ''' Export to a shp with the field mappings
    '''
arcpy.FeatureClassToFeatureClass_conversion("\\\\gisserver\DATASTORE\GP\\BIGFOOT SASQUATCH.sde\\SASQUATCH.DBO.SASQUATCH_POINTS",\
                                            arcpy.env.scratchFolder, "SASQUATCH_POINTS.shp", "", fms)
    ''' convert lidar elevaton to meters
    '''
arcpy.CalculateField_management(os.path.join(arcpy.env.scratchFolder,u'SASQUATCH_POINTS.shp'), "ELEVATION", '!ELEVATION!*0.3048', "PYTHON_9.3")


gpx = ET.Element("gpx", xmlns="http://www.topografix.com/GPX/1/1",
                 xalan="http://xml.apache.org/xalan",
                 xsi="http://www.w3.org/2001/XMLSchema-instance",
                 creator="Esri",
                 version="1.1")




def prettify(elem):
    """Return a pretty-printed XML string for the Element.
    """
    from xml.dom import minidom
    rough_string = ET.tostring(elem, 'utf-8')
    reparsed = minidom.parseString(rough_string)
    return reparsed.toprettyxml(indent="  ")






def featuresToGPX(inputFC, outGPX, zerodate, pretty):
    ''' This is called by the __main__ if run from a tool or at the command line
    '''


    descInput = arcpy.Describe(inputFC)
    if descInput.spatialReference.factoryCode != 4326:
        arcpy.AddWarning("Input data is not projected in WGS84,"
                         " features were reprojected on the fly to create the GPX.")


    generatePointsFromFeatures(inputFC , descInput, zerodate)


    # Write the output GPX file
    try:
        if pretty:
            gpxFile = open(outGPX, "w")
            gpxFile.write(prettify(gpx))
        else:
            gpxFile = open(outGPX, "wb")
            ET.ElementTree(gpx).write(gpxFile, encoding="UTF-8", xml_declaration=True)
    except TypeError as e:
        arcpy.AddError("Error serializing GPX into the file.")
    finally:
        gpxFile.close()






def generatePointsFromFeatures(inputFC, descInput, zerodate=False):


    def attHelper(row):
        # helper function to get/set field attributes for output gpx file


        pnt = row[1].getPart()
        valuesDict["PNTX"] = str(pnt.X)
        valuesDict["PNTY"] = str(pnt.Y)


        Z = pnt.Z if descInput.hasZ else None
        if Z or ("ELEVATION" in cursorFields):
            valuesDict["ELEVATION"] = str(Z) if Z else str(row[fieldNameDict["ELEVATION"]])
        else:
            valuesDict["ELEVATION"] = str(0)


        valuesDict["NAME"] = row[fieldNameDict["NAME"]] if "NAME" in fields else " "
        valuesDict["DESCRIPT"] = row[fieldNameDict["DESCRIPT"]] if "DESCRIPT" in fields else " "




        if "DATETIMES" in fields:
            row_time = row[fieldNameDict["DATETIMES"]]
            formatted_time = row_time if row_time else " "
        elif zerodate and "DATETIMES" not in fields:
            formatted_time = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(0))
        else:
            formatted_time = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(0)) if zerodate else " "


        valuesDict["DATETIMES"] = formatted_time


        return
    #-------------end helper function-----------------




    def getValuesFromFC(inputFC, cursorFields ):


        previousPartNum = 0
        startTrack = True


        # Loop through all features and parts
        with arcpy.da.SearchCursor(inputFC, cursorFields, spatial_reference="4326", explode_to_points=True) as searchCur:
            for row in searchCur:
                if descInput.shapeType == "Polyline":
                    for part in row:
                        newPart = False
                        if not row[0] == previousPartNum or startTrack is True:
                            startTrack = False
                            newPart = True
                        previousPartNum = row[0]


                        attHelper(row)
                        yield "trk", newPart


                elif descInput.shapeType == "Multipoint" or descInput.shapeType == "Point":
                    # check to see if data was original GPX with "Type" of "TRKPT" or "WPT"
                    trkType = row[fieldNameDict["TYPE"]].upper() if "TYPE" in fields else None


                    attHelper(row)


                    if trkType == "TRKPT":
                        newPart = False
                        if previousPartNum == 0:
                            newPart = True
                            previousPartNum = 1


                        yield "trk", newPart


                    else:
                        yield "wpt", None


    # ---------end get values function-------------




    # Get list of available fields
    fields = [f.name.upper() for f in arcpy.ListFields(inputFC)]
    valuesDict = {"ELEVATION": 0, "NAME": "", "DESCRIPT": "", "DATETIMES": "", "TYPE": "", "PNTX": 0, "PNTY": 0}
    fieldNameDict = {"ELEVATION": 0, "NAME": 1, "DESCRIPT": 2, "DATETIMES": 3, "TYPE": 4, "PNTX": 5, "PNTY": 6}


    cursorFields = ["OID@", "SHAPE@"]


    for key, item in valuesDict.items():
        if key in fields:
            fieldNameDict[key] = len(cursorFields)  # assign current index
            cursorFields.append(key)   # build up list of fields for cursor
        else:
            fieldNameDict[key] = None


    for index, gpxValues in enumerate(getValuesFromFC(inputFC, cursorFields)):


        if gpxValues[0] == "wpt":
            wpt = ET.SubElement(gpx, 'wpt', {'lon':valuesDict["PNTX"], 'lat':valuesDict["PNTY"]})
            wptEle = ET.SubElement(wpt, "ele")
            wptEle.text = valuesDict["ELEVATION"]
            wptTime = ET.SubElement(wpt, "time")
            wptTime.text = valuesDict["DATETIMES"]
            wptName = ET.SubElement(wpt, "name")
            wptName.text = valuesDict["NAME"]
            wptDesc = ET.SubElement(wpt, "desc")
            wptDesc.text = valuesDict["DESCRIPT"]
    ''' force a wypt symbol
    '''            
            wptSym = ET.SubElement(wpt, "sym")
            wptSym.text = "Animal Tracks"             


        else:  #TRKS
            if gpxValues[1]:
                # Elements for the start of a new track
                trk = ET.SubElement(gpx, "trk")
                trkName = ET.SubElement(trk, "name")
                trkName.text = valuesDict["NAME"]
                trkDesc = ET.SubElement(trk, "desc")
                trkDesc.text = valuesDict["DESCRIPT"]
                trkSeg = ET.SubElement(trk, "trkseg")


            trkPt = ET.SubElement(trkSeg, "trkpt", {'lon':valuesDict["PNTX"], 'lat':valuesDict["PNTY"]})
            trkPtEle = ET.SubElement(trkPt, "ele")
            trkPtEle.text = valuesDict["ELEVATION"]
            trkPtTime = ET.SubElement(trkPt, "time")
            trkPtTime.text = valuesDict["DATETIMES"]






if __name__ == "__main__":
    ''' Gather tool inputs and pass them to featuresToGPX
    '''


    inputFC = os.path.join(arcpy.env.scratchFolder,u'SASQUATCH_POINTS.shp')
    outGPX = os.path.join(arcpy.env.scratchFolder, OutName)
    ''' Need for this to open in Base Camp
    '''    
    zerodate = "0"
    pretty = "#"
    featuresToGPX(inputFC, outGPX, zerodate, pretty)
    ''' Delete the shp when it's all over
    '''    
    arcpy.Delete_management(os.path.join(arcpy.env.scratchFolder,u'SASQUATCH_POINTS.shp'))

Finally, since my data natively has an elevation field, there is no need to do a field mapping for it.

On-Demand GPS Provisioning Widget Thingy: Step 1

On-Demand GPS Provisioning Widget Thingy: Step 2

On-Demand GPS Provisioning Widget Thingy: Step 3

On-Demand GPS Provisioning Widget Thingy: Step 4

Outcomes