topic Re: laharz in Python Questions

laharz

charlesmoser — Wed, 21 Mar 2018 21:23:11 GMT

Wondering if somebody can help me with some errors generated by the LaHarz model. I am trying to run the Laharz Distal Zones standalone python tool. It was created by a USGS developer and it is supposed to work with ArcGIS 10.X. There are four required inputs: 1) filled DEM, 2) text file of volumes (integers), separated by commas 3) a text file of XY coordinates and 4) a string denoting a prefix for output grids. Below is the message in Geoprocessing Results. The full python script will be attached.

Executing: laharzdistalzones.py C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example fill mtrwest_ C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\laharz_textfiles\volumesrockfall.txt C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\point_textfiles_for_ArcMap\points.txt Rock_Avalanche
Start Time: Wed Mar 21 17:19:08 2018
Running script laharzdistalzones.py...
Parsing user inputs:
Running Laharz_py
_________ Input Values _________
Input Surface Raster Is:fill
Volume textfile :C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\laharz_textfiles\volumesrockfall.txt
Starting coordinates file :C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\point_textfiles_for_ArcMap\points.txt
Drainage identifier :mtrwest_
Rock Avalanche Selected
_________ Paths on Disk _________
full path fill :C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\fill
full path dir :C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\dir
full path flac :C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\flac
full path str :C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\str
_________ Creating DEM Array _________
_________ Get NumPyArray Dimensions _________
Shape is: (986, 1251) (rows, colums)
Number of rows is: 986
Number of columns is: 1251
_________ Set Window Boundaries _________
wXmin (TOP): 0
wXmax (BOTTOM): 985
wYmin (LEFT): 0
wYmax (RIGHT): 1250
_________ Convert Textfiles to Arrays _________
Volume List is: [1000.0, 3000.0, 10000.0, 300000.0]
Points entered: [[587737.0, 5204869.0], [579305.0, 5196801.0], [577930.0, 5191581.0]]
Cross Section Area List is: [20.0, 42.0, 93.0, 896.0]
Planimetric Area List is: [2000.0, 4160.0, 9283.0, 89628.0]
_________ Creating startpts_g _________
Failed script laharzdistalzones.py...

Traceback (most recent call last):
File "D:\Programs\ArcGIS\Desktop10.4\ArcToolbox\Toolboxes\laharz_py_example\laharz_py\distal_inundation.py", line 1621, in <module>
main(argv[1], argv[2], argv[3], argv[4], argv[5], argv[6])
File "D:\Programs\ArcGIS\Desktop10.4\ArcToolbox\Toolboxes\laharz_py_example\laharz_py\distal_inundation.py", line 1169, in main
tmpStartPoints = ExtractByPoints(Input_surface_raster,startCoordsList,"INSIDE")
File "d:\programs\arcgis\desktop10.4\arcpy\arcpy\sa\Functions.py", line 1409, in ExtractByPoints
extraction_area)
File "d:\programs\arcgis\desktop10.4\arcpy\arcpy\sa\Utils.py", line 53, in swapper
result = wrapper(*args, **kwargs)
File "d:\programs\arcgis\desktop10.4\arcpy\arcpy\sa\Functions.py", line 1405, in Wrapper
return _wrapToolRaster(u"ExtractByPoints_sa", unicode(result.getOutput(0)))
AttributeError: 'unicode' object has no attribute 'getOutput'

Failed to execute (laharzdistalzones.py).
Failed ta Wed Mar 21 17:19:09 2018 (Elapsed Time: 1.06 seconds)

Re: laharz

DanPatterson_Retired — Wed, 21 Mar 2018 21:37:06 GMT

just a caution, but file paths like

C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\laharz_textfiles\volumesrockfall.txt

not raw encoded in scripts ( ie r' .... name .... '

r'C:\Users\cmoser\Downloads\laharz_py_example\mtrain_example\laharz_textfiles\volumesrockfall.txt'

will produce Unicode errors in python 3.

Which leads me to begin to think that you might want to change the source and destination locations of files being used by the script.

Re: laharz

charlesmoser — Wed, 21 Mar 2018 22:31:00 GMT

Thank you for the comment. After reading your comment, I went ahead and moved the workspace folder to the root directory of my D drive, like in the documentation. Then, I hardcoded the path, using r'path to address the issue you mentioned. Unfortunately, I am still getting the error. The error seems to be associated with the ExtractByPoints geoprocessing tool. I know this can fail if the points in the points text file are too far from streams in the streams grid. However, I just checked that and the points are right on top of the streams.

Re: laharz

DanPatterson_Retired — Wed, 21 Mar 2018 22:49:39 GMT

same coordinate system of course?

Re: laharz

charlesmoser — Thu, 22 Mar 2018 01:54:53 GMT

Yes. Same coordinate system. Actually, the data that I am using is USGS' sample data set that comes with the toolbox.

Re: laharz

DanPatterson_Retired — Thu, 22 Mar 2018 02:55:42 GMT

You say it comes in toolbox form, but you are running the script in standalone mode. The first thing to check would be if the tool works in the toolbox first. If it does, then use the toolbox, since there may be script/tool validation going on behind the scenes. Usually when a toolbox is provided there is reasons beyond the 'nice' look and that may be accounting for some of the issues.

So I would give that a try first Charles before proceeding with trying to narrow down why the script itself isn't working (and that Unicode thing still bugs me for some reason)

Re: laharz

charlesmoser — Thu, 22 Mar 2018 04:53:35 GMT

Sorry. I was a bit sloppy with language. I mean stand-alone as in not in field calculator or model-builder. The python script I was running is actually a python tool in a toolbox. I think you are on to something with the Unicode business. Again, I think it is the ExtractByPoints processing tool that is failing. I noticed one of the parameters is the coordinate list. So I printed out the coordinate list. Notice the printed out coordinates in bold. Disregard "startpoint.". Each coordinate pair is followed by NaN NaN. Are those hidden characters that need to be stripped out?

Volume List is: [1000.0, 3000.0, 10000.0, 300000.0]
Points entered: [[587737.0, 5204869.0], [579305.0, 5196801.0], [577930.0, 5191581.0]]
Cross Section Area List is: [20.0, 42.0, 93.0, 896.0]
Planimetric Area List is: [2000.0, 4160.0, 9283.0, 89628.0]
start point. 587737 5204869 NaN NaN
start point. 579305 5196801 NaN NaN
start point. 577930 5191581 NaN NaN
Length of list: 3
_________ Creating startpts_g _________
Failed script laharzdistalzones.py...

Traceback (most recent call last):
File "D:\laharz_py_example\laharz_py\distal_inundation.py", line 1625, in <module>
main(argv[1], argv[2], argv[3], argv[4], argv[5], argv[6])
File "D:\laharz_py_example\laharz_py\distal_inundation.py", line 1173, in main
tmpStartPoints = ExtractByPoints(Input_surface_raster,startCoordsList,"INSIDE")
File "d:\programs\arcgis\desktop10.4\arcpy\arcpy\sa\Functions.py", line 1409, in ExtractByPoints
extraction_area)
File "d:\programs\arcgis\desktop10.4\arcpy\arcpy\sa\Utils.py", line 53, in swapper
result = wrapper(*args, **kwargs)
File "d:\programs\arcgis\desktop10.4\arcpy\arcpy\sa\Functions.py", line 1405, in Wrapper
return _wrapToolRaster(u"ExtractByPoints_sa", unicode(result.getOutput(0)))
AttributeError: 'unicode' object has no attribute 'getOutput'

Failed to execute (laharzdistalzones.py).

Re: laharz

DanPatterson_Retired — Thu, 22 Mar 2018 09:36:18 GMT

NaN means not a number. Every point is represented by X, Y, Z and M... since Z and M aren't included in the point geometry, they are assigned NaN.

I am not sure if your inputs are to be in that form. I would have expected a list as shown in your 'Points entered' line above.ie just the X,Y values in the form shown

Re: laharz

XanderBakker — Thu, 22 Mar 2018 11:40:58 GMT

What is the extent of your filled DEM? Are the points located inside the extent of the DEM?

Re: laharz

charlesmoser — Thu, 22 Mar 2018 12:22:08 GMT

The points do fall within the extent of the DEM. Again, the data is actually straight from the creator of the tool as a test set. I visually inspected the coordinates, and I turned the text file into a shapefile just to verify that the points do intersect with the stream network.

I think what I will do is comment out the code that creates the coordinates list from a text file and hard code the coordinates to see what happens. I will post the results.

Incidentally, if anybody is interested, the URL for the reference and a link to the zip file with the tools is here:

https://pubs.er.usgs.gov/publication/ofr20141073

Re: laharz

charlesmoser — Fri, 23 Mar 2018 01:52:48 GMT

Eureka! I found it! the problem really was the call to the ExtractByPoints method. I finally decided to paste the ExtractByPoints tool into ModelBuilder, ran my data, then exported the model to Python. I discovered that the call to ExtractByPoints is way different than in the USGS code:

Here is the USGS Code:

tmpStartPoints = ExtractByPoints(Input_surface_raster,"'587737 5204869';'579305 5196801';'577930 5191581'","INSIDE")

Here is the code that works from ModelBuilder:

tmpStartPoints = arcpy.gp.ExtractByPoints_sa(Input_surface_raster, "'587737 5204869';'579305 5196801';'577930 5191581'", "INSIDE")

I have to add, Dan Patterson, who responded earlier, asked about import statements. To be fair to the original author of the code, he did include the import statement, from arcpy.sa import *. So why does the code not work for me unless I spell out the full call to the method? I have no idea if it's a Python version issue. I am using Python 2.7. I should get my coworker to try the tool and see whether or not he gets the same error, using the original code. Maybe there is something screwy with my environment. At any rate, I am very happy that the tool works for me now.

Thanks to everybody who responded to this post!

Re: laharz

DanPatterson_Retired — Fri, 23 Mar 2018 02:01:19 GMT

by chance did their script do a

from arcpy.sa import *

Re: laharz

charlesmoser — Fri, 23 Mar 2018 02:11:04 GMT

Here are the import statements in the original code:

import sys, string, os, arcpy, math, time

from arcpy import env

from arcpy.sa import *

from math import *

And the code checks out the Spatial Analyst extension. I wonder why I had to explicitly state the full object to get the tool to work.

I have tried it back and forth several times and believe me the tool works like a charm when I state the full object and method, arcpy.gp.ExtractByPoints_sa.

I have Python 2.7. I don’t know if that has anything to do with it.

Re: laharz

DanPatterson_Retired — Fri, 23 Mar 2018 03:57:53 GMT

This ... from arcpy.sa import * ...

should never be done in my opinion, even though the help documentation shows it as correct.

importing as you did is best, but I am surprised that it failed if you used their import method.

It might be for that version of arcpy with python 2.7