I've started a practice that seems to make many errors/crashes go away.Layers/rasters needed for Arc Hydro are:
*I do work all over my County and so have county-wide "master" geodatabase for these that I use on all my Arc Hydro Projects.
If I have these layers, I can recreate the hydrologically correct DEM (HC DEM) anytime I want and re-delineate the watersheds.
So my practice is this: As I'm developing my HC DEM, and have to modify the agreedem, innerwalls, outerwalls layers, I can save the BatchPoints to my project folder and then delete the geodatabase and folder with rasters in it (Target Locations). This starts everything fresh and new. Then I can use model builder to reprocess the watersheds. When I'm ready to re-delineate the watersheds, I copy the BatchPoints into the geodatabase and set the BatchDone value to "0".
I run the watershed delineation, etc. via the Arc Hydro menus though I could possibly do this via Model Builder also. Since I've started doing this (and following solutions in the postings above on this thread), my crashes and errors have all gone away! Here is the ModelBuilder model for my latest project. I don't use outerwalls on this project, but that could be added.
[ATTACH=CONFIG]18585[/ATTACH]
Hey Mark,
Wanted your opinion on something:
I am working on a hydrological modeling project in a relatively mixed agricultural area using a 10m resolution DEM captured via Radar. The study area is topographically diverse, covering both flat agricultural plains and the more rugged, incised valleys.
I am using the ArcHydro DEM Reconditioning tool to burn in a vector watercourse layer. I’m trying to find the right balance for the AGREE algorithm parameters to ensure flow consistency without over-distorting the terrain.
I am currently considering these parameters:
Number of cells for stream buffer: 3
Smooth drop: 5
Sharp drop: 30Any suggestions on how to set these parameters would be really appreciated.Also, are there any documentations on how to set these values for diverse terrain?
Hello Mark and anyone else,
I am having working on a project delineating HUC12 watersheds to HUC14s, and I keep having this error occur that prevents me from creating proper BatchPoints. From what I can tell, the issue is because of the Str and StsLnk layers not forming a closed line (as seen in my images below), this causes more problems because the map acts like the stream ends at that point, rather than continue on to the joining section.
Any ideas for how to approach this? Thanks,
Sam
C,
I'll share the file Target Locations and file structure on one of mine. Maybe a picture will "paint a thousand words".
My project folder is "ArcHydro" and my .mxd is named "ArcHydro", so that could confuse things. But the .gdb is named after the .mxd, not the project folder.
I keep the rawdem raster out in the project folder as well as other layers no part of the Arc Hydro process.
Cheers!
Mark
Hi there!
I posted in an older thread of yours, but then found this one and thought I'd post here as well.
I'm trying to follow your recommended steps.
I'm using ArcHydro with ArcMap 10.8.2.
I moved the NHD_Flowline and DEM file and saved the .mxd file to my C drive.
I'm getting the COM error in the Terrain Pre-processing step under Sink Prescreening (though I previously had success here when using a different DEM file).
I did both parts of step 2.
I'm a little unclear on step 3, so I have stopped here and am attaching some screenshots-- Should the raster file link have something else there?
Any advice would be most appreciated.
Best wishes,
C
You may be aware that ESRI is moving GeoNet to a new platform. To preserve this thread I've exported it to a Word Document. The option to export to a pdf wasn't working and it looks like some paragraph numbering was lost in the simple copy/paste to Word.
Once the new GeoNet is up and running, and if we last this thread, I will post the document somewhere I think it is searchable.
Best,
Mark - Original Arc Hydro Problem Solvers poster.
Thanks, Mark. I got it figured out; those unnaturally straight lines were the culprit. They are underground pipes, and not useful for my purpose. Weirdly, though, before I asked for input, I removed them and tried running the workflow, but I got the same weird behavior. So, I decided to try again; I removed those underground pipes again, saved, closed out of all Esri products, ended all Esri processes still running in Task Manager, and then ran my workflow again (I've a script that runs the ArcHydro tools I need sequentially). Of course, it worked then.
The things you suggested are going in my back pocket, for when something else doesn't work. Especially the "Data Management Terrain Preprocessing." I didn't know that this existed. Since everything is set in our script, I don't have to worry about this. But, often, the script doesn't work (spontaneously--it's great fun when this happens), and I have to run things individually. It may be helpful to set these!
Thanks again.
Annina,
I don't recall seeing that recently.
First: I assume you have filled the sinks under the DEM Manipulation tools.. If not, make sure you do that.
Second: The black lines (DrainageLines) appear to cross each other. Some look like they follow flow paths defined by the DEM (irregular in alignments), others look long and sweeping and like they wouldn't follow a path defined by the DEM. These appear to have have more generalized alignments. Make sure your agreestream layer is correct. I try to limit the agreestream polylines in natural area and only put them in where the DEM misses where water flows under roads or bridges. In developed areas, I put in agreestream polylines where there are pipes.
Third: Make sure you are setting the datasets correctly in the Terrain Processing>Data Management Terrain Processing menu.
Hope this helps.
Mark,
Was wondering if you ever had run into issues with holes in the AdjointCatchment feature class? The Catchment feature class has -1 as the NextDownID, which would be expected on the edge of a watershed, but these are in the middle of the watershed. It's obviously affecting the watershed delineation.
I thought it had something to do with the streams layer including the underground conduit (the straight lines in the attached image), but I removed those and tried running but got the same result.
Any ideas on what to try?
Annina
Purple: Catchments; Yellow: AdjointCatchments; Black lines: DrainageLines
Darin,
My initial thought is to suggest you check and ensure that the geoprocessing extents includes the extent of your data. In older versions the processing extents would change during a prior process and not change back if one of the prior processes didn't execute successfully. In my case the geoprocessing extents would match the patch points extents and cut off watersheds to the extent limits of the batch points. A remedy was to reset the geoprocessing extents.
I have not used the "append coastal catchments" tool. Could be an error in that process that I've not encountered. If you find the processing extents is the issue, you may want to check the processing extents before and after you run append coastal catchments.
I have run into a similar situation as you and gotten watersheds delineated round a small area instead of what I expected. In those cases the point I clicked was not right on the str grid and I'd get a very small watershed delineated as a result. I then would use the flow trace tool and click on the fdr cells all round the point of interested until I was certain I was on a cell that would work. A bit tedious, but it was a work around.
Hope something here helps,
I am new to Arc Hydro, but my colleagues are using it for drainage area delineation and it appears to work well. I have downloaded Arc Map 10.8 and installed the version of Arc Map for 10.8x64. Everything appears to work well for filling sinks, flow direction, flow accumulation, stream definition, stream segmentation, drainage line processing, catchment grid delineation, catchment polygon processing, adjoint catchment processing and the append coastal catchments. Then when I go to point definition, I make sure that my stream snap is changed to the stream link, snap a point. Instead of the program zooming out to the drainage area, it stays focused on the point that I have snapped. If I accept the watershed boundary I get a small line around the point that I have placed. I can look inside the attributes table and it shows the name that I have given the watershed but the length is 0 and the area is 0.
I have been in contact with tech support and no one can figure out what is going on. I have also been in contact with collogues who have also looked through the files and they can generate the drainage area without an issue. I sent the files to tech support and they were able to generate a drainage area. I have uninstalled and reinstalled both Arc Map 10.8 and Arc Hydro and I get the same issue. I am able to generate a watershed boundary within the spatial analyst tools but nothing inside of Arc Hydro.
I am hoping someone on here has had a similar issue or can assist me in correcting this issue.
Thanks
Darin
I can't tell from those error messages what might be going on. The "error message" I'm thinking of is the one that appears while processing. An example is that provided in the image by Anne-Marie Dubois in her post on Aug 8, 2019 7:22 AM. If you can screen shot the run time error messages it might reveal something.
I did a Google search on "arcpy memory error" a few of the results could help your Python programmer. (I suspect done this type of search already):
Hope this helps. I look forward to seeing the solution. I want to get my Python black belt doing Arc Hydro for me soon so your experience will be goo to follow.
Here is a screenshot of the errors we get; thanks for taking a look.
The processing is done on a local machine; however, the NHD dataset is located on a network server. The weird part (well, not too weird, as I know Arc) is that the script has worked, even with the network dataset. I have a feeling there's something in the script that's causing the issue (but crossing my fingers you have an easier solution).
Thanks,
Thanks for the ego boost. I am experienced through the "beating the head against the wall" ArcHydro methods gauntlet, so I guess that makes me an "expert". I am not a Python pro at all. I have gotten accustom to deciphering the ESRI error messages. If you can send the text or a screen shot of the error message, I'll see if that can point to a potential problem.
Are you processing the project on your local machine or over a network? That can make a difference, in my experience.
Best, Mark
Mark Boucher,
My team has been working with ArcHydro tools for a long while, and our team leader, a black belt in Python scripting, created a script that would do what GeoHMS did, so that our engineers wouldn't have to go through all of the Arc Pre-Processing steps individually. Lately we've been getting "MemoryError" issues, even on small basins. The only things that have changed in the script are minor tweaks to make the process faster.
On my computer, the process seems to get stuck at the Compute Basins, Reaches, and Nodes section, but the errors come at other places on other machines.
I've set extents, cleared the temp folder, and turned off background geoprocessing, but still am getting the error. Any ideas on where to go next?
You've been a great help on here, so I thought I'd ask the expert before calling Esri!
I felt that after so much research and frustration, I needed to share my findings regarding an error that kept showing when I used the Sink Evaluation tool (see image)
It turns out that my system decimal symbol was set to "," instead of "." which caused the Field Calculator to interrupt the process and trigger the COM component error. This might be more useful for the French community but anyhow, I think it's good to be put out there.
Hi Mark,
Just closing off this issue. "Evaluate Depressions" hadn't been 'migrated' to Python. It has been now thanks to Christine in ESRI and python version works much faster now even with float raster. See the thread.
Garret
I haven't used that tool. Running the project on the local C: drive does speed processing up quite a bit. If you aren't doing that, you may want to try it.
Hi all,
I am having difficulty running Evaluate Depressions on a 1600x2500 grid in 32-bit integer centimetres. I have also tried 16-bit but it still hangs at the same point (Computing drainage area for depressions...).
It runs fine on floating point grid but it is slow (4 hours) so I was hoping that it would be faster for integer grids.
I would like to know:
If anyone has successfully Evaluated Depressions on an integer grid of such a size?
Has anyone noticed significant performance improvement on integer grids compared to floating point?
Thanks! Garret
Sorry. I haven't had the opportunity to use ArcGIS Pro.
This is an excellent Arc Hydro thread that I've depended on throughout my Arc Hydro experience.
Has anyone had success using Arc Hydro for ArcGIS Pro for stormwater workflows? I'm trying to take into consideration streets and drainage networks for watershed delineation, like I've done in ArcMap 10.5 and 10.6. Unfortunately, I'm encountering a lot of problems including ERROR 999999 and Failed to Execute EucAllocation.
I’ve tried re-organizing my data and troubleshooting different ways as well per other stratgies I've learned with ArcMap but to no avail. Perhaps they still need to work out some bugs or revert back to Target Location folders?
You help's much appreciated!
#archydrotools, #archydro drainage delineation, #archydro tools
Hello!
I have a problem when calculating the Longest Flowpath using HEC-GeoHMS.It turns out that the program starts to compile, stays for more than 20 minutes calculating, after that the program is closed, I open it again and all the progress of the Longest flowpath calculation is lost.
I have tried deleting the temporary files, changing the location of the files and everything said above.I need if someone can help me with this problem please.Thanks
I am using arc hydro 10.1 and there are some solutions to these error which works for me
- define workspace location and processing extent in c drive
- Whenever you have to use arc hydro, always define a geodatabase then start using the tool
- always delete the temp files.
May be this will help.
Hi Siri, I am having the exact same problem. I have been using ArcHydro for years, and am usually able to troubleshoot problems that come up, but this one has me stumped. Just last week I performed some ArcHydro basin analyses that went smoothly. The day before yesterday, the IT folks installed ArcGIS 10.4 on my machine. I then downloaded and installed ArcHydro tools for 10.4, and everything went fine up to creation of catchment polygons. Complete ArcMap crash. A catchment feature class was created, but like with your experience, the HydroID field is all "Null". I started again from scratch and had the same result.
The basins I am working on are generally small. Prior to 10.4, most ArcHydro operations were taking no more than a couple minutes. If you've found some solution to this I'd appreciate hearing about it.
My most recent discovery is more generally related to geoprocessing and is not specific to Arc Hydro. However, since Arc Hydro is heavy on geoprocessing, I wanted to post this here.
My latest Arc Hydro project had ArcMap freezing up when saving and at other times. I'd have to wait forever or, out of impatient frustration, I would "X" out and/or cancel it using Task Manager. Then I heard about something in the geoprocessing options that seems to have put and end to that!
This seems to make saving much, much faster and I haven't had it crash yet.
Hope this tip helps!
My projects are not so large. I don't know if the large size of the tables causes problems. I have heard, and do practice normally, putting data in geodatabases to reduce the instability issues that can come when using simple shape files.
Thanks, Mark. The community is grateful for the service you provide. BTW, I am processing large basins (Ganges, Indus, Brahmaputra, Amu Darya, Syr Darya, separately, using a 90m DEM) so I wonder whether I'm running into issues with the attribute tables being too large. I don't recall running into problems when I process small basins.
I just today had to upgrade to 10.4. I also downloaded the 10.4 Arc Hydro install and am about to install it. If I notice the same problems I'll let you know.
I've never worked with deranged basins, or "create sink structures", so I'm not sure I'd be of any help. Sorry.
p.s. I was running on an HP desktop, 8 cores, 32 GB RAM. Thinking the problems were somehow tied to my installation, I began running ArcGIS 10.4 on a Parallels VM on my macbook. Exact same behavior - continual crashes. Makes me think the problem began with recent versions. I was running hydrotools under 10.1 and 10.2 with no problems
the maddening thing is that when ArcHydro crashes map there is (apparently) no log to diagnose what went wrong. I'm getting the impression it has something to do with writing or modifying the files that it creates. For example, I can get through the processing step all the way to creating the catchment polygons. The processing takes several minutes, then ArcMap crashes ("encounters a serious error"). However, the "catchment" feature layer is in fact created, but it's attribute table doesn't have HydroIDs in it. No amount of coaxing will get hydrotools to execute past this step.
I seem to be digging myself into a deeper hole. Can't finish an archydro project due to the tools throwing errors or ArcMap crashing. I keep starting clean projects in new directories, following your advice, hoping something will magically get resolved, but now merely attempting to set default locations causes ArcMap to crash. Installed 10.4.With last project I got all the way through defining the catchment grid but it exited with error at the polygon processing step. I've cleared my temp folder, have lots of RAM and diskspace. I've successfully been creating catchments for years.
I want to use a shapefile I have for a lake as the only sink in a deranged basin, but the Create Sink Structures routine will not accept the shapefile. Is this because it expects to find HydroIDs in the attribute table?
My practice is to create a tool box and put several models in it. I start the names of the models with numbers so I can keep the order straight. The models may have a dozen steps or just a couple. This is how I've kept one crash from a big model from requiring a whole new start.
I also will sometimes if errors are persistent, I close the mxd and delete all the output (gdbs and all). I keep any of the main inputs. Then I reboot. This is sometimes the only thing that gets me past an error. Never know why this is, but it works.
Hi Dan,
What format is the input raster? Make sure it's an esri GRID format, and make sure it's saved in a folder, not a geodatabase. Also make sure than none of your outputs are set to intermediate, if you're using a model.
Carlisle
Hi,
I've been having an issue with ArcHydro models for some time now, and finally have some time to look into it. The model i'm working with gives me what I want, but it keeps deleting some of the output after it runs, or after the mxd is saved and reopened. This happens to only a few of the outputs, but happens every time usually to the same ones, which are both raster and vector. They are behaving as though they are set to Intermediate, but that is definitely not the case and I also make sure to add them to the display and save the mxd. Fortunately, this does not happen to the ultimate outputs which I need, like fdr (flowdirection) but it makes it very difficult and frustrating to go back and troubleshoot or tweak the model since I lose several steps along the way. I don't have the time to go piece by piece, running the model, and figuring out what happens at every step along the way, so I'm hoping someone has encountered this before or has any thoughts. Below is a screen shot of the model, with the steps numbered in order. The outputs which have a tendency to vanish after the model completes are PreFillDEM, SinkPoly, and fil...where do they go and why?!?
Thanks for your extremely useful thread on troubleshooting ARCHYDRO. I've implemented your suggestions and I'm still getting the same error when I attempt to run the depression evaluation tool. I'm attempting to run this tool on a 1m LIDAR DEM about 650 mb in size, that has already been prescreened for small sinks. The tool will run up to the drainage area calculation step and then crash with the following message:
System.Runtime.InteropServices.COMEXCEPTION (0x80041098): Error 010316: Unable to open the input raster: C:\USERS\DAN\APPDATA]LOCAL\TEMP\ARCFFE6\AH2015~1\G_G2011
Any thoughts on what might be causing this error?
Thanks so much
Dan
I have not used the flow direction with sinks tool. I look forward to seeing a solution to your problem.
Hi Everyone,
I'm stumped and wondering if someone can help explain something. I'm using ArcHydro to do some analysis where there are known sinks, and trying to use the Flow Direction with Sinks tool. In the resulting Fdr grid, I'm used to seeing only the D8 directions (1,2,4,8,16,etc.), but i'm now seeing around 60 different numbers in the VALUE field ranging from 1 to 255. The non-D8 values do not have a high cell COUNT relative to the D8 but they are not insignificant. Is this correct and should I continue with my analysis, or is there something wrong here?
thanks,Carlisle
Come explanations regarding Catchments vs AdjointCatchments and Watersheds vs. Subwatersheds are here: Catchment vs watershed
I just posted something about Target Locations at Re: Batch point generation not working.
As for the storing the raster in a geodatabase: This can be done, but Arc Hydro, be default, keeps the raster data in a separate folder. See the link above regarding target locations.
Sir,
I have similar problems, as you mentioned, with DEM Fill. In fact, I am having all these problems while working on the tutorial data as prescribed by ArcHydro itself so I assume the fault is not in the data. I use Windows 8.1 and ArcMap 10.2.2. I do get a feeling that the error may be due to incorrect target location, which I am unable to locate as it is not mentioned clearly for Win 8.1 ( There is no C:/Users/Temp...). Also, I don't understand what " donot store a raster in a geodatabase" means. Can you please help me with this. I get the same error message as Mr. Mallya has with his problems in DEM Reconditioning.
Thank you.
I just got 10.2.2 installed on a laptop and am testing the new version of Arc Hydro. All worked well until I got to fill sinks. After restarting ArcMap, etc., I reset the processing extents to be that of the rawdem I'm using: an area a little bigger than my study extents. All worked smoothly after that!
So, somewhere in the process, Arc Hydro might change or set the processing extents and if all does not work just right, it doesn't get changed back and that causes problems. If you have problems, see if that fixes it.
It would be nice if Arc Map had a way to show me what the processing extents are or had a way in Model builder to have a layer (like rawdem) be used to set the processing extents for the mxd. Those features would be really nice.
Well... it is tax day...
Seriously, this is one of those "generic" error messages. I seems like you are doing things that reduce this error: local drive, target locations, etc. Make sure you have the Spatial Analyst extension turned on and make sure the layers are all in the same coordinate system.
Even recently I had some problems with the catchment polygon processing (raster to polygon). It took a PC reboot to "fix" it. I'm at a loss to understand the cause. It has been working fine lately.
The Arc Hydro processing is doing a lot. Sometimes I sense that I get an error if it has been a while since I saved or closed and restarted ArcMap. I have never heard anyone talk about this, but I wonder if the "undo" list gets too long for ArcMap to keep track of when doing all the geoprocessing and that causes a memory or other problem.
I will soon be getting v10.2.2 and so I'll be able to experience upgrade errors then. Maybe its a version issue.
Sorry I can't be of more specific help.
Mark: First of all I apologize for re-posting my query from another forum thread. I get the following error (see attached). I have made sure the target locations are set correctly as you have described. My files are on local drive, and I have also gone through most of the useful suggestions you have provided in this post, but with no success.
Can you kindly guide me in solving this error.
System Info:
Windows 7, 64 bit
ArcGIS 10.2, ArcHydro for 10.2
Some who follow Arc Hydro or this thread might have info on bathymetry, but you'll have a better chances of finding an answer if you go to the main GeoNet page and search bathymetry. You might find a broader audience for your question there. This thread is specific to ArcHydro and does not specifically cover data sources.
Here's a good example of what you could find with such a search.
We got the point. The latest support for Bathymetry
Hello Every One
How do I get Bathmetry data
I would like to use it in hydrodynamic modelling.
A DEM Reconditioning problem was solved. Could be a memory issue. See thread at DEM Reconditioning >> Empty AgreeDEM
I don't find the GeoNet forum that easy to use. I'm going to try out Geographic Information Systems Stack Exchange and see if it is a good replacement. I'll still look here from time to time. If GeoStacker works better, I may abandon this format. Just not that good of a format for a forum. Go to this discussion to see what others think.
https://community.esri.com/polls/1059?et=watches.email.poll_comment#comment-6408
Hello, I have been using Arcmap version 10.2.2 and have failed to recondition the DEM using the GEO-HMS steps. I have read the above posts with no luck and have had the following results:
1) The surface reconditions but only over the stream and ignores the surface surrounding it. (checked the negative space to see if i interpreted this wrong but this was not the case)
2) Allows me to recondition all the way to polygon processing however fails to continue from there. (This happened once in 2 days of attempting this)
Is this an arcmap version issue?
EDIT: The error for HEC-Geo HMS can all be fixed with changing the target location to an .mdb rather than a .gdb.
This was mentioned in a post above but it was a solution for 10.1; however works in 10.2.2 as well.
The agree stream layer I use is the storm drain layer that helps define the flow pattern in flat urbanized areas. It also includes “streams” I use to cut through rural road embankments and freeway embankments where there are culverts.
It actually worked!!!
I changed my target locations for vector files from "gdb" to "mdb" (Personal geodatabase). That might have had something to do with it as well as usng GIS 10.1. I am not sure.
I was creating clips of my DEM and stream shape file as well but I still created/saved them in a gdb (not a mdb).
Any thoughts on the matter?! Thank you!
Hello Mr. Boucher,
Thank you for your quick reply. I actually had done that too. Still no luck.
By agreestream layer, you mean the NHLFlowline (USGS)/the stream layer I will use on my DEM layer in AgreeDEM? IF yes, I have that in a GeoDataBase (.gdb) as well. I am trying it with arcGIS 10.1 now (I was using 10.2) I will let you know how it goes.
Thank you
Layla - Have you tried moving your data and project to your local PC hard drive? I don't see that in that on your list of things you've tried. Make sure your folder path is short and simple with no spaces (I used C:\GIS\MoragaPt25 for my last one). Put the agreestream layer in a geodataset. I found that seems to work best. Start a new mxd and add base layers to is. Mine are rawdem, agreestream, innerwalls, outerwalls (only use outerwalls once in a while). By starting a new mxd, the target locations get set automatically. Let us know if you done this, and if not, if when you do it, if it helps.
Hello,
I have already asked about this issue with another set of data before, but I am stuck on DEM Reconditioning again with HRESULT E_FAIL error (returned frmoa call to a COM component)
I have tried everything i could find on this on the forum. I have deleted and created the layers over and over again for the past 2 days. I am all out of ideas. I have used my DEM, my NHD layer and zoomed extent and used the display extent for my "Environment Extend". My background processing is off. My Temp Folder is empty and results are removed. I also set my target locations to a folder (that i made for the project) for raster data and a gdb with the same name (under the raster folder) for the vector data.
I also set the target location for all three of the following: Ap UtilitiesConfig, DefaultConfig and HydroConfig.
Is there anything I am missing?
The name of my msx is short with no number or space too. Same with the name of my folder and gdb.
Your reply is greatly appreciated. Thank you
A problem I have encountered is that some vector files are being processed without necessary fields. The DrainageLine file needs a field labeled "DrainID" for AdjointCatchment to process. If it does not have the field, just add it and set it equal to "GridID."
Signed in members can post, follow updates, and more. New here? Register a free account.
Find useful guides, FAQs, and documents to help you navigate and make the most of Esri Community.
