Exporting/extracting HEC-RAS data using Python

[ShaunCavey]

I work with hundreds of HEC-RAS models, and typically I would open each one up to copy/paste a user-defined output table to a spreadsheet.  Then use some Visual Basic to manipulate to a way I want it - so time consuming!  Well, I've been making small programs with Python for the last few months and I want to experiment with HEC-RAS and see what I can figure out.  So my question is how do I go about extracting cross section data from a HEC-RAS model?

Is there a HEC-RAS python module I don't know about?  Do I have to 'hack' into the HEC-RAS program or .SDF export file?

I've always thought how incredible it would be to automate an export process, and I feel like there's got to be a way.  Any help or suggestions to lead me the right way is greatly appreciated.

[ShaunCavey]

Update: 
Had sort of an epiphany the other day and noticed that if you changed the prj, geometry, flow, or plan files to a text file then they are perfectly legible to read in as input cards.  Never really thought it about like this, but I guess that's how the HEC-RAS program works.  Probably just reads in these input cards, and searches for keywords to store the appropriate input values in the right variables.

Reading input lines from a text file and storing info in different variables when keywords are flagged isn't difficult.  But the shear amount of different keywords and variables is crazy I feel.  With this is in mind, I emailed the ACOE to see if I could get their source code for reading the input files only.  No response yet.  It also crossed my mind that maybe I could file a request under the Freedom of Information Act.  I don't think I'd be able to get the "brains" of the HEC-RAS program, but I feel receiving code that reads the input files and stores it into its separate variables would be game.

[saurabhkulkarni]

Try the code below, it should give you a general idea of how you can approach this.

You'd need to point the "folderLocation" variable in the code below to the folder path and the code will spit out a csv file with details extracted from Hec-RAS geometry files.
https://gist.github.com/anonymous/df899701271a62ff4543#file-gistfile1-py

########################################################################
import os
import re
import time
import csv
from operator import floordiv


folderLocation = "x:\folderlocation\Models"
listSubFolders = os.listdir(folderLocation)
flowChangeLocationsTable = []
streamGeometry = {}
for subFolder in listSubFolders:
    subFolderPath = folderLocation + "\\" + subFolder
    if os.path.isdir(subFolderPath):
        listSubFolderFiles = os.listdir(subFolderPath)
        for file in listSubFolderFiles:
            #Geometry files in hecras have a ".g0*" extension, the code below filters to look that geometry files only
            if file.find(".g") != -1:
                openFile = open(subFolderPath + "\\" + file, "r")
                readFile= openFile.read()
                pattern1 = re.compile(r"Type RM Length L Ch R =.+\n")
                pattern2 = re.compile("(?<=\n#Mann=).+(?![A-z]+)")
                pattern3 = re.compile(r"(?<=Type RM Length L Ch R =).+(\d+)")
                manningsCoeff = pattern2.finditer(readFile)
                stations = pattern3.finditer(readFile)

                stationsList = []
                for station in stations:
                    if int(station.group().split(",")[0].strip()) == 1:
                        stationsList.append(station.group().split())
                manningsCoeffList = []
                for coeffs in manningsCoeff:
                    numberOfIter = floordiv(int(coeffs.group().split(",")[0])-1, 3) + 1
                    newStartLocation = coeffs.start() + readFile[coeffs.start():].find("\n") + 1
                    TempList = []
                    while numberOfIter > 0:
                        line = readFile[newStartLocation:newStartLocation + readFile[newStartLocation:].find("\n")]
                        newStartLocation = newStartLocation + readFile[newStartLocation:].find("\n") + 1 
                        numberOfIter = numberOfIter - 1
                        for item in line.split():
                            TempList.append(item)
                    manningsCoeffList.append(TempList)
                streamGeometryData = map(list.__add__,stationsList,manningsCoeffList)
                streamGeometry[file] = streamGeometryData

outputCheckMannings = csv.writer(open(folderLocation + "\\" +'streamGeometry'+ str(time.gmtime().tm_sec) + '.csv', 'wb'))

list = []
for key,value in streamGeometry.items():
    temp = []
    
    for i in range(len(value)):
        temp1 = []
        temp1.append(key)
        for item in value:
            temp1.append(item)
        temp.append(temp1)
        
    for items in temp:
        finalRow = []
        for item in items:
            for i in item.split(","):
                finalRow.append(i)
        outputCheckMannings.writerow(finalRow)

========================================================================================

Update: 
Had sort of an epiphany the other day and noticed that if you changed the prj, geometry, flow, or plan files to a text file then they are perfectly legible to read in as input cards.  Never really thought it about like this, but I guess that's how the HEC-RAS program works.  Probably just reads in these input cards, and searches for keywords to store the appropriate input values in the right variables.

Reading input lines from a text file and storing info in different variables when keywords are flagged isn't difficult.  But the shear amount of different keywords and variables is crazy I feel.  With this is in mind, I emailed the ACOE to see if I could get their source code for reading the input files only.  No response yet.  It also crossed my mind that maybe I could file a request under the Freedom of Information Act.  I don't think I'd be able to get the "brains" of the HEC-RAS program, but I feel receiving code that reads the input files and stores it into its separate variables would be game.