se_toolkit - Control File Documentation

[bbieb]

Is there any documentation on creating a control files that are used with the se_toolkit?  For example using the control file with the asc2sde tool? ftp://ftp.esri.com/pub/staff/vangelo/se_toolkit/docs/dat/asc2sde.html
Thanks.
brian

[VinceAngelo]

I've had an overhaul of the DAT control file documentation on my "to do" list for a while now,
but the time to accomplish it has been sparse. There is a lot of information in the old Forums
(forums.esri.com), with details on specific applications, but samples/geonames.ctl and the
text provided in the existing asc2de, ascinfo, and sdeupdate documentation are all that is
provided with an install (besides the source tree, of course).

What in particular are you trying to accomplish?

- V

[bbieb]

I had the live feeds running on my sde (http://www.arcgis.com/home/item.html?id=9ff56fbe3c4f4b749fd6066424685280) this winter and when the NWS changed their us.cap file, the Weather Warning/Watches feed broke.  I am finally getting around to fixing it and there needs to be quite a bit more parsing of string fields to get the data out of the csv file.  What I really need is the string functions that can be utilized in the control file.  Hopefully I can figure it out from there.
SUBSTR - have an example, but what else is there?

[VinceAngelo]

I added a CSVMODE table tag to simplify CSV parsing, though I didn't spend a whole lot
of time trying to break it.

You'll probably want to look a the SubToken class (in dat_string.c), which breaks strings
with arbitrary delimiters (e.g., '1|2|a:b:c|3').

I'm doing release QA for setk10.0 at the moment, and hope to have it published soon.

- V

[bbieb]

Hi Vince,
Thanks for your prompt replies.  I've been looking at your code but haven't been able to decipher how to split or trim the following:
Severe Thunderstorm Warning issued August 08 at 4:59PM EDT expiring August 08 at 5:30PM EDT by NWS Tallahassee http://www.srh.noaa.gov/tlh/

I need to parse out "Severe Thunderstorm Warning", "August 08 at 4:59PM EDT" and "August 08 at 5:30PM EDT".  Tried using SubToken but if I use "issued" for the delimiter, it uses "i", "s", etc for delimiters.

[VinceAngelo]

The SubToken delimiter is character oriented -- "issued" becomes "i" or "s" or "u" or "e" or "d".

Do you have 'sed' available?

 
echo "Severe Thunderstorm Warning issued August 08 at 4:59PM EDT expiring August 08 at 5:30PM EDT by NWS Tallahassee http://www.srh.noaa.gov/tlh/" | \
sed 's/ issued /|/' | \
sed 's/ expiring /|/' | \
sed 's/ by /|/' | \
sed 's/ http/|http/'
 
 

- V

[VinceAngelo]

I'm working on a solution to address this parsing issue inside se_toolkit -- I'm adding a
"StrPos" class:

 
 StrPos(strcol,"string"[,{START | END | BEFORE | AFTER}])
StrIPos(strcol,"string"[,{START | END | BEFORE | AFTER}])

With it you could use [this would be more legible with a fixed-width font]:

 
event         String                            - 200 N
$stop1        StrPos(event," issued ",BEFORE)   -   4 N
$start2       StrPos(event," issued ",AFTER)    -   4 N
$stop2        StrPos(event," expiring ",BEFORE) -   4 N
$start3       StrPos(event," expiring ",AFTER)  -   4 N
$stop3        StrPos(event," by ",BEFORE)       -   4 N
event_class   SubStr(event,1,$top1)             -  80 N
event_issued  SubStr(event,$start2,$stop2)      -  80 N
event_expires SubStr(event,$start3,$stop3)      -  80 N

Use the "Contact Author" page from ArcScripts to send me your contact info, ArcSDE release,
and platform, and I'll get you a "setk93b41" snapshot with the StrPos class (setk100 will include
this at general release).

- V

[VinceAngelo]

Ack.  The above won't work, since SubStr is {col,start,length}, but this should:

 
event         String                            - 200 N
$stop1        StrPos(event," issued ",BEFORE)   -   4 N
$start2       StrPos(event," issued ",AFTER)    -   4 N
$sub1         Substr(event,$start2)             - 200 N    
$stop2        StrPos($sub1," expiring ",BEFORE) -   4 N
$start3       StrPos($sub1," expiring ",AFTER)  -   4 N
$sub2         Substr($sub1,$start3)             - 200 N  
$stop3        StrPos($sub2," by ",BEFORE)       -   4 N
event_class   SubStr(event,1,$stop1)            -  80 N
event_issued  SubStr($sub1,1,$stop2)            -  80 N
event_expires SubStr($sub2,1,$stop3)            -  80 N

- V

[VinceAngelo]

The StrPos and SubStr classes have been updated to support the above usage, and a
full setk93b41 release is now available at the FTP site.  The next release of 'se_toolkit'
will be setk100.

- V

[bbieb]

Thanks Vince, it works as advertised!
brian