Latest Contributions by VinceAngelo

Ack.  The above won't work, since SubStr is {col,start,length}, but this should: event         String                            - 200 N $stop1        StrPos(event," issued ",BEFORE)   -   4 N $start2       StrPos(event," issued ",AFTER)    -   4 N $sub1         Substr(event,$start2)             - 200 N    $stop2        StrPos($sub1," expiring ",BEFORE) -   4 N $start3       StrPos($sub1," expiring ",AFTER)  -   4 N $sub2         Substr($sub1,$start3)             - 200 N  $stop3        StrPos($sub2," by ",BEFORE)       -   4 N event_class   SubStr(event,1,$stop1)            -  80 N event_issued  SubStr($sub1,1,$stop2)            -  80 N event_expires SubStr($sub2,1,$stop3)            -  80 N - V ... View more

I'm working on a solution to address this parsing issue inside se_toolkit -- I'm adding a "StrPos" class: StrPos(strcol,"string"[,{START | END | BEFORE | AFTER}]) StrIPos(strcol,"string"[,{START | END | BEFORE | AFTER}]) With it you could use [this would be more legible with a fixed-width font]: event         String                            - 200 N $stop1        StrPos(event," issued ",BEFORE)   -   4 N $start2       StrPos(event," issued ",AFTER)    -   4 N $stop2        StrPos(event," expiring ",BEFORE) -   4 N $start3       StrPos(event," expiring ",AFTER)  -   4 N $stop3        StrPos(event," by ",BEFORE)       -   4 N event_class   SubStr(event,1,$top1)             -  80 N event_issued  SubStr(event,$start2,$stop2)      -  80 N event_expires SubStr(event,$start3,$stop3)      -  80 N Use the "Contact Author" page from ArcScripts to send me your contact info, ArcSDE release, and platform, and I'll get you a "setk93b41" snapshot with the StrPos class (setk100 will include this at general release). - V ... View more

The SubToken delimiter is character oriented -- "issued" becomes "i" or "s" or "u" or "e" or "d". Do you have 'sed' available? echo "Severe Thunderstorm Warning issued August 08 at 4:59PM EDT expiring August 08 at 5:30PM EDT by NWS Tallahassee http://www.srh.noaa.gov/tlh/" | \ sed 's/ issued /|/' | \ sed 's/ expiring /|/' | \ sed 's/ by /|/' | \ sed 's/ http/|http/' - V ... View more

Just to close up this thread, I did some experimentation with some custom tools to measure access performance of a random-generated dataset... First I generated a 100k row shapefile with roughly the same size .shp as .dbf (50.8Mb and 49.1Mb, respectively). Then I used ArcGIS 10 to populate it in a file geodatabase (66.7Mb). Next I used a 'C' app to time how long it took to read the files (pure I/O, without any semantic parsing). On my reference machine (a 4-CPU/4Gb RAM laptop running 64-bit Windows 7) [and after an initial read pass to cache the I/O], it took 220, 188, and 265 milliseconds to access the .shp, .dbf, and a0000000a.gdbtable files (respectively). Each time represents ~1 micro- second per 512-byte block (kicking the blocksize up to 256k with setvbuf dropped all access times to under 100ms). I then wrote a custom app with the File Geodatabase API, and applied the millisecond timer to break down the processing times on a FGDB query into three components -- Open (OpenGeodatabase + Geodatabase.OpenTable + table.GetFieldInformation + various FieldInfo calls), Read (table.Search + EnumRows.Next), and Get (row.Get*), plus a total: FGDBa 0 1229 1673 2902 Parsing just the geometry data (without a change to the selection list) was much faster: FGDBg 0 1170 0 1170 As was limiting the selection list to just "SHAPE" (though, curiously, not as fast as a "*" column list and just calling GetGeometry): FGDBs 16 1122 48 1186 I then instrumented a shapefile reader with the same millisecond timer and compared the access performance using five different access methodologies: A) Bind query on all columns B) Bind query on *only* the .shp component (simulating an empty .dbf) C) Using getter by column number on all columns D) Using getter by column name on all columns E) Using a custom getStream function Timing is in milliseconds, with values Open (includes column describe), Read, Get, and Total -- 10 cols: shpA 0 2387 0 2387 shpB 0 1622 0 1622 shpC 0 2372 280 2652 shpD 16 2480 484 2980 shpE 0 2449 125 2574 To determine the impact of column count I reorganized the dBase attributes into more columns of shorter length (to preserve the overall size), and then re-ran the measurements with 20, 50 and 100 columns using the FGDB and shape methodologies above: 20 cols: FGDBa 0 1240 4096 5336 FGDBg 16 1310 94 1420 FGDBs 16 1404 0 1420 shpA 0 2480 0 2480 shpB 0 1638 0 1638 shpC 0 2434 405 2839 shpD 0 2355 1202 3557 shpE 0 2464 188 2652 50 cols: FGDBa 15 2200 17628 19843 FGDBg 0 2090 31 2121 FGDBs 16 2105 16 2137 shpA 0 2574 0 2574 shpB 0 1638 0 1638 shpC 0 2399 736 3135 shpD 0 2577 6003 8580 shpE 0 2620 219 2839 100 cols: FGDBa 0 3819 61669 65488 FGDBg 0 3370 46 3416 FGDBs 16 3417 16 3494 shpA 0 2652 0 2652 shpB 0 1638 0 1638 shpC 0 2840 1014 3854 shpD 0 2844 24846 27690 shpE 0 2620 485 3105 Since the time progression on both FGBDa and shpD suggest a "big O N-squared" algorithm issue (doubling the columns increases duration by roughly four times), I re-wrote my shapefile findColByName function to use a circular search algorithm (each search starts with the column after the last found position), and then re-tested (methodology F): 10 cols: shpF 0 2299 447 2746 20 cols: shpF 0 2465 468 2933 50 cols: shpF 0 2780 808 3588 100 cols: shpF 0 2574 2028 4602 In conclusion, it appears that the getter calls on attributes with the FILEGDB API do have a significant performance cost (this is especially true when there are a large number of attributes in the table). Fortunately for your case, you have the option of recoding to delay the use of 'Get' attribute accessors until they are actually necessary. I've recommended that a circular search algorithm be used in the getter and setter functions, and that consideration be given to overloading the Row accessors using an integer radix (vice name) which should give uniform access cost without regard to the attribute retrieval order. - V ... View more

I added a CSVMODE table tag to simplify CSV parsing, though I didn't spend a whole lot of time trying to break it. You'll probably want to look a the SubToken class (in dat_string.c), which breaks strings with arbitrary delimiters (e.g., '1|2|a:b:c|3'). I'm doing release QA for setk10.0 at the moment, and hope to have it published soon. - V ... View more

I've had an overhaul of the DAT control file documentation on my "to do" list for a while now, but the time to accomplish it has been sparse. There is a lot of information in the old Forums (forums.esri.com), with details on specific applications, but samples/geonames.ctl and the text provided in the existing asc2de, ascinfo, and sdeupdate documentation are all that is provided with an install (besides the source tree, of course). What in particular are you trying to accomplish? - V ... View more

What product are you using? Is the dataset in a database? Which one? As a rule, time fields are difficult to process as-is, especially when you get to the "within two hours of local dawn" kinds of queries. - V ... View more

Which RDBMS?  Are you using OS authentication?  Are the boxes on the same domain? - V ... View more

I had a situation where an external process was doing updates on a million row table, but only a few dozen were actually changed (plus a handful were deleted and a score added).  I couldn't use a trigger, so I created a parallel table containing the primary key (a string with a GUID in it), and a perfect hash (aka 'digest') of the dozens of active columns.  Then I could create a second hash table with the "revised" contents, compare the hashes to generate lists of added, deleted, and changed rows, and then update the "current" hash table based on the needed action.  This was very successful to generate a low-bandwidth change feed to keep an external database in sync; you could use something like this to track deltas between subsequent passes. - V ... View more

I was explaining why 9.3.1 could not use 11g to Direct Connect to 9.2.  I'm surprised that 10.0 could, since there was adequate reason why it shouldn't, but 10.1 doesn't because 9.2 has aged out. - V ... View more

Oracle 11g was released after ArcSDE 9.2.   11g support was introduced at ArcSDE 9.3, so 11g server DLLs for ArcSDE 9.2 were never created. - V ... View more

The caret (^) is a DOS line continuation character. The Unix shell continuation is a backslash (\). So to port your script you'd need to do a global replace. The following would work: cat foo.bat | sed 's:\^$:\\:' > foo.sh [Caret is regexp special character, as is backslash, so they need to be escaped; the dollar sign matches end-of-line, so this won't alter non-terminal carets the way "tr '^' '\\'" would.] As always with Unix, there's many ways to do this -- You could also use 'vi' remove all the terminal carets (:1,$s/\^$//) and concatenate the lines with auto-repeat of "J". - V ... View more

While neither ArcSDE 9.1 or Oracle 9i are supported by their respective vendors, the mechanism for changing a lost SDE password remains the same: 1) Stop the application server service (usually 'esri_sde') 2) Use the administrative login to connect to Oracle ("sqlplus / as sysdba") 3) Change the SDE user password ("alter user SDE identified by newPassw0rd") 4) Replace the existing service: a - Use 'sdeservice -o list' to obtain the service properties b - use 'sdeservice -o delete' to remove the existing service c - Use 'sdeservice -o create' to recreate the service with the new administrative password 5) Start the new application server service 6) Write down the new password and place it in a locked filing cabinet Changing the passwords for non-SDE users and creating new users are covered completely by the database documentation and mentioned in the Esri documentation. - V ... View more

You contact Tech Support through the mechanism prescribed by your license agreement.  For most US sites, this is through the toll-free number; for international sites, through your local distributor. Most organizations with site licenses (e.g. universities) have a central tier-1/contact person who in turn makes the call if it's out of their league.  Evaluation installs are sometimes supported by the local marketing office. - V ... View more

Starting over "from scratch" involves first dropping every object previously created during the original 'sdesetup'. At this point you should probably start an incident with Tech Support. - V ... View more