Dynamic Data Updating

[NeoGeo]

We have numerous automated scripts which update different datasets every few minutes, which are being consumed by a Flex viewer through REST.  I am running into a couple different problems with the viewer updating. 
1)  With vector layers, it seemed like I was having the most trouble with the rest cache, but it does not seem like a good solution to clear the rest cache every 10 minutes because that kills performance and IE will not pull up the map at all usually on the first attempt after the rest cache has been cleared. 

2)  On raster datasets, it is even worse.  Even if I cleared the browser cache, cleared the rest cache, restarted IIS, and restarted the browser, it would not update.  The only thing that would help it update is to restart the service.  People don't seriously consider restarting the service every 10 minutes to be a good solution do they? (programmatically via agssom.exe)  I realize there is only like a 2 second window where it is restarting, but people do notice hiccups like that and it does affect their perception of the viewer quality.

What are you guys doing for your dynamic data? Also has anyone seen any more information on usage of disableClientCaching and doesn't that only help with browser cache?

Thanks

[RaviNarayanan]

Hello,

Can you please provide more information about what is updated in the vector data. For example,
a. are new rows added to existing data?
b. is the map document updated (rendering of layer resource, new layers added etc)?

For a. you will not need to clear the REST cache. the newly added rows will be returned in a query response or in the map generated with exportMap operation.

For b. you will need to clear the REST cache so any updates to the map is reflected in the Map and Layer resources.

For raster data is it a single raster dataset where the entire raster data is being replaced with a newer file?

The disableClientCaching applies only for cached tiles. The tile response header from that service will indicate the browser to not cache the tile.

Ravi

[NeoGeo]

Nothing in the mxd/msd is changed for the vector or raster data.  In both cases the entire underlying data files are being replaced as all of the datasets we are doing this for are time sensitive so the old data (more than 15 min old) is no longer of any use.  The new data is formatted the same and has the same name, same symbology, and same labeling.  If the service is not restarted, the data does not update in the Flex Viewer.  I started using AGSSOM.exe to restart the services each time the data is updated, but now I am having problems with the viewer not coming up on the first two attempts.  I am not sure whether it is related or not at this point.  In my AGS Server logs I get errors that look like this:
ERROR Configuration Hurricane.MapServer is not started. 2011-07-29 09:07:19 Hurricane.MapServer   1013   2504
1013 is the error code btw.


Thanks for your help.

[SrinivasVinnakota]

Neogeo,
Could you please provide more specific information regarding
1. Version of ArcGIS Server.
2. Are you using a MxD or MsD to publish your mapService.
3. Where is your data stored - SDE, FileGDB, Shapefile?


You mentioned that your services use "time-sensitive" data. If you are using ArcGIS Server v10.0 did you enable time on the layers in your map document (in ArcMap)?

As for the error "Configuration Hurricane.MapServer is not started.", this is expected if you are attempting to connect to Hurricane service before it starts. The time to start a map service depends on many factors, including accessing all of its layer data.

[NeoGeo]

My problem at the beginning of the post was that one of the scripts was using the wrong parameter for AGSSOM and the script was not catching the error because it is an external system call to run AGSSOM.  After fixing that, it does update properly, but clearing the rest cache so often still causes the problems mentioned earlier where the map does not come up the first time after the cache has been cleared.  I set the cache to clear less often since it seems to be updating as long as the services get restarted so that should help some.

To answer your question, this was ArcGIS server 9.3.1 so I have not gotten to try the time option in 10 yet as we will not be able to switch until we can figure out why Image Server 10 mosaic datasets won't work with our aerial photography (tiffs), which is what I am trying to figure out now. 

Thanks for all the suggestions as always!