ArcGIS Document Open Efficiency

Idea created by Priemere on Mar 31, 2013
    Performance for opening an ArcGIS Document (MxD, SxD, GxD) needs to be improved.  It is a significant irritant to many users, and an excuse for others to just not use the application at all.
    Current architecture appears to prepare everything that might ever be needed immediately upon opening the document.  Why not offer some “just in time” options to minimize that up-front work?
    Most end users are not particularly adept at managing their documents, and often end up accumulating hundreds of layers.  Many of those layers are disabled, yet significant time can be spent connecting to each data source across a network environment.  Worse, often a user will be hunting around through a group of documents, waiting a long time to open each, just to see the table of contents, and realize that it is the wrong document, and move on to the next one!
    To substantiate, here are some statistics from monitoring more than a hundred users over a period of time, during which there were more than 3000 document open events.  We measured the elapsed time, and categorized with breaks at 10 seconds, 30 seconds, 1 minute, and 5 minutes.  Only 10% of the open events were less than 10 seconds, about half were more than 30 seconds, 20% more than a minute.  Thankfully, only 2% were more than 5 minutes, but about a quarter of all the users had had this off-putting experience, and more than three quarters had at least one experience of more than a minute. 
    Suggestion is that the Open process be staged in several JIT (just in time) events.  These could be used from the ArcGIS map interface, and with developer methods that are exposed.  Each layer would contain a number of properties to flag the status.  When a particular property is requested, but still unknown, then and only then is the data source inspected for the required information.
    Another suggestion is a user tolerance for startup time, along with a progress meter and/or interrupt option.  Work through the above JIT stuff until the allowed time has expired.  For multiple data frames, only process the active data frame first.  And for any disabled layer, defer processing it (and all descendants) until after enabled layers have been processed.  When the tolerable amount of time has elapsed, interrupt the process, and do what you can with what is available.  Perhaps put the map into “Pause” mode if not all of the visible layers have been processed yet, so the user can disable things before resuming the drawing.