topic Problems with large datasets in File Geodatabase API Questions

Problems with large datasets

ArturRedzko — Thu, 19 Jan 2012 19:50:19 GMT

I'm testing huge datasets (millions of shapes) with different coverages for my country using API 1.1 with C++ and found some probably known problems :

- poor performance for large datasets due to not working spatial index
- no support for compressed datasets

The same data using ArcMap 10 loads very fast (~1 minute) and the spatial index for envelope selection is very quick. Also compression works too. Using API usually I give up after 10 min of waiting.
Are you going to fix and implement missing features in the next API release (1.2) and when do plan to release the update?
Are there any tricks that I can you right now to speed up reading of tables (besides table->LoadOnlyMode(true))?

Thanks
Artur

Re: Problems with large datasets

VinceAngelo — Fri, 20 Jan 2012 11:08:14 GMT

LoadOnlyMode(true) doesn't speed up query operations. It's purpose is to drop the
spatial index to improve INSERT performance, but while in LoadOnlyMode, the feature
class query performance will be degraded (no index).

It's possible that toggling load mode (from false to true and back to false) will improve
spatial query performance (depending on the fragmentation of the old index), but the
ordering of the spatial features inside the table itself (spatial fragmentation) might have
a bigger impact on overall draw performance than a stale spatial index (and its impact
would be evenly applied to both access APIs).

"Draw" performance can also be impacted by unnecessary attribute retrieval (only
'get' the attribute(s) you need for rendering).

- V

Re: Problems with large datasets

ArturRedzko — Tue, 24 Jan 2012 13:07:20 GMT

Thanks. I hope that you will be able to release new update soon.