topic Re: Extremely big layer in ArcSDE in Data Management Questions

Extremely big layer in ArcSDE

ModyBuchbinder — Mon, 02 Jul 2012 12:18:01 GMT

Hello all

We have an unusual polygon layer in ArcSDE that does not behave well.
It has only 200k-300k polygons but most of them has 100-200 parts and each part can include up to 5000 vertexes.
In some extents you cannot identify or select the polygons.
When you decrease the number of total polygons (using definition query) the same polygons can be selected or identified.
It looks like we are getting to some limitation of ArcSDE.
There is no error messages on the screen or in log files.
We are using ArcSDE 10 on Oracle11g.
Does anybody run into this problem?
Is there any tuning (Oracle, server config) that we can try to do?

Thanks
Mody

Re: Extremely big layer in ArcSDE

VinceAngelo — Mon, 02 Jul 2012 13:23:12 GMT

I've loaded 600+ million row point layers and very large polygon layers that qualified as
"extremely big," but I think your problem is that the geometries are "extremely complex."
So the first question would have to be: Is there any way to can generalize the layer, and
limit times you need to use it at full detail?

I've found that point-in-polygon performance with a 1:100k country boundaries layer is
improved by an order of manitude or three when a 5 degree grid is intersected with it,
chunking complex polygons (Russia, Canada,..) resulting in ~2000 polygons (vice 167).

There are two types of polygon parts, exterior rings and interior rings. You'll often get
better performance from drawing hundreds of simple polygons as multi-part (exterior)
polygons, but complex interiors often are processed faster with single exterior parts.

While there may be limitations to ST_GEOMETRY storage, they'll mostly be in regard to
row-chaining of the blocks containing the pages storing each geometry.

- V

Re: Extremely big layer in ArcSDE

ModyBuchbinder — Tue, 03 Jul 2012 03:40:11 GMT

Hi Vince

Thanks for the answer. As you said the problem is the complexity of the polygons and not their number.
Most polygons have many islands and not holes (think about Greece).
The only operation needed is really to select polygons by point (there might be many polygon in one point) and display.
It all stored in st_geometry format. We did some tuning on the spatial index.
We can understand and accept slow performance, but we would like to get a result from the query.

Any specific tuning you can think about?

Thanks
Mody

Re: Extremely big layer in ArcSDE

VinceAngelo — Tue, 03 Jul 2012 10:14:43 GMT

If you think about the way the spatial index works, you'll see that it's far better to have many
polygons than a few very complex ones. In a point-in-poly application, I'd split any large
polygons (in grid cells, not area) rather than make the grid larger. In a worst case you might
return two neighboring parts, but since you already need to handle multiple return rows,
that might not be an issue.

There's just no way to tune point-in-poly search with large complex polygons.

- V