Check Geometry not finding invalid self-touching polygons

5503
14
08-02-2011 02:17 PM
MartinDavis
New Contributor
I have a dataset with over 24M polygons in it, which I need to load into SDE.  I am using FME to perform the load.  The load fails due to some polygons having invalid structure according to ESRI rules.  The invalid polygons consist of an outer shell and holes which touch the shell at a single point.  This is invalid according to the ESRI polygon structure rules (which I have seen documentation for, but can't find a link for right now).  Instead, according to the ESRI rules these kinds of area should be formed as "inverted polygons".

The attached screenshot shows an example of an invalid polygon.

Because of the large size of the dataset, I need an automated way to detect and fix these invalid polygons.  The obvious tool to do this is Check Geometry and Repair Geometry.The problem is that Check Geometry does not identify these polygons.  Also, Repair Geometry does not repair them. 

Is this a bug in Check Geometry? 
Is Check Geometry expected to work on FGDBs?
Is there another way of detecting and fixing these polygons?

I'm running ArcGIS 10.
0 Kudos
14 Replies
ChrisSnyder
Regular Contributor III
What is the native format of these polygons? Could it be that FME is causing the problem? Maybe load the datinto a FGDB 1st, clean it up (may not be neccessary), and then load it into SDE?
0 Kudos
MartinDavis
New Contributor
Thanks for the suggestions...

The native format that we're dealing with is FGDB.  I'm pretty sure that FME is not the cause of the problem, since I can inspect the structure of the polygons (using Edit/Sketch Properties) and verify that it contains a "self-touching" hole, rather than being in the ESRI-valid "inverted polygon" structure.

But I agree that fixing it in Arc and then retrying the load is a good idea, to verify the cause of the problem.  I don't have immediate access to the load environment, so this will take a little while.

As for fixing the polygons to have correct structure, running them through the Union tool corrects the structure to be ESRI-valid.  So this is a potential fix. However, I still need a way to identify all invalid polygons in the 24M-feature FGDB.  If Check Geometry won't do this, I'll need to think of some other way of detecting them (possibly via some custom Python code).
0 Kudos
ChrisSnyder
Regular Contributor III
Dumb question maybe, but why are you using FME to load FGDB data into SDE? Why not just use the build in ESRI tools? Seems the ESRI tools would be more bullet proof for ESRI formats...
0 Kudos
MartinDavis
New Contributor
Fair question.  We're working in the environment of a large corporate spatial data warehouse (the BC government), and the standard pipeline for loading SDE uses FME.

But it would be interesting to know if using ESRI tools would work where FME fails.  I don't think that's a long-term solution, however.
0 Kudos
ChrisSnyder
Regular Contributor III
Don't know about the wisdom of using a 3rd party software to convert an ESRI format into another ESRI format, but rules are rules, and FME is a Canadian product I believe.

After cleaning the data in FGDB format (Union or the RepairGeometry tool) you could run the FeatureCompare tool to identify what features changed. RepairGeom tool should give you a specific list of OBJECTIDs that were "fixed". I have had some bad experiences with the RepairGeometry tool... Be sure to do some random sampling of the features that are "fixed" to make sure it comes out the way it should.

Note that if your data contains overlapping polygons, the union tool will intersect your data (split the geometry) at the overlap bounadries...
0 Kudos
MartinDavis
New Contributor
Yes, FME is pretty much the home team, so we're all kinda partial to using it where it fits.  And generally we've had good experiences with the software and the support (not to mention that the licensing is cheaper!)

We've done some more testing, and found out that:

* indeed, using a pure ESRI pipeline to load the FGDB into SDE (via ArcCatalog) SOLVES the polygon problem.  The polygon structure is reshaped into an ESRI-valid structure (an inverted polygon).

* It turns out the the latest FME Beta (2012?) still exhibits the load failure on the invalid data. 

* That let us verify that re-shaping the polygon using Union did actually fix the problem, since teh updated dataset loaded into SDE via FME 2012 Beta with no problems.

Thanks for the warning about Union.  We realize that it's a pretty blunt tool, so we want to be very careful about what we update using it.  Since Check Geometry and Repair Geometry are non-starters about fixing these invalid polygons, I"m hoping that we can write some custom Python code to identify just the polygons in error, and then fix each one individually, to cause the least amount of change in the dataset.
0 Kudos
KenHartling
Esri Contributor
Martin,

Could I get a copy of some of these problem polygons?  I'm pretty sure I know why the check/repair geometry tools didn't catch it, but I'd like to confirm.

Thanks,
Ken
ESRI
Geoprocessing Product Engineer
0 Kudos
MartinDavis
New Contributor
Sure.  I'm attaching a ZIP file containing an FGDB with 3 polygons.  One of them has the invalid structure (the one with 3 holes all touching).

Any assistance you can provide will be very welcome.
0 Kudos
KenHartling
Esri Contributor
Thanks.  I'll get back to you once I've got some info.
Ken
0 Kudos