topic Re: Error: �??two donuts or two outer shells overlap�?�, in Data Management Questions

Error: ???two donuts or two outer shells overlap???,

Anonymous User — Thu, 24 Apr 2014 11:02:20 GMT

Original User: Jamal432@gmail.com

Error: �??two donuts or two outer shells overlap�?�,

I wanted to copy enterprise geodatabase layer to file geodatabase but I got the error below (�??two donuts or two outer shells overlap�?�):

[ATTACH=CONFIG]33329[/ATTACH], [ATTACH=CONFIG]33330[/ATTACH], [ATTACH=CONFIG]33331[/ATTACH]

What might be the issue here?

Thank you

Best

Jamal

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Thu, 24 Apr 2014 12:12:38 GMT

Original User: vangelo

The issue is that two donuts or two outer shells overlap (a violation of polygon topology
restrictions).

It would help if you did some research on shape topology. I use the reference work from
which the Esri libraries were designed:

E. Clementini, P. Di Felice, and P. van Oosterom, "A Small Set of Formal Topological Relationships Suitable for End-User Interaction," in Advances in Spatial Databases - Third International Symposium, SSD '93. vol. 692, D. Abel and B. C. Ooi, Eds. Berlin: Springer-Verlag, 1993, pp. 277-295.

This will also explain what a self-intersecting simple_line boundary is (though it was the
Open GIS Consortium that named the more restrictive SIMPLE_LINE that way).

- V

PS: The older ArcSDE documentation has a primer on shape validation which should also
be useful.

Re: Error: �??two donuts or two outer shells overlap�?�,

JamalNUMAN — Thu, 24 Apr 2014 16:05:11 GMT

The issue is that two donuts or two outer shells overlap (a violation of polygon topology
restrictions).

It would help if you did some research on shape topology. I use the reference work from
which the Esri libraries were designed:

E. Clementini, P. Di Felice, and P. van Oosterom, "A Small Set of Formal Topological Relationships Suitable for End-User Interaction," in Advances in Spatial Databases - Third International Symposium, SSD '93. vol. 692, D. Abel and B. C. Ooi, Eds. Berlin: Springer-Verlag, 1993, pp. 277-295.

This will also explain what a self-intersecting simple_line boundary is (though it was the
Open GIS Consortium that named the more restrictive SIMPLE_LINE that way).

- V

PS: The older ArcSDE documentation has a primer on shape validation which should also
be useful.

Thanks Vince,

What is confusing me the most, at this stage, is why should this be an issue. I�??m not applying any type of rules or topology. All what I do is copying and pasting.

By the way, if I IMPORT the same layer from the enterprise geodatabase to the file geodatabase, it works! Then why the issue is related to copy\paste approach? how the import does work?

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Thu, 24 Apr 2014 16:48:26 GMT

Original User: vangelo

Which is why you need to study the documentation on shape validation -- It does NOT
mean the same thing as applying rules to feature classes. This topology is about how
actual geometry primitives work, self-referentially. Without this understanding, you
will always be confused with respect to data conversion errors.

- V

Re: Error: �??two donuts or two outer shells overlap�?�,

JamalNUMAN — Thu, 24 Apr 2014 17:33:21 GMT

Which is why you need to study the documentation on shape validation -- It does NOT
mean the same thing as applying rules to feature classes. This topology is about how
actual geometry primitives work, self-referentially. Without this understanding, you
will always be confused with respect to data conversion errors.

- V

That�??s for sure. Understanding will answer why But will never solve the issue.

But how IMPORT is different from COPY\PASTE? We are here talking about same features that will be copied at the end of the day to the file geodatabase? How they are accepted if they are imported but not if they are copied\pasted?

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Thu, 24 Apr 2014 18:39:37 GMT

Original User: vangelo

They are different processes. Export/Import runs the equivalent of Repair
Geometry on a feature if it fails on insert and tries again. If the insert fails
on the second attempt then it rejects the feature and goes on. This is both
good and bad, since invalid data that can be coaxed into something that
looks valid will pass, but then you've got garbage in your database.

- V

Re: Error: �??two donuts or two outer shells overlap�?�,

JamalNUMAN — Fri, 25 Apr 2014 19:22:58 GMT

They are different processes. Export/Import runs the equivalent of Repair
Geometry on a feature if it fails on insert and tries again. If the insert fails
on the second attempt then it rejects the feature and goes on. This is both
good and bad, since invalid data that can be coaxed into something that
looks valid will pass, but then you've got garbage in your database.

- V

Thanks Vince for the input. This is very useful

Best

Jamal

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Fri, 25 Apr 2014 19:58:01 GMT

Original User: mboeringa2010

They are different processes. Export/Import runs the equivalent of Repair
Geometry on a feature if it fails on insert and tries again. If the insert fails
on the second attempt then it rejects the feature and goes on. This is both
good and bad, since invalid data that can be coaxed into something that
looks valid will pass, but then you've got garbage in your database.

This is some vital information missing from the ArcGIS for Desktop Help that might be added (by the way, in general I regard ESRI's Help and documentation efforts of a very high level, the Help and whitepapers are one huge reservoir of knowledge and of great use once you acquaint yourself with the basic jargon).

According to the Help, and running the Import option from ArcCatalog, the Import option is associated with the Feature Class to Feature Class tool. I don't see this "repair" step listed in the Help.

Re: Error: �??two donuts or two outer shells overlap�?�,

JamalNUMAN — Sun, 27 Apr 2014 08:40:59 GMT

This is some vital information missing from the ArcGIS for Desktop Help that might be added (by the way, in general I regard ESRI's Help and documentation efforts of a very high level, the Help and whitepapers are one huge reservoir of knowledge and of great use once you acquaint yourself with the basic jargon).

According to the Help, and running the Import option from ArcCatalog, the Import option is associated with the Feature Class to Feature Class tool. I don't see this "repair" step listed in the Help.

Many thanks for the input.

Now I�??m really in trouble due the data flow issues between file\enterprise databases. I can�??t transfer the data in both directions. Data will be lost

�?� Import\export\load data tools will apply the �??repair geometry�?� . The enterprise layer contains 44657 while the exported file layer is reduced to 42769

[ATTACH=CONFIG]33385[/ATTACH], [ATTACH=CONFIG]33386[/ATTACH]

�?� I wanted to apply topology for the layer in question but I couldn�??t copy\paste that layer to dataset to apply the topology.

[ATTACH=CONFIG]33387[/ATTACH]

�?� I tried to join the enterprise layer with the exported file layer to see which features are deleted, but it appears that the ArcGIS keeps giving the same error!

[ATTACH=CONFIG]33388[/ATTACH]

What might be the solution in this case?

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Sun, 27 Apr 2014 09:18:33 GMT

Original User: mboeringa2010

This is some vital information missing from the ArcGIS for Desktop Help that might be added (by the way, in general I regard ESRI's Help and documentation efforts of a very high level, the Help and whitepapers are one huge reservoir of knowledge and of great use once you acquaint yourself with the basic jargon).

According to the Help, and running the Import option from ArcCatalog, the Import option is associated with the Feature Class to Feature Class tool. I don't see this "repair" step listed in the Help.

To follow up on this: I did now notice the "Check & Repair" step on loading data in an SDE Enterprise Geodatabase, is actually mentioned under the Repair Geometry Help topic:

"SDE geodatabases automatically check and repair feature geometries when the features are uploaded to the database, so using the Check Geometry and Repair Geometry tools with SDE feature classes is unnecessary."

Re: Error: �??two donuts or two outer shells overlap�?�,

MarcoBoeringa — Sun, 27 Apr 2014 09:26:13 GMT

Now I�??m really in trouble due the data flow issues between file\enterprise databases. I can�??t transfer the data in both directions. Data will be lost

�?� Import\export\load data tools will apply the �??repair geometry�?� . The enterprise layer contains 44657 while the exported file layer is reduced to 42769

I am slightly surprised, I would have expected it to be the other way around: dirty data in the File Geodatabase with 44657 records, and then a certain number of records dropped on import to the Enterprise Geodatabase because of failure to comply with the (Clementini) rules and not repairable: 42769 records in the Enterprise Geodatabase.

How did you get this data in the Enterprise Geodatabase? Theoretically, the situation you created shouldn't be possibly (but I guess the answer may be the FME workflow you set up for ingesting the CAD files...).

I guess the next step might be to get back to the original (CAD) data, import this into a File Geodatabase first (possibly using your FME workflow, but with a changed target in the File Geodatabase instead of Enterprise Geodatabase if possible), run Repair Geometry / Integrate against the Feature Class in the File Geodatabase, and only then attempt to import the data in the Enterprise Geodatabase.

But I may be missing some vital steps in your current workflow, I only have a fragmented overview based on the different threads you started.

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Sun, 27 Apr 2014 10:17:33 GMT

Original User: Jamal432@gmail.com

I am slightly surprised, I would have expected it to be the other way around: dirty data in the File Geodatabase with 44657 records, and then a certain number of records dropped on import to the Enterprise Geodatabase because of failure to comply with the (Clementini) rules and not repairable: 42769 records in the Enterprise Geodatabase.

How did you get this data in the Enterprise Geodatabase? Theoretically, the situation you created shouldn't be possibly (but I guess the answer may be the FME workflow you set up for ingesting the CAD files...).

I guess the next step might be to get back to the original (CAD) data, import this into a File Geodatabase first (possibly using your FME workflow, but with a changed target in the File Geodatabase instead of Enterprise Geodatabase if possible), run Repair Geometry / Integrate against the Feature Class in the File Geodatabase, and only then attempt to import the data in the Enterprise Geodatabase.

But I may be missing some vital steps in your current workflow, I only have a fragmented overview based on the different threads you started.

Precisely. This is what is confusing me the most at the moment.

Our workflow is simple:

1. Our CAD data (hatches) are converted to polygons using the FME. The FME stores these polygones in file geodatabase layer. Many editors do this sort of work at the same time at their local machines and their local file geodatabase.

2. Next, each editor will copy and paste the converted polygons form his\her local file geodatabase layer to an ENTERPRISE layer.

3. Now, this enterprise layer is copied and pasted to other file geodatabase to be published by the ArcGIS server and then to be read by a web mapping application. This is done for performance issue. Services that is published from file geodatabase are faster than those published from enterprise.

Yes, I expect that all features that are living in Enterprise layers are fine. This appears not to be true as the error of �??two donuts or two outer shells overlap�?� comes up when coping from enterprise to file.

How to resolve the issue of �??two donuts or two outer shells overlap�?�? the way from enterprise to file should be OK all the time. But my case is a counter example.

Re: Error: �??two donuts or two outer shells overlap�?�,

JamalNUMAN — Sun, 27 Apr 2014 11:41:14 GMT

Precisely. This is what is confusing me the most at the moment.

Our workflow is simple:

1. Our CAD data (hatches) are converted to polygons using the FME. The FME stores these polygones in file geodatabase layer. Many editors do this sort of work at the same time at their local machines and their local file geodatabase.

2. Next, each editor will copy and paste the converted polygons form his\her local file geodatabase layer to an ENTERPRISE layer.

3. Now, this enterprise layer is copied and pasted to other file geodatabase to be published by the ArcGIS server and then to be read by a web mapping application. This is done for performance issue. Services that is published from file geodatabase are faster than those published from enterprise.

Yes, I expect that all features that are living in Enterprise layers are fine. This appears not to be true as the error of �??two donuts or two outer shells overlap�?� comes up when coping from enterprise to file.

How to resolve the issue of �??two donuts or two outer shells overlap�?�? the way from enterprise to file should be OK all the time. But my case is a counter example.

The issue of �??two donuts or two outer shells overlap�?� is escalated. Now, I�??m not able even to display the layer or export\load\import.

[ATTACH=CONFIG]33389[/ATTACH]

What other options do I still have to extract my data from this enterprise layer?

this error �??two donuts or two outer shells overlap�?� is killing me

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Sun, 27 Apr 2014 11:49:44 GMT

Original User: mboeringa2010

2. Next, each editor will copy and paste the converted polygons form his\her local file geodatabase layer to an ENTERPRISE layer.

Maybe the problem is with this step. As Vince suggested in the quote below, the COPY/PASTE may not be similar to IMPORT/EXPORT, with COPY/PASTE apparently not cleaning shapes before entry in the geodatabase (Vince: correct if wrong!). This causes faulty shapes to be uploaded to the Enterprise Geodatabase through the COPY/PASTE action.

You may need to have your partners run Repair Geometry, and possibly Integrate, before copying / pasting the data to the Enterprise Geodatabase. Alternatively, don't let them copy / paste data to the enterprise geodatabase, but use IMPORT/EXPORT tools to transfer the data to the Enterprise Geodatabase.

As to the specifics why COPY/PASTE actually differs from IMPORT/EXPORT, maybe it has something to do with ArcObjects based processes (copy / paste) versus processes on the ArcSDE level of the software stack (import / export), but this is just a wild guess. Maybe Vince can elaborate a bit more...

They are different processes. Export/Import runs the equivalent of Repair
Geometry on a feature if it fails on insert and tries again. If the insert fails
on the second attempt then it rejects the feature and goes on.

Re: Error: �??two donuts or two outer shells overlap�?�,

MarcoBoeringa — Sun, 27 Apr 2014 12:17:10 GMT

The issue of �??two donuts or two outer shells overlap�?� is escalated. Now, I�??m not able even to display the layer or export\load\import.
...
What other options do I still have to extract my data from this enterprise layer?

You might be able to recuperate much of the data by using the FME / Data Interoperability extension against the layer and exporting to some other non-enterprise database format, or possibly using SQL Server tools for repairing geometries.

(There is a MakeValid command, but you will need SQL knowledge and I don't know if there are risks running this against an ESRI Enterprise Geodatabase, e.g. there might be issues with area / perimeter after geometries have been cleaned via SQL. Also see here and here).

Other option: restore from recent backup and see if you can restore the layer by re-importing data.

Last but not least: contact ESRI support... maybe a local representative with thorough ArcGIS knowledge could help you out, these kind of things are difficult to solve from a distance...

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Sun, 27 Apr 2014 16:36:44 GMT

Original User: Jamal432@gmail.com

You might be able to recuperate much of the data by using the FME / Data Interoperability extension against the layer and exporting to some other non-enterprise database format, or possibly using SQL Server tools for repairing geometries.

(There is a MakeValid command, but you will need SQL knowledge and I don't know if there are risks running this against an ESRI Enterprise Geodatabase, e.g. there might be issues with area / perimeter after geometries have been cleaned via SQL. Also see here and here).

Other option: restore from recent backup and see if you can restore the layer by re-importing data.

Last but not least: contact ESRI support... maybe a local representative with thorough ArcGIS knowledge could help you out, these kind of things are difficult to solve from a distance...

Thank you very much Macro for the massive help and valuable input.

�?� Copy\Paste doesn�??t repair features and thus if the copied\pasted features have errors then the Copy\Paste action never completes. It ends up with an error as the enterprise database doesn�??t accept incorrect features. This is what i understood from Vince too.

�?� The Import\Export\Load commands does repair the features but it doesn�??t give any indications which features are DELETED\repaired. We may need to apply the �??check geometry�?� before copy\paste or Import\Export\Load to have the list of incorrect features that will be deleted\repaired.

�?� I went for the BACKUP option to resolve the issue. But I need to ensure not be engaged in such very critical problem in the future.

Re: Error: �??two donuts or two outer shells overlap�?�,

MarcoBoeringa — Mon, 28 Apr 2014 08:25:18 GMT

We may need to apply the �??check geometry�?� before copy\paste or Import\Export\Load to have the list of incorrect features that will be deleted\repaired.

This seems wise. Run Check Geometry, and possibly Repair Geometry, as often as possible in your workflow. The sooner you detect serious geometric issues, even if caused by another step in your workflow than the initial data generation in CAD, the better. The more points in your entire workflow that you run Check Geometry, the higher the chance of homing in on potential (digitization) problems and problematic data conversion steps.

It would also be wise to start collecting some data on what problems occur if caused by digitization errors. You may than be able to put on paper some "best practices" and warnings concerning digitization issues, and distribute this among colleagues to make them aware of the issues. It may lead to better practices regarding digitization. The CAD colleagues may not even be aware of any issues (of course, this assumes issues are in initial data generation - which is likely - and not caused by subsequent conversion steps - less likely, but still very much possible).

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Mon, 28 Apr 2014 16:54:05 GMT

Original User: Jamal432@gmail.com

This seems wise. Run Check Geometry, and possibly Repair Geometry, as often as possible in your workflow. The sooner you detect serious geometric issues, even if caused by another step in your workflow than the initial data generation in CAD, the better. The more points in your entire workflow that you run Check Geometry, the higher the chance of homing in on potential (digitization) problems and problematic data conversion steps.

It would also be wise to start collecting some data on what problems occur if caused by digitization errors. You may than be able to put on paper some "best practices" and warnings concerning digitization issues, and distribute this among colleagues to make them aware of the issues. It may lead to better practices regarding digitization. The CAD colleagues may not even be aware of any issues (of course, this assumes issues are in initial data generation - which is likely - and not caused by subsequent conversion steps - less likely, but still very much possible).

Many thanks Macro.

We use the FME to extract data from CAD as the ArcGIS fails to read CAD hatches while the FME does.

Please, have a look how our FME workbench extracts the data from the CAD.

[ATTACH=CONFIG]33420[/ATTACH]

In words:

1. The Landuse hatches are converted to polygons
2. The Boundary hatch is converted to polygon
3. The Roads are derived by subtracting the Landuse from the Boundary (Roads = Boundary �?? Landsue).
4. The Landuse and Roads are collected in one layer
5. Later, this layer is copied and pasted to an enterprise layer

The Geometry Validation is applied three times in the FME workbench. Nevertheless, the ArcGIS detects errors as the �??check geometry�?� is applied.

Back to my issue:
Please, consider the attached file geodatabase layer. It has �??self intersections�?� error. Nevertheless, Copy\Paste from file geodatabse to enterprise works fine without fixing the error (repair geometry).

[ATTACH=CONFIG]33421[/ATTACH], [ATTACH=CONFIG]33422[/ATTACH], [ATTACH=CONFIG]33423[/ATTACH]

My question here: how the Enterprise Geodatabase Layer (SQL) accepts features that have error?

Re: Error: �??two donuts or two outer shells overlap�?�,

MarcoBoeringa — Thu, 01 May 2014 06:04:38 GMT

My question here: how the Enterprise Geodatabase Layer (SQL) accepts features that have error?

I can't answer this Jamal, but just a number of quick observations (need to have a closer look yet):

- I was able to reproduce the addition of the geometry number 51 with the geometric error to my SQL Server 2012 Express (all patched up) database using the Copy / Paste. This really is not good, but may be explained by a simple acceptance of the "geometry-as-is" by the Copy / Paste operation, and inserting it straight in the database. This screams for the ability to run Check Geometry and Repair Geometry against Enterprise Geodatabases as well, not just File Geodatabases (which isn't currently possible: the tool doesn't accept Enterprise Geodatabases as data source). I don't know if this would actually classify as a "bug" or is "as-build", maybe Vince can elaborate a bit more...

- The geometry with the error is the roads polygon in the layer

- As an addition to the above, I noticed, although you already build in 3 "Geometry Validator" steps in your FME model, you don't actually check and repair the road and boundary layers as the end products of your model. I would suggest adding two more "Geometry Validator" instances to the model, to tackle this problem. See the attached image.

- I would also strongly suggest to run "Check Geometry" and "Repair Geometry" against the data created by FME in your File Geodatabase, before adding it to the SQL Server. And possibly also run "Integrate" and / or create a "Topology" for the layer, to check for overlapping or non-contiguous areas in the polygon layers. A Topology will tell you more than just running Integrate, as it will highlight possible issues, instead of just "fixing" them, and give you more control over what is happening or how it should be fixed.

Do note that all of these tools, and especially Integrate, can lead to small shifts in the vertex coordinates of the vertices making up the geometries, so small shifts compared to the original CAD data, may occur.

Re: Error: �??two donuts or two outer shells overlap�?�,

Anonymous User — Thu, 01 May 2014 11:26:00 GMT

Original User: Jamal432@gmail.com

I can't answer this Jamal, but just a number of quick observations (need to have a closer look yet):

- I was able to reproduce the addition of the geometry number 51 with the geometric error to my SQL Server 2012 Express (all patched up) database using the Copy / Paste. This really is not good, but may be explained by a simple acceptance of the "geometry-as-is" by the Copy / Paste operation, and inserting it straight in the database. This screams for the ability to run Check Geometry and Repair Geometry against Enterprise Geodatabases as well, not just File Geodatabases (which isn't currently possible: the tool doesn't accept Enterprise Geodatabases as data source). I don't know if this would actually classify as a "bug" or is "as-build", maybe Vince can elaborate a bit more...

- The geometry with the error is the roads polygon in the layer

- As an addition to the above, I noticed, although you already build in 3 "Geometry Validator" steps in your FME model, you don't actually check and repair the road and boundary layers as the end products of your model. I would suggest adding two more "Geometry Validator" instances to the model, to tackle this problem. See the attached image.

- I would also strongly suggest to run "Check Geometry" and "Repair Geometry" against the data created by FME in your File Geodatabase, before adding it to the SQL Server. And possibly also run "Integrate" and / or create a "Topology" for the layer, to check for overlapping or non-contiguous areas in the polygon layers. A Topology will tell you more than just running Integrate, as it will highlight possible issues, instead of just "fixing" them, and give you more control over what is happening or how it should be fixed.

Do note that all of these tools, and especially Integrate, can lead to small shifts in the vertex coordinates of the vertices making up the geometries, so small shifts compared to the original CAD data, may occur.

Thanks Macro,

�?� All I wanted to do is to get one layer that contains Roads and Landuse with no errors at the level of FME to avoid further work in the ArcGIS

�?� Correct. I didn�??t apply the �??Geometry validator�?� for the Roads layer and Boundary layer. My final layer is neither the Roads nor the Boundary but one layer that combines both Land use and Roads. This is why I didn�??t apply the �??Geometry validator�?� Roads or Boundary but I did before the �??LanduseAndRoadsPolygons_FME�?� Layer. This is the layer that is collected in the enterprise layer as it represents one urban master plan for a particular community. Of course, this layer contains the Roads polygon(s) that has issues. I focused on the Roads layer (but not on the entire layer) because it has the issue.

[ATTACH=CONFIG]33519[/ATTACH]

�?� Topology\repair geometry\integrated could be solutions. What is horrible is that the enterprise layer accepts features with errors, and that is in turn, generates big problems such as �??two donuts or two outer shells overlap�?� and leads to deadlock.

I think that the enterprise layer (SQL or Oracle) should be designed not to accept features with errors as this will lead to destroy the data.