Error: ???two donuts or two outer shells overlap???,

[Anonymous User]

Original User: Jamal432@gmail.com

Error: �??two donuts or two outer shells overlap�?�,

I wanted to copy enterprise geodatabase layer to file geodatabase but I got the error below (�??two donuts or two outer shells overlap�?�):

[ATTACH=CONFIG]33329[/ATTACH], [ATTACH=CONFIG]33330[/ATTACH], [ATTACH=CONFIG]33331[/ATTACH]

What might be the issue here?

Thank you

Best

Jamal

[MarcoBoeringa]

Now I�??m really in trouble due the data flow issues between file\enterprise databases. I can�??t transfer the data in both directions. Data will be lost

�?� Import\export\load data tools will apply the �??repair geometry�?� . The enterprise layer contains 44657 while the exported file layer is reduced to 42769

I am slightly surprised, I would have expected it to be the other way around: dirty data in the File Geodatabase with 44657 records, and then a certain number of records dropped on import to the Enterprise Geodatabase because of failure to comply with the (Clementini) rules and not repairable: 42769 records in the Enterprise Geodatabase.

How did you get this data in the Enterprise Geodatabase? Theoretically, the situation you created shouldn't be possibly (but I guess the answer may be the FME workflow you set up for ingesting the CAD files...).

I guess the next step might be to get back to the original (CAD) data, import this into a File Geodatabase first (possibly using your FME workflow, but with a changed target in the File Geodatabase instead of Enterprise Geodatabase if possible), run Repair Geometry / Integrate against the Feature Class in the File Geodatabase, and only then attempt to import the data in the Enterprise Geodatabase.

But I may be missing some vital steps in your current workflow, I only have a fragmented overview based on the different threads you started.

[Anonymous User]

Original User: Jamal432@gmail.com

I am slightly surprised, I would have expected it to be the other way around: dirty data in the File Geodatabase with 44657 records, and then a certain number of records dropped on import to the Enterprise Geodatabase because of failure to comply with the (Clementini) rules and not repairable: 42769 records in the Enterprise Geodatabase.

How did you get this data in the Enterprise Geodatabase? Theoretically, the situation you created shouldn't be possibly (but I guess the answer may be the FME workflow you set up for ingesting the CAD files...).

I guess the next step might be to get back to the original (CAD) data, import this into a File Geodatabase first (possibly using your FME workflow, but with a changed target in the File Geodatabase instead of Enterprise Geodatabase if possible), run Repair Geometry / Integrate against the Feature Class in the File Geodatabase, and only then attempt to import the data in the Enterprise Geodatabase.

But I may be missing some vital steps in your current workflow, I only have a fragmented overview based on the different threads you started.

Precisely. This is what is confusing me the most at the moment.

Our workflow is simple:

1. Our CAD data (hatches) are converted to polygons using the FME. The FME stores these polygones in file geodatabase layer. Many editors do this sort of work at the same time at their local machines and their local file geodatabase.

2. Next, each editor will copy and paste the converted polygons form his\her local file geodatabase layer to an ENTERPRISE layer.

3. Now, this enterprise layer is copied and pasted to other file geodatabase to be published by the ArcGIS server and then to be read by a web mapping application. This is done for performance issue. Services that is published from file geodatabase are faster than those published from enterprise.

Yes, I expect that all features that are living in Enterprise layers are fine. This appears not to be true as the error of �??two donuts or two outer shells overlap�?� comes up when coping from enterprise to file.


How to resolve the issue of �??two donuts or two outer shells overlap�?�? the way from enterprise to file should be OK all the time. But my case is a counter example.

[JamalNUMAN]

Precisely. This is what is confusing me the most at the moment.

Our workflow is simple:

1. Our CAD data (hatches) are converted to polygons using the FME. The FME stores these polygones in file geodatabase layer. Many editors do this sort of work at the same time at their local machines and their local file geodatabase.

2. Next, each editor will copy and paste the converted polygons form his\her local file geodatabase layer to an ENTERPRISE layer.

3. Now, this enterprise layer is copied and pasted to other file geodatabase to be published by the ArcGIS server and then to be read by a web mapping application. This is done for performance issue. Services that is published from file geodatabase are faster than those published from enterprise.

Yes, I expect that all features that are living in Enterprise layers are fine. This appears not to be true as the error of �??two donuts or two outer shells overlap�?� comes up when coping from enterprise to file.


How to resolve the issue of �??two donuts or two outer shells overlap�?�? the way from enterprise to file should be OK all the time. But my case is a counter example.

The issue of �??two donuts or two outer shells overlap�?� is escalated. Now, I�??m not able even to display the layer or export\load\import.

[ATTACH=CONFIG]33389[/ATTACH]

What other options do I still have to extract my data from this enterprise layer?

this error �??two donuts or two outer shells overlap�?� is killing me

[Anonymous User]

Original User: mboeringa2010

2. Next, each editor will copy and paste the converted polygons form his\her local file geodatabase layer to an ENTERPRISE layer.

Maybe the problem is with this step. As Vince suggested in the quote below, the COPY/PASTE may not be similar to IMPORT/EXPORT, with COPY/PASTE apparently not cleaning shapes before entry in the geodatabase (Vince: correct if wrong!). This causes faulty shapes to be uploaded to the Enterprise Geodatabase through the COPY/PASTE action.

You may need to have your partners run Repair Geometry, and possibly Integrate, before copying / pasting the data to the Enterprise Geodatabase. Alternatively, don't let them copy / paste data to the enterprise geodatabase, but use IMPORT/EXPORT tools to transfer the data to the Enterprise Geodatabase.

As to the specifics why COPY/PASTE actually differs from IMPORT/EXPORT, maybe it has something to do with ArcObjects based processes (copy / paste) versus processes on the ArcSDE level of the software stack (import / export), but this is just a wild guess. Maybe Vince can elaborate a bit more...

They are different processes.  Export/Import runs the equivalent of  Repair
Geometry on a feature if it fails on insert and tries again.  If the insert fails
on the second attempt then it rejects the feature and goes on.

[MarcoBoeringa]

The issue of �??two donuts or two outer shells overlap�?� is escalated. Now, I�??m not able even to display the layer or export\load\import.
...
What other options do I still have to extract my data from this enterprise layer?

You might be able to recuperate much of the data by using the FME / Data Interoperability extension against the layer and exporting to some other non-enterprise database format, or possibly using SQL Server tools for repairing geometries.

(There is a MakeValid command, but you will need SQL knowledge and I don't know if there are risks running this against an ESRI Enterprise Geodatabase, e.g. there might be issues with area / perimeter after geometries have been cleaned via SQL. Also see here and here).

Other option: restore from recent backup and see if you can restore the layer by re-importing data.

Last but not least: contact ESRI support... maybe a local representative with thorough ArcGIS knowledge could help you out, these kind of things are difficult to solve from a distance...

[Anonymous User]

Original User: Jamal432@gmail.com

You might be able to recuperate much of the data by using the FME / Data Interoperability extension against the layer and exporting to some other non-enterprise database format, or possibly using SQL Server tools for repairing geometries.

(There is a MakeValid command, but you will need SQL knowledge and I don't know if there are risks running this against an ESRI Enterprise Geodatabase, e.g. there might be issues with area / perimeter after geometries have been cleaned via SQL. Also see here and here).

Other option: restore from recent backup and see if you can restore the layer by re-importing data.

Last but not least: contact ESRI support... maybe a local representative with thorough ArcGIS knowledge could help you out, these kind of things are difficult to solve from a distance...

Thank you very much Macro for the massive help and valuable input.

�?� Copy\Paste doesn�??t repair features and thus if the copied\pasted features have errors then the Copy\Paste action never completes. It ends up with an error as the enterprise database doesn�??t accept incorrect features. This is what i understood from Vince too.

�?� The Import\Export\Load commands does repair the features but it doesn�??t give any indications which features are DELETED\repaired. We may need to apply the �??check geometry�?� before copy\paste or Import\Export\Load to have the list of incorrect features that will be deleted\repaired.

�?� I went for the BACKUP option to resolve the issue. But I need to ensure not be engaged in such very critical problem in the future.

[MarcoBoeringa]

We may need to apply the �??check geometry�?� before copy\paste or Import\Export\Load to have the list of incorrect features that will be deleted\repaired.

This seems wise. Run Check Geometry, and possibly Repair Geometry, as often as possible in your workflow. The sooner you detect serious geometric issues, even if caused by another step in your workflow than the initial data generation in CAD, the better. The more points in your entire workflow that you run Check Geometry, the higher the chance of homing in on potential (digitization) problems and problematic data conversion steps.

It would also be wise to start collecting some data on what problems occur if caused by digitization errors. You may than be able to put on paper some "best practices" and warnings concerning digitization issues, and distribute this among colleagues to make them aware of the issues. It may lead to better practices regarding digitization. The CAD colleagues may not even be aware of any issues (of course, this assumes issues are in initial data generation - which is likely - and not caused by subsequent conversion steps - less likely, but still very much possible).

[Anonymous User]

Original User: Jamal432@gmail.com

This seems wise. Run Check Geometry, and possibly Repair Geometry, as often as possible in your workflow. The sooner you detect serious geometric issues, even if caused by another step in your workflow than the initial data generation in CAD, the better. The more points in your entire workflow that you run Check Geometry, the higher the chance of homing in on potential (digitization) problems and problematic data conversion steps.

It would also be wise to start collecting some data on what problems occur if caused by digitization errors. You may than be able to put on paper some "best practices" and warnings concerning digitization issues, and distribute this among colleagues to make them aware of the issues. It may lead to better practices regarding digitization. The CAD colleagues may not even be aware of any issues (of course, this assumes issues are in initial data generation - which is likely - and not caused by subsequent conversion steps - less likely, but still very much possible).

Many thanks Macro.

We use the FME to extract data from CAD as the ArcGIS fails to read CAD hatches while the FME does.

Please, have a look how our FME workbench extracts the data from the CAD.

[ATTACH=CONFIG]33420[/ATTACH]

In words:

1. The Landuse hatches are converted to polygons
2. The Boundary hatch is converted to polygon
3. The Roads are derived by subtracting the Landuse from the Boundary (Roads = Boundary �?? Landsue).
4. The Landuse and Roads are collected in one layer
5. Later, this layer is copied and pasted to an enterprise layer

The Geometry Validation is applied three times in the FME workbench. Nevertheless, the ArcGIS detects errors as the �??check geometry�?� is applied.

Back to my issue:
Please, consider the attached file geodatabase layer. It has �??self intersections�?� error. Nevertheless, Copy\Paste from file geodatabse to enterprise works fine without fixing the error (repair geometry).

[ATTACH=CONFIG]33421[/ATTACH], [ATTACH=CONFIG]33422[/ATTACH], [ATTACH=CONFIG]33423[/ATTACH]

My question here: how the Enterprise Geodatabase Layer (SQL) accepts features that have error?

[MarcoBoeringa]

My question here: how the Enterprise Geodatabase Layer (SQL) accepts features that have error?

I can't answer this Jamal, but just a number of quick observations (need to have a closer look yet):

- I was able to reproduce the addition of the geometry number 51 with the geometric error to my SQL Server 2012 Express (all patched up) database using the Copy / Paste. This really is not good, but may be explained by a simple acceptance of the "geometry-as-is" by the Copy / Paste operation, and inserting it straight in the database. This screams for the ability to run Check Geometry and Repair Geometry against Enterprise Geodatabases as well, not just File Geodatabases (which isn't currently possible: the tool doesn't accept Enterprise Geodatabases as data source). I don't know if this would actually classify as a "bug" or is "as-build", maybe Vince can elaborate a bit more...

- The geometry with the error is the roads polygon in the layer

- As an addition to the above, I noticed, although you already build in 3 "Geometry Validator" steps in your FME model, you don't actually check and repair the road and boundary layers as the end products of your model. I would suggest adding two more "Geometry Validator" instances to the model, to tackle this problem. See the attached image.

- I would also strongly suggest to run "Check Geometry" and "Repair Geometry" against the data created by FME in your File Geodatabase, before adding it to the SQL Server. And possibly also run "Integrate" and / or create a "Topology" for the layer, to check for overlapping or non-contiguous areas in the polygon layers. A Topology will tell you more than just running Integrate, as it will highlight possible issues, instead of just "fixing" them, and give you more control over what is happening or how it should be fixed.

Do note that all of these tools, and especially Integrate, can lead to small shifts in the vertex coordinates of the vertices making up the geometries, so small shifts compared to the original CAD data, may occur.

[Anonymous User]

Original User: Jamal432@gmail.com

I can't answer this Jamal, but just a number of quick observations (need to have a closer look yet):

- I was able to reproduce the addition of the geometry number 51 with the geometric error to my SQL Server 2012 Express (all patched up) database using the Copy / Paste. This really is not good, but may be explained by a simple acceptance of the "geometry-as-is" by the Copy / Paste operation, and inserting it straight in the database. This screams for the ability to run Check Geometry and Repair Geometry against Enterprise Geodatabases as well, not just File Geodatabases (which isn't currently possible: the tool doesn't accept Enterprise Geodatabases as data source). I don't know if this would actually classify as a "bug" or is "as-build", maybe Vince can elaborate a bit more...

- The geometry with the error is the roads polygon in the layer

- As an addition to the above, I noticed, although you already build in 3 "Geometry Validator" steps in your FME model, you don't actually check and repair the road and boundary layers as the end products of your model. I would suggest adding two more "Geometry Validator" instances to the model, to tackle this problem. See the attached image.

- I would also strongly suggest to run "Check Geometry" and "Repair Geometry" against the data created by FME in your File Geodatabase, before adding it to the SQL Server. And possibly also run "Integrate" and / or create a "Topology" for the layer, to check for overlapping or non-contiguous areas in the polygon layers. A Topology will tell you more than just running Integrate, as it will highlight possible issues, instead of just "fixing" them, and give you more control over what is happening or how it should be fixed.

Do note that all of these tools, and especially Integrate, can lead to small shifts in the vertex coordinates of the vertices making up the geometries, so small shifts compared to the original CAD data, may occur.

Thanks Macro,

�?� All I wanted to do is to get one layer that contains Roads and Landuse with no errors at the level of FME to avoid further work in the ArcGIS

�?� Correct. I didn�??t apply the �??Geometry validator�?� for the Roads layer and Boundary layer. My final layer is neither the Roads nor the Boundary but one layer that combines both Land use and Roads. This is why I didn�??t apply the �??Geometry validator�?� Roads or Boundary but I did before the �??LanduseAndRoadsPolygons_FME�?� Layer. This is the layer that is collected in the enterprise layer as it represents one urban master plan for a particular community. Of course, this layer contains the Roads polygon(s) that has issues. I focused on the Roads layer (but not on the entire layer) because it has the issue.

[ATTACH=CONFIG]33519[/ATTACH]

�?� Topology\repair geometry\integrated could be solutions. What is horrible is that the enterprise layer accepts features with errors, and that is in turn, generates big problems such as �??two donuts or two outer shells overlap�?� and leads to deadlock.

I think that the enterprise layer (SQL or Oracle) should be designed not to accept features with errors as this will lead to destroy the data.