topic Re: Null/Empty Geometry in Python in Python Questions

Null/Empty Geometry in Python

LeeGodfrey1 — Tue, 16 Aug 2016 14:05:53 GMT

I have a series of Feature Classes ( Polyline/arc type) that have no Geometry associated with them. They have been converted through multiple different systems ( Mid Miff, Cable Cad ( custom viewer) and now GDB) and it was lost along the way. This is just part of the process unfortunately, and has been identified with this old set of data for years.

If I load the feature class into ArcMap and zoom to layer, nothing is visible, they have shape length of zero, going to individual ones displays nothing, as expected of having no Geometry and no shape length.
However, I can add a Double field for XCoord and YCoord, and using Calculate Geometry I can get the coordinate values for the midpoint of the line. Which allows me to then export the data out to a table, and then using display XY Coordinates I can see the locations with the associated attributes. This is the end goal of what I want, and manually in ArcMap it works.

My issues is that I have hundreds of these, and I am attempting to automate the process using Python.

I have attempted to get midpoint using: Midpoint = Feature.shape.positionAlongLine(0.50,True).firstPoint however this does not work because it states that I have Null or Empty geometry.

The below works, in that it will at least read the features, but it still returns values of None for the coordinates ( even though when I manually try in ArcMap I get values)

fields = ["SHAPE@XY","X","Y"]

            with arcpy.da.UpdateCursor(path,fields) as cursor:
               for row in cursor:
                   x,y = row[0]
                    print("{},{}".format(x,y))
                   row[1] = x
                   row[2] = y
                   cursor.updateRow(row)

I am wondering if anyone has had any success with dealing with Null/Empty Geometries in Python coding.

Thank you anyone for your assistance

I have attached a sample of the data below in a zipped GDB.

Re: Null/Empty Geometry in Python

DanPatterson_Retired — Tue, 16 Aug 2016 14:26:36 GMT

try Shape@ if they are polyline or polygons (and I think even points), the centroid of the geometry will be returned IF it can be returned

Re: Null/Empty Geometry in Python

JakeSkinner — Tue, 16 Aug 2016 14:50:57 GMT

Hi Lee,

Have you tried the add XY Coordinates tool?

Re: Null/Empty Geometry in Python

XanderBakker — Tue, 16 Aug 2016 15:17:40 GMT

In case that you select a feature (from the attribute table) and you can zoom at, and it shows the selection in the map, but not the feature itself, you may want to check if the Spatial Index is valid. At least this has happened to me, when using an Enterprise geodatabase (Oracle in this case).

Is it possible to attache a small sample of the data to run some tests?

Re: Null/Empty Geometry in Python

RebeccaStrauch__GISP — Tue, 16 Aug 2016 15:45:52 GMT

Have you try to Check Geometry—Help | ArcGIS for Desktop and then Repair Geometry—Help | ArcGIS for Desktop (on a copy of course)

re you code above.... the print statements shouldn't be indented (see https://community.esri.com/people/curtvprice/blog/2014/09/25/posting-code-blocks-in-the-new-geonet?sr=search&searchId=6f8262c1-3ab6-452d-a2ed-af85b1a375e7&searchIndex=1‌ although I think it changed with the friday update... and you can do it from the ...More on the toolbars of every window)

There are other things you can try including Feature Vertices To Points—Help | ArcGIS for Desktop (and others in that section)...depending on what you final goal is. There are code samples for each.

Re: Null/Empty Geometry in Python

LeeGodfrey1 — Tue, 16 Aug 2016 17:09:19 GMT

Hello Xander,

I did some checks with spatial index in trying to validate it but had no changes. I have attached a GDB that is some of the original data that I haven't done any tests on.

- Lee

Re: Null/Empty Geometry in Python

LeeGodfrey1 — Tue, 16 Aug 2016 17:25:12 GMT

Cursor = arcpy.SearchCursor(path)
            row = Cursor.next()
            while row:
                Midpoint = row.shape.positionAlongLine(0.50,True).firstPoint
                print Midpoint.X
                print Midpoint.Y
                row = Cursor.next()

This section of code ended up working, it was brought forward by a co-worker. I had thought I'd used it before but apparently I did not. Thank you all for responses. The answer was in my commented out in my code all along, and was pretty simple.

Re: Null/Empty Geometry in Python

XanderBakker — Sat, 11 Dec 2021 08:31:55 GMT

I just started an edit session and I notice that the geometries I have tested consist of two points and have the same start and end point:

That yields the SHAPE_Length is 0 and in some operations the geometry will not be considered valid.

If I run this code:

fc = r'D:\Xander\GeoNet\XYvalues\gdb\ESRIForumSampleDataSet_Lee.gdb\POLE_arc'
with arcpy.da.SearchCursor(fc, ('SHAPE@')) as curs:
    for row in curs:
        polyline = row[0]
        print polyline.firstPoint.X, polyline.firstPoint.Y, polyline.lastPoint.X, polyline.lastPoint.Y, polyline.length‍‍‍‍‍

It will yield, X and Y of the firstPoint, X and Y of the lastPoint and the length of the line:

272575.775391 741323.871826 272575.775391 741323.871826 0.0
272542.675415 741357.241821 272542.675415 741357.241821 0.0
272536.0354 741340.311829 272536.0354 741340.311829 0.0
272460.255432 741441.681824 272460.255432 741441.681824 0.0
272364.155396 741540.721802 272364.155396 741540.721802 0.0
‍‍‍‍‍

You can see start and end are the same and the length is 0.

Re: Null/Empty Geometry in Python

RebeccaStrauch__GISP — Tue, 16 Aug 2016 20:51:42 GMT

Sounds to me like those are points...not lines.

Same start/end with length..poly. Same start/end no length...point.

From my coverage "daze"

Re: Null/Empty Geometry in Python

JoshuaBixby — Sat, 11 Dec 2021 08:31:58 GMT

The data has definitely been mangled along the way. When digging through the sample data, you appear to have 2 different types of corruption occurring. Looking at the JSON representations for OID 1 and 10, respectively:

{"curvePaths":[[[272575.775390625,741323.87182617187],{"a":[[272575.775390625,741323.87182617187],[272575.20600000001,741323.30700000003],1,1,1.5707963267948966,0.802018882730549,0.99999999996371092]}]],"spatialReference":{"wkid":2200,"latestWkid":2200}}

{"curvePaths":[[[274805.416015625,743884.35400390625],[274805.416015625,743884.35400390625]]],"spatialReference":{"wkid":2200,"latestWkid":2200}}

For OID 1, which represents the vast majority of data in the sample data set, the geometry is effectively a circle that is being represented by a corrupted ellipse (See JSON curve object of the ArcGIS REST API). Changing the arc reference from minor (1) to major (0) will create a valid circular ellipse.

For OID 10, which represents roughly <10% of the sample data, the geometry is much more corrupted. There are fragments of either a circle or ellipse, but not enough to generate a valid geometry without making some assumptions.

The following code goes through the sample dataset and updates the geometries represented by OID 1:

>>> import json
>>> cur = arcpy.da.UpdateCursor("POLE_arc", ["OID@","SHAPE@"])
>>> for oid, shape in cur:
...     shape_dict = json.loads(shape.JSON)
...     try:
...         shape_dict["curvePaths"][0][1]["a"][2] = 0
...         shape = arcpy.AsShape(json.dumps(shape_dict), True)
...         cur.updateRow([oid, shape])
...     except Exception as e:
...         print "OID {} can't be updated".format(oid)‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

For the geometries represented by OID 10, the best you can do is turn them into a point.

Re: Null/Empty Geometry in Python

RandyBurton — Wed, 17 Aug 2016 04:28:42 GMT

Could you make an assumption that the exceptions (those like OID 10) are circles with a similar diameter, rotation, etc. and update the geometry, instead of converting them to points?

Re: Null/Empty Geometry in Python

LeeGodfrey1 — Wed, 17 Aug 2016 12:11:29 GMT

That's great, thank you. I hadn't thought to look for just the first point in the line as all the other work in these datasets was set for the midpoint. Very helpful 🙂

Re: Null/Empty Geometry in Python

JoshuaBixby — Wed, 17 Aug 2016 14:14:28 GMT

Yes, one could do that. Looking over all of the recovered circles/ellipses, the diameters all seem very close to each other, so I think a fairly reasonable and accurate assumption could be made. I will update my original code when I can find the time.

UPDATE: Updated/complete code provided subsequently by rvburton‌.

Re: Null/Empty Geometry in Python

LeeGodfrey1 — Wed, 17 Aug 2016 19:35:03 GMT

I have tried Add XY Coordinates, it does not work because they are

polylines unfortunately. Thank you though.

Re: Null/Empty Geometry in Python

LeeGodfrey1 — Wed, 17 Aug 2016 19:54:16 GMT

I have tried Shape@XY, it will successfully read each line of the feature,

but it is still unable to assign the coordinate to it. When I use calculate

geometry manually in ArcMap it will assign it to them, so it does exist

somewhere, but unfortunately Shape@ does not pick them up. thank you though.

Re: Null/Empty Geometry in Python

RandyBurton — Sat, 11 Dec 2021 08:32:01 GMT

This builds on Joshua Bixby's comments and proposed solution. It assumes there are only 2 different types of corruption in your data - either the curve object has invalid data or the curve object is basically missing. Joshua's code fixes those features where the curve has invalid data. In the exception block, this code creates a curve object using the assumption that the curves are circles similar to all the others. This worked with the sample dataset. If there are additional types of corruption in the full dataset, you may need to modify the exception code.

This solution requires 10.3 or higher. Load the feature in ArcMap and run the following code in the Python window.

import json
cur = arcpy.da.UpdateCursor("POLE_arc", ["OID@","SHAPE@"])
for oid, shape in cur:
    shape_dict = json.loads(shape.JSON)
    try:
        shape_dict["curvePaths"][0][1]["a"][2] = 0
        shape = arcpy.AsShape(json.dumps(shape_dict), True)
        cur.updateRow([oid, shape])
    except:
        x1 = shape_dict["curvePaths"][0][0][0]
        y1 = shape_dict["curvePaths"][0][0][1]
        x2 = x1 - 0.544675488298497
        y2 = y1 - 0.540181258002587
        shape_dict["curvePaths"][0][1] = {"a":[[x1,y1],[x2,y2], 0, 1, 1.57079632679, 0.80206559811, 0.999999999964]}
        shape = arcpy.AsShape(json.dumps(shape_dict), True)
        cur.updateRow([oid, shape])‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

And a thank you to Joshua.

Re: Null/Empty Geometry in Python

GIS_Solutions — Fri, 09 Nov 2018 06:19:39 GMT

Hi @Randy

I am having a similar problem, I would be glad if you help.

I need to delete these empty geometry data,

I was doing synchronization operations with the task scheduler, I created 5 replicas, 2 of them were created geometric notwork.
Synchronization fails because the geometric network does not accept empty or null geometry data.

How can I delete this data with python without opening arcmp with the data that is empty geometry?

this way I prepared python code.
The code does not work in the synchronization section when there is an empty geometry.

My goal is to do the synchronization process with python every day automatically.

# Set the necessary product code
import arceditor

# Import arcpy module
import arcpy
v172_16_200_86_GIS_sde__5_ = "Database Connections\\172.16.200.86_GIS.sde"
v172_16_200_86_GIS_sde__6_ = v172_16_200_86_GIS_sde__5_
v172_16_200_86_GIS_NETWORK_sde__2_ = v172_16_200_86_GIS_sde__5_
v172_16_200_86_GIS_NETWORK_sde = "Database Connections\\172.16.200.86_GIS_NETWORK.sde"
arcpy.SynchronizeChanges_management(v172_16_200_86_GIS_sde__5_, "PE_Hat_09082018", v172_16_200_86_GIS_NETWORK_sde, "FROM_GEODATABASE1_TO_2", "IN_FAVOR_OF_GDB1", "BY_OBJECT", "DO_NOT_RECONCILE")
arcpy.Compress_management(v172_16_200_86_GIS_GISdbsa_sde)
arcpy.DisconnectUser('Database Connections\\172.16.200.86_GIS_GISdbsa.sde', "ALL")
arcpy.AcceptConnections('Database Connections\\172.16.200.86_GIS_GISdbsa.sde', True)

Re: Null/Empty Geometry in Python

JoshuaBixby — Sat, 11 Dec 2021 08:32:04 GMT

Fortunately for you, the ArcPy Data Access cursors return Python None whether there is an empty geometry or the field is NULL (empty geometry and NULL are not the same, but the ArcPy DA cursors treat them the same). Something like the following code will remove all rows where the cursor returns a None.

fc = # path to feature class
with arcpy.da.UpdateCursor(fc, "SHAPE@") as cur:
    for shp, in cur:
        if shp is None:
            cur.deleteRow()‍‍‍‍‍

Re: Null/Empty Geometry in Python

GIS_Solutions — Thu, 15 Nov 2018 07:32:59 GMT

Thank you so much

This code works well, but it gives error when there are users. I'm throwing users like this (arcpy.DisconnectUser('Database Connections\\172.16.200.86_CBS_cbsdbsa.sde', "ALL")) can't access the code at this time.

How can I do this after killing users?

@Joshua Bixby