topic Re: Schema lock woes in ArcGIS Pro 2.3 in Python Questions

Schema lock woes in ArcGIS Pro 2.3

EricEagle — Sat, 11 Dec 2021 06:39:07 GMT

Can someone tell me in my code where I'm not properly dismissing schema locks? Currently when I attempt to run the tessellation, I get ERROR 000464: Cannot get exclusive schema lock. Either being edited or in use by another application.

In this case I am not running the application, only the IDE. What am I missing?

Background: I am taking huge point data files that come to me unprojected, automating a UTM zone projection to put it into metric units, and then making a determination of whether the extent is so big (it fluctuates) that it warrants splitting into smaller chunks, which I will do through tessellation. I am not using minimum bounding geometry because Esri has helpfully limited the Envelope functionality to advanced licenses, and all of my users are on standard licenses

import arcpy

def get_vector_center(vector):
    sample_mean = "sample_mean"
    sample_center = arcpy.MeanCenter_stats(vector, "in_memory/{}".format(sample_mean))
    return sample_center


tempgdb = "C:/Temp/scratchdb.gdb"
input_sample = "C:/Temp/dots.shp"

arcpy.env.workspace = tempgdb
arcpy.env.overwriteOutput = True

# prep data
utm_fc = "C:/Temp/foundation.gdb/utm_zones_with_epsg"
center = get_vector_center(input_sample)

# do a join
utm_zone = arcpy.SpatialJoin_analysis(utm_fc, center, "in_memory/zone", "#", "KEEP_COMMON", "#", "CONTAINS")

# get the zone
with arcpy.da.SearchCursor("in_memory/zone", ['EPSG']) as vcursor:
    for row in vcursor:
        epsg = (int(row[0]))
print("Got EPSG code {}".format(epsg))

# construct the sr object
sr = arcpy.SpatialReference(epsg)
sr.factoryCode = epsg
sr.create()

# project the original content with the right utm zone
output_layer = "output_utm"
fixed_data = arcpy.Project_management(input_sample, output_layer, sr)

# now manually create envelope in correct projection since no advanced license

e = arcpy.Describe(output_layer)

xmin = e.Extent.XMin
ymin = e.Extent.YMin
xmax = e.Extent.XMax
ymax = e.Extent.YMax

coordinates = [(xmin, ymin), (xmin, ymax), (xmax, ymax), (xmax, ymin)]
polygon = arcpy.CreateFeatureclass_management(tempgdb, "poly_from_extent", "POLYGON", spatial_reference=sr)
feature_class = polygon[0]

# write the geometry from extent to create envelope

with arcpy.da.InsertCursor(feature_class, ['SHAPE@']) as cursor:
    cursor.insertRow([coordinates])

o = arcpy.Describe(feature_class)

# now check for >100km2 area to see if we should tesselate for faster processing

t_out = "parent_tgrid"
oe = o.Extent

with arcpy.da.SearchCursor(feature_class, 'SHAPE@AREA') as acursor:
    for row in acursor:
        if row[0] > 100000000:
            print("{} greater than 10K km2, splitting...".format(row[0])
            tgrid = arcpy.GenerateTessellation_management(t_out, oe, "SQUARE", 100000000, sr)
            print("Splitting complete")
        else:
            print("point extent smaller than 10K km2, not splitting.")


‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Schema lock woes in ArcGIS Pro 2.3

JoshuaBixby — Mon, 06 May 2019 22:11:23 GMT

Can you elaborate on:

Either being edited or in use by another application.

Are you saying that another application is working with the data sources while your script is running?

Re: Schema lock woes in ArcGIS Pro 2.3

EricEagle — Mon, 06 May 2019 22:17:55 GMT

Hi Joshua,

That's just part of the standard error message. I am only running this in an IDE (PyCharm in this case). So no, ArcGIS Pro/ArcMap are not active or accessing any of this data.

Re: Schema lock woes in ArcGIS Pro 2.3

LukeWebb — Sat, 11 Dec 2021 06:39:09 GMT

Im trying to understand what your achieving with this code:

with arcpy.da.SearchCursor(feature_class, 'SHAPE@AREA') as acursor:
    for row in acursor:
        if row[0] > 100000000:
            print("{} greater than 10K km2, splitting...".format(row[0])
            tgrid = arcpy.GenerateTessellation_management(t_out, oe, "SQUARE", 100000000, sr)
            print("Splitting complete")
        else:
            print("point extent smaller than 10K km2, not splitting.")

To me, you need to do something with TGrid, otherwise you are just overwriting each iterations previous output. I think this is also what causes your issue as you are still "Referencing" the output in the python script, so it will lock the shapefile maybe.

You could test this theory, by changing it to:

with arcpy.da.SearchCursor(feature_class, 'SHAPE@AREA') as acursor:
    for row in acursor:
        if row[0] > 100000000:
            print("{} greater than 10K km2, splitting...".format(row[0])
            tgrid = arcpy.GenerateTessellation_management(t_out, oe, "SQUARE", 100000000, sr)
            print("Splitting complete")
            del(tgrid)
        else:
            print("point extent smaller than 10K km2, not splitting.")

Re: Schema lock woes in ArcGIS Pro 2.3

EricEagle — Tue, 07 May 2019 14:33:53 GMT

Hi Luke,

Today after a reboot I'm not experiencing the schema lock. I wonder if there was a zombie ArcMap process locking the fgdb, since I had earlier examined the output using ArcMap (though I did disconnect/remove the database from the map viewer).

In answer to your other question:

row[0] represents the envelope. It will only ever have one row so while it looks like it's iterating, it's really not. It feels kind of duct tapey and there is probably a smoother way to do it, but when it works it gives me the result I want with very little overhead.

What I'm actually doing is checking the size of that envelope in m2, and if it is over a certain size, create a tessellation grid covering the extent. Then I will split each feature into its own feature class and iterate the analysis over those chunks. I feel like this is easier than multiprocessing but maybe not. The whole thing driving this is that the tools I am using:

* Generate Tessellation (building 50m radius hex cells to aggregate very dense points)

* Summarize Within (doing the sum for each cell)

are IO bound, serial, single-processor tools and are not robust in terms of environments (e.g., Generate Tessellation could be more efficient by being done by mask, but it is only available to issue it an extent, which is very inefficient). So even though I have 36 cores available, these tools will take a ridiculously long time to process. I have found that splitting a huge extent into smaller chunks buys me some marginal performance gains (also if it fails I don't lose everything).