topic Feature classes locked by ArcGIS server services in Geoprocessing Questions

Feature classes locked by ArcGIS server services

JacksonTrappett — Wed, 21 Apr 2010 16:45:05 GMT

I have several models that we use to update our datasets on a weekly or monthly basis due to changes in parcels and new annexations, etc. The final output of these models overwrites production feature classes in our SDE geodatabase. These same feature classes are used by our ArcGIS Server services for our web maps. Of course every time we run the models I first have to stop the services so that the SDE datasets are not locked while they run/overwrite.

What I'm looking for is one of two things:
A. A way in model builder to stop and then restart the services only during the part of the model that is overwriting the feature class. Can services be stopped and started in Model builder? I haven't seen a way yet. This option would allow me to run our large models (some of them take 30 to 45 minutes to run) without having to have the services stopped for the entire time, which I don???t like to do because it makes them unavailable to customers for the duration.

B. A better publishing model for ArcGIS Server that doesn't lock the datasets. The only one I've been able to come up with so far would be to have two copies of the same SDE database. One of them would be the one we update with our models and edit, and the other would be the one the ArcGIS server services are hooked into. The two would be linked by one way replication, which can synchronize the changes without having to stop the services. This seems like the best option to me but it requires quite a bit of work to create and maintain another SDE geodatabase.

Anyone have ideas for me on either A or B or both? If A is possible I will probably just go with it since it is much easier to set up and maintain.

Feature classes locked by ArcGIS server services

KevinHibma — Thu, 22 Apr 2010 16:08:31 GMT

Jackson,

You'll probably address this issue more on the Server side than you will with your Models.

Have a look at this help topic: http://help.arcgis.com/en/arcgisserver/10.0/help/arcgis_server_dotnet_help/index.html#//0093000000mr000000.htm
and search for: <SchemaLockingEnabled>
(this is a 10 topic but this specific point should be the same for 9.3.1).

As stated, use caution when using this tag. Though it sounds like you're already aware of the risks and what you need to do.

Hope this helps.

Feature classes locked by ArcGIS server services

JacksonTrappett — Thu, 22 Apr 2010 20:21:56 GMT

Kevin, thank you for the reply.

I took a look at the <SchemaLockingEnabled> property of the service configuration and it seems like a good solution but I finally settled on using the AGSSOM tool inside a script in my model to stop and start the service while the overwrite is taking place (as suggested by Andrew in the ArcGIS Server General forum.)

Here's a link to the AGSSOM tool for anyone interested in this functionality:
http://resources.esri.com/arcgisserver/adf/dotnet/index.cfm?fa=codeGalleryDetails&scriptID=16293

Re: Feature classes locked by ArcGIS server services

GaryBraun — Tue, 01 Jun 2010 17:37:57 GMT

I'm having virtually identical problem as the one posted by Jackson, it's a major P.I.T.A. to refresh data in SDE once it' s being consumed by a web service so I'm looking forward to trying out the <SchemaLockingEnabled> in 10.

I do have one follow up question along the same lines though; why would I have schema locks on an feature class that isn't being used by any of my web services? At least it shouldn't be used by the web services because it isn't referenced in any of the MXD files they are using, yet I can't delete the feature class unless I shut down all of my ArcGIS services first. What's up with that?

Re: Feature classes locked by ArcGIS server services

MeleKoneya — Thu, 03 Jun 2010 02:33:00 GMT

Like others in this thread, we have been having some issues updating our SDE feature classes when they are used by ArcGIS Server Map Service. I just tried adding the schemalockenabled tag set to false on a map service on ArcGIS Server 9.3.1 SP1 and was not able to update the feature class as it returned a lock on the feature class. We have been using the 'sledge hammer' approach of stopping all ArcGIS Map Services while loading data, but it would be nice to not have to stop these services.

I am wrong in believing that this functionality was added in SP1?

We also break connections to our ArcSDE databases using SDEMON -o kill, but this does not work against the ArcGIS Map Services so we have to stop them.

I am curious as to how others are dealing with these locking issues so let me know if anyone in this thread has made some progress in dealing with this.

Thanks,

Mele

Re: Feature classes locked by ArcGIS server services

JasonWise — Mon, 07 Jun 2010 14:30:54 GMT

I think you can avoid all these issues by deleting and appending features instead of overwriting a whole feature class.

If you run a tool that replaces a feature class, then you're actually deleting a feature class and creating a new one. In other words, you're changing the database schema, even though the schema ends up looking just like what you started with (same feature class and fields).

An alternative is to keep the existing feature class, delete the features from it, and append new features into it. This way, you don't need a schema lock, even though the end result is the same as the other method.

Re: Feature classes locked by ArcGIS server services

RobertHu — Mon, 07 Jun 2010 17:55:02 GMT

Hi Jason,

Thanks for your info.

I'm currently using sde commands in shell script to update those locked FCs,
sdetable -o truncate ...
sdeexport -o create ... | sdeimport -o append ...

Our data maintenance people don't have access to the sde commands. I've been looking for a way so that they can do the same thing in ArcCatalog, but never realized the Delete feature and Append tools can do it.

Thanks again!

Re: Feature classes locked by ArcGIS server services

MeleKoneya — Mon, 07 Jun 2010 19:18:44 GMT

Thanks Jason.

I have rewritten my import code to Delete and then Append. By doing so it alleviates the Locking issues both with ArcGIS Server and ArcMap Desktop clients accessing the data. Great suggestion.

Re: Feature classes locked by ArcGIS server services

BartPouteau — Thu, 10 Jun 2010 16:44:10 GMT

I think you can avoid all these issues by deleting and appending features instead of overwriting a whole feature class.

If you run a tool that replaces a feature class, then you're actually deleting a feature class and creating a new one. In other words, you're changing the database schema, even though the schema ends up looking just like what you started with (same feature class and fields).

An alternative is to keep the existing feature class, delete the features from it, and append new features into it. This way, you don't need a schema lock, even though the end result is the same as the other method.

I'm having similar problems and was considering this approach. My concern is that since our data is versioned, repeatedly deleating records only to repopulate them along with a few changes is going to have a significant impact on the size of our delta tables. Can anyone using this method with versioned data comment on how significant the impact actually is?

Thanks.

Bart

Re: Feature classes locked by ArcGIS server services

RandyKreuziger — Thu, 10 Jun 2010 17:13:30 GMT

Bart,
You don't have to version the feature classes that have there records deleted and appended. There is no reason to that I can think off.

Re: Feature classes locked by ArcGIS server services

BartPouteau — Wed, 16 Jun 2010 16:28:08 GMT

Randy,

I may be misunderstanding your reply, but the data in questions wasn't versioned to accommodate this process. It was versioned for other business process reasons.

Bart

Re: Feature classes locked by ArcGIS server services

JaredAndre — Wed, 16 Jun 2010 17:25:09 GMT

My recommendation would be to work within the versioning framework, programmatically creating a child version to add/delete/modify your features, especially if versioning is already a business process requirement. You can then even run an automated reconcile/post process to push the updated records up to the ArcGIS server 'display' version, or alternatively run the updated features through your normal QA/QC processes. It's a little more overhead than simply overwriting feature classes in their entirety, but works pretty well and fits nicely within the standard versioning/business process without impacting other ArcGIS data operations.

Re: Feature classes locked by ArcGIS server services

MeleKoneya — Tue, 22 Jun 2010 14:23:53 GMT

I created the following to append features

       public static void AppendFeatures(string inFC, string outFC)
        {
            ESRI.ArcGIS.Geoprocessor.Geoprocessor GP = new ESRI.ArcGIS.Geoprocessor.Geoprocessor();
            Append AppendFeatures = new Append();
            AppendFeatures.schema_type = "NO_TEST";
            AppendFeatures.inputs = inFC;
            AppendFeatures.target = outFC;
            RunTool(GP, AppendFeatures, null);
        }

RunTool is calling a method with code that Chad Cooper wrote and is available at http://www.esri.com/news/arcuser/0410/gpandnet.html

The code above works great on everything but Annotation,    any suggestions on how to append to Annotation?    Threre is AppendAnnotationFeatureClasses Tool, but it creates a new feature class from inputs rather than appending to an existing feature class.

Thanks,

Mele

Re: Feature classes locked by ArcGIS server services

TamaraRosenzweig — Thu, 02 Jun 2011 23:44:49 GMT

Like others in this thread, we have been having some issues updating our SDE feature classes when they are used by ArcGIS Server Map Service. I just tried adding the schemalockenabled tag set to false on a map service on ArcGIS Server 9.3.1 SP1 and was not able to update the feature class as it returned a lock on the feature class. We have been using the 'sledge hammer' approach of stopping all ArcGIS Map Services while loading data, but it would be nice to not have to stop these services.

I am wrong in believing that this functionality was added in SP1?

We also break connections to our ArcSDE databases using SDEMON -o kill, but this does not work against the ArcGIS Map Services so we have to stop them.

I am curious as to how others are dealing with these locking issues so let me know if anyone in this thread has made some progress in dealing with this.

Thanks,

Mele

Did you restart the SOM on the server after you updated the cfg files? If you don't, it won't recognise the addition of the schemalockingenabled tag.

Tamara

Re: Feature classes locked by ArcGIS server services

chapcole — Mon, 01 Jul 2013 15:34:33 GMT

we have faced a similar issue at my office, depending on the process and the what the intentions of your results are I would recommend establishing the workspace and the scratch workspace of the model to be on a local drive. This allows you to work seperate from SDE and can prevent from having model intermediates or outputs from locking down services. I actually recommend creating variables and setting them as parameters so that for each time you use the model you can establish a new work space. It may create some redundancy issues but as long as you are aware of it I see no real problem. If you are not familiar with enviroments and workspaces it might be good to check out esri's advanced model builder book.