Custom SOE Slow

[MiklosNadas]

We had a custom SOE solution developed to perform a trace on a geometric network of pipes, valves, connections, and hydrants. Our service area has over 500,000 connections, 170,000 pipes, 150,000 valves, and 80,000 hydrants. This SOE integrates with Cityworks, so we can log valves being turned on and off. We are hoping that field crews can utilize this tool to check which properties are affected by valve shut offs.

Each time a valve is turned on or off, the code runs a geometric trace on the entire network. This trace takes about 2 minutes to run. We would like to decrease the amount of time this tool takes to run. It appears that the geometric trace is the function that takes a majority of the 2 minutes to run, so from a coding perspective, there doesn't seem to be a way to make the code more efficient (except to isolate the trace to run on a subset of the network rather than the entire network). We may consider this option.

After reviewing documentation on fine tuning ArcGIS Server and feature services, I couldn't find anything about improving response time for custom SOEs.

Is there a way to add more resources (CPU? RAM?) and dedicate it to a specific SOE?

If I added more RAM or CPUs to the server, would that improve the tool's efficiency?

If I added an additional ArcGIS Server, would that speed it up?

Are there any other recommendations?

Our server is at 10.6.1

Thanks

[nicogis]

If the same trace done in arcgis desktop takes 2 minutes the soe cannot improve the response.

what's the dbms ? Have you tried tuning on dbms ?

In soe you can use costruct event  to inizialize something to prepare your trace

RAM or CPUs to the server depends if your server when you use service of soe use actual ram / cpu in % ( http://www.wiki.gis.com/wiki/index.php/Platform_Performance  )

Can you give further details ?

[MiklosNadas]

Hey!

Thanks for your response. After looking at the code a little more closely, I realize I was confused by their messages of "Trace Started" and "Trace Ends". It turns out there was a lot of other things going on besides network tracing.

I did follow your suggestion and ran a few traces using ArcMap. These traces happened almost instantaneously. So, I dove into the code a bit more and realized that the code starts by setting all the pipes, valves, hydrants, and connections in the entire network and sets their status to "On". It then gets all the pipes, valves, hydrants, and connections that are returned from the trace as disconnected and sets their status to "Off". It's these two queries and edit sessions that are taking the bulk of the "trace".

Yes the dbms is tuned and we use MS SQL Server. I will look into getting these edit sessions to be more and try to find a way to make it a bit more efficient.

[nicogis]

Have you discover using SQL Server Profiler the problem ?

[MiklosNadas]

My DBA ran SQL Server Profiler and indicates there is one query that runs for 40 seconds. After investigating, the query selects all the connections that are disconnected from the geometric network (which turns out to be quite a few) and updates a field for their status from "Connected" to "Disconnected".

He said that nothing else seems to be out of place and the database is tuned. He assumes it's a code issue.

[nicogis]

query in 40 sec? Can you give details on this query ?

Should see the source code of soe...

Have you in costruct event  put code to initialize network / prepare request ect ?

[MiklosNadas]

Because it's a network trace tool, it's supposed to show every connection that is disconnected. We have a few connections that are outside the network and are immediately flagged as disconnected (such as abandoned connections or connections connected to abandoned pipes). I have had to delete most of these connections because it returns a massive amount of connections, about 50k. After deleting most of the easy connections, this SQL query now returns about 20k and it updates the status to "disconnected". A 20k query and update still takes quite a long time unless you're directly connected to the SQL Server database.

At this point, I have determined the only option available to me is to modify the code and try to only run on the connections within the near area of the valve that is being turned off.

Thanks for all your help and ideas!