Extreme difference in identify times between 100.2 and 100.4

1460
11
03-26-2019 11:26 AM
JoeHershman
MVP Regular Contributor

I have an application that was upgraded to the 100.4 version of the API.  In testing this application I see huge difference in the overall performance of the identify.  What is odd is that not all layers perform worse, some actually seem to perform better.  However, the ones that perform worse are significant (4 seconds in one case, 1.5 in another).  While I have not done an exhaustive evaluation of the layers themselves, it would seem that layers with more data a decrease in performance of the largest amounts.

What is going on here?  Because of the magnitude of the increase with specific layers I have an identify operation go from 0.6 seconds to 7.5 seconds.  It should go without saying that having a user wait 7 seconds for identify results is not acceptable

I also observe a noticeable difference in the performance of a polygon search on a large offline database although I have not logged the actual performance.

These tests are using the same offline replicas, identifying the same layers. The only difference being, I changed the API from 100.4 to 100.2 (and fixed the breaking changes)

Layer NameTime Seconds (API = 100.4)Time Seconds (API = 100.2)Increase (ms)Identify Time Change
Gas Leaks0.07095870.0839489-1.29902Decrease
Repaired Gas Leaks0.2568430.007996724.88463Increase
Change Request0.00699620.0179866-1.09904Decrease
Meters1.54305910.0169923152.60668Increase
Regulator Station0.01898870.01699070.1998Increase
Compressor Station0.01199330.0171815-0.51882Decrease
Critical Valves0.0639610.01799094.59701Increase
Non Critical Valves0.17789030.018987515.89028Increase
Test Stations0.00899480.0149901-0.59953Decrease
Coupon0.01499130.0169892-0.19979Decrease
Relief Valve0.01599030.0169946-0.10043Decrease
Pressure Monitoring Device0.01998760.01798430.20033Increase
Rectifier0.0259840.01598990.99941Increase
Odorizer0.01499060.0159895-0.09989Decrease
Pig Structure0.01998650.0159910.39955Increase
Sniff Test Location0.02598560.01898870.69969Increase
Locator Device0.06695750.1082403-4.12828Decrease
Exposed Pipe0.01099390.01099390Decrease
Encroachment0.01699040.01698968E-05Increase
WOF0.01598940.0169907-0.10013Decrease
TFIR0.01499060.01699-0.19994Decrease
Drip0.03497960.01698841.79912Increase
Crossing0.0269850.01698950.99955Increase
Fittings0.83748830.027983580.95048Increase
Rectified Pipe4.1904450.0079933418.24517Increase
Gas Pipe Casing0.01698890.0169911-0.00022Decrease
Customer Service Zones0.01699030.0189886-0.19983Decrease
Shop Location0.01598990.0189872-0.29973Decrease
Total7.56338980.6271185

Thanks

-Joe

Thanks,
-Joe
0 Kudos
11 Replies
MichaelBranscomb
Esri Frequent Contributor

Joe,

A couple of questions:

- What layer type(s) is the identify operation running against?

- What specific identify operation and overloads are you using?

Thanks

Mike

0 Kudos
JoeHershman
MVP Regular Contributor
private async Task IdentifyByBoundMap(MapView mapView, GeoViewInputEventArgs args, double tolerance, List<IdentifyLayerResult> identifyLayerResults)
{
	var start = DateTime.Now;
	Log.Info($"Start Time: {start}");
	//This is so result set is in map TOC order
	for (int i = Map.OperationalLayers.Count - 1; i >= 0; i--)
	{
		FeatureLayer mapLayer = Map.OperationalLayers[i] as FeatureLayer;

		if ( mapLayer?.PopupDefinition == null || mapLayer.IsVisible == false ) continue;
		if ( mapLayer.FeatureTable.TableName == null ) continue;

		var startLayer = DateTime.Now;
		var result = await mapView.IdentifyLayerAsync(mapLayer, args.Position, tolerance, false, 5);

		var endLayer = DateTime.Now - startLayer;
		Log.Info($"{mapLayer.Name}: {endLayer.TotalSeconds}");

		identifyLayerResults.Add(result);
	}

	var timeSpanAll = DateTime.Now - start;
	Log.Info($"Identify All Time: {timeSpanAll.TotalSeconds}");
}‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

The tolerance is 10. 

As mentioned below, it would seem that line features (Db about 650 MB) are the ones that show the extreme increase.  Meters which is a point asset but is large (over 1 GB) shows a large increase but nothing like the lines.  I also did identify on other .geodatabase files for linear assets and these also show a 4 second time.  We have multiple line layers that are the same actual data, just symbolized differently.  Also because of how server synchronizes data we are required to have all the layer the data on the client or data won't sync correctly from the server  

The entire class is attached if that would be helpful.  This is implemented as a TriggerAction attached to the MapView

Thanks,
-Joe
0 Kudos
dotMorten_esri
Esri Notable Contributor

So these are all local feature layers? If so, we should have some good improvements for Update 5 regarding query performance on local data, which would likely help identify too. Also see https://community.esri.com/message/821017-query-performance#comment-824863 

0 Kudos
JoeHershman
MVP Regular Contributor

That is nice to hear but my question would still be do you understand what would have changed to have that significant a hit on performance.  I am looking at a 500x increase in time to identify these specific layers (lines in large offline replicas seem to be the huge hit) after upgrading the API.  If folks are unclear on what the issue is I am concerned on if it will be resolved.  Currently am in a position that we can not deliver the application using the 100.4 API because of this issue.  

Thanks,
-Joe
0 Kudos
JoeHershman
MVP Regular Contributor

Michael Branscomb‌ I sent this info in email to Morten also.  But what I have found during further testing is the the huge hit is associated to us having a definition expression on the layer.  I also noticed in other testing how much impact a definition expression has in general on redraw. 

We are required to have definition expression attached to every layer because of the shortcomings in the ArcGIS Server sync model.  [The issue is that if a value in the database is changed to no longer meet the maps's definition expression or the query used when the replica is generated it will not sync.  This is considered to be 'works as designed' and as far as I know there is no plan to change]  Because of this we are required to bring all features onto the client and apply a definition expression so that the features will still sync.

Thanks,
-Joe
0 Kudos
JoeHershman
MVP Regular Contributor

Doing a little more evaluation and it seems that the most incredible negative difference is with certain line features, the databases in these cases are also rather large at about 650 MB,

Thanks,
-Joe
0 Kudos
by Anonymous User
Not applicable

Is 100.4 sending different http requests to the server?

(You can compare traffic using ArcGISHttpClientHandler.HttpRequestBegin)

0 Kudos
JoeHershman
MVP Regular Contributor

This is a local replica, not a ServiceTable.  I guess I just have to wait on 100.5 and hope for the best, but I find it concerning that no one has indicated that they understand why the performance is so extremely degraded.  As of now cannot release the upgraded application.

Thanks,
-Joe
0 Kudos
dotMorten_esri
Esri Notable Contributor

OK so I've been running various Identify Performance Benchmark against Geodatabase for v100.2, 100.2.1, 100.3, 100.4 and latest 100.5 build, and I'm not able to reproduce the big slowdown. 

I'm actually seeing IdentifyLayers to become about 15% faster in 100.2.1, and a small performance drop in IdentifyLayer at 100.3, but nothing that matches the size of performance change you're experiencing.

That leads me to believe it might be data specific - is there any chance this data could be shared with us, so we can try and reproduce?

I'm using BenchmarkDotNet to run against 38 various layers all in a local geodatabase, using the following code:

        [Benchmark, System.STAThread]
        public Task<int> IdentifyAllLayers()
        {
            return RunOnUIThread(async () =>
            {
                var result = await mapView.IdentifyLayersAsync(new System.Windows.Point(WindowWidth / 2, WindowHeight / 2), 10, false).ConfigureAwait(false);
                if (result.Count == 0)
                    throw new System.Exception("Expected one layer returning");
                return result.Count;
            });
        }

        [Benchmark, System.STAThread]
        public Task<int> IdentifyEachLayer()
        {
            return RunOnUIThread(async () =>
            {
                int count = 0;
                foreach (var layer in map.OperationalLayers.OfType<FeatureLayer>())
                {
                    var result = await mapView.IdentifyLayerAsync(layer, new System.Windows.Point(WindowWidth / 2, WindowHeight / 2), 10, false);
                    count += result.GeoElements.Count;
                }
                if (count == 0)
                    throw new System.Exception("No identify results");
                return count;
            });
        }

Btw I suggest you use IdentifyLayersAsync to identify all layers in the map, as this seems to be a lot faster than doing an identify on each layer one by one (In my case about a factor of 10 !).

0 Kudos