Hi Everyone,
We are currently in the process of rebuilding our virtual ArcGIS Servers on a Nimble Storage Adaptive Flash Array and I am seeing random performance problems. At this stage our SQLServer has been migrated to this storage and we have seen a decline in performance, and when comparing rendering times of imagery between the new and the old infrastructure there is an initial massive performance hit.
I know that there are lots and lots of other underlying networking hardware and OS configuration issues that could also be contributing to the issue, but at this stage I am just looking to track down any other ArcGIS Server users that are running ArcGIS Server on a Nimble so that we can start to get an idea if there any other factors that could be causing the performance issues or whether the Nimble isn't capable of handling this type of use case.
Also, as of this afternoon I am on leave for a few weeks so if any comments are left, it will be a few weeks before I can come back with any additional info etc.
Regards
Steve
I am not familiar with Nimble, but, you mentioned imagery, and I have to wonder something... When I was in the market for a Mac Mini a couple of years ago, I went with an SSD. Apple used two drives at the time, one manufactured by Samsung, and one manufactured by Toshiba - it was pretty much luck of the draw for whichever one they installed (you couldn't choose).
Problem was, the SandForce controller on the Toshiba SSD had quite a few issues and really took performance hits when dealing with encrypted or already compressed data. Not saying that's an issue for you, but there may be something in the set-up of Nimble that is causing you strife.
Can you run any benchmarks?
Hi Chris,
The IT guys tell me that it has 124Tb of disk storage and there is a 6Tb SSD cache in front of that. They have said that there is no way to allocate what is in cache and what isn't. My theory is that on the initial hit to the imagery the data is not in cache so it takes ages to drag it off the disk into cache and then serve it out. Once it is in cache it is faster than the NetApp. If you then run the test again it is quick. If you don't touch the imagery for a period of time it seems to get flushed from cache so the next time it is hit it takes a long time to drag it off the disk again and into cache. This was confirmed when running the tests 24hrs apart. Here are some bench marks that I have done. Some in PerfQA and some as raw disk benchmarks outside of any ArcGIS software.
In the PerfQA runs, look at the test iteration duration.
PerfQA - NetApp
---------------------------------------
RENDERING RESULTS
PerfQA Analyzer (Standalone) 10.3.0.168
---------------------------------------
Test Machine:
User:
ArcGIS Product: Desktop
ArcGIS Version: 10.3.1.4959
Map: H:\StorageSpeedTesting\NetAppSource.mxd
Date/Time: 19/05/2016 4:28:35 PM
---------------------------------------
Hardware Acceleration: Disabled
---------------------------------------
--- Iteration #1 ---
Extent #1 (XMin, YMin, XMax, YMax = 332183.623, 7187094.172, 463014.145, 7288313.999)
Rendering Start Time: [2016-05-19 16:28:35,316]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00047, 0.48561, 0.00125, 0.00048, 0.00002, 0.48784
Extent #2 (XMin, YMin, XMax, YMax = 388704, 7255838, 434159, 7282970)
Rendering Start Time: [2016-05-19 16:28:35,847]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.40605, 0.00124, 0.00048, 0.00002, 0.40795
Extent #3 (XMin, YMin, XMax, YMax = 432883, 7247799, 436236, 7249800)
Rendering Start Time: [2016-05-19 16:28:36,283]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 1.15738, 0.00128, 0.00047, 0.00002, 1.15932
Extent #4 (XMin, YMin, XMax, YMax = 393566, 7226441, 448302, 7259111)
Rendering Start Time: [2016-05-19 16:28:37,469]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.35397, 0.00127, 0.00049, 0.00002, 0.35591
Extent #5 (XMin, YMin, XMax, YMax = 409338, 7229189, 441658, 7248480)
Rendering Start Time: [2016-05-19 16:28:37,859]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.98752, 0.00127, 0.00050, 0.00002, 0.98948
Extent #6 (XMin, YMin, XMax, YMax = 322807, 7245887, 377543, 7278557)
Rendering Start Time: [2016-05-19 16:28:38,873]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.39867, 0.00123, 0.00048, 0.00002, 0.40057
Extent #7 (XMin, YMin, XMax, YMax = 432344, 7248328, 435018, 7249924)
Rendering Start Time: [2016-05-19 16:28:39,310]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.37011, 0.00129, 0.00059, 0.00003, 0.37217
Extent #8 (XMin, YMin, XMax, YMax = 437788, 7251023, 439102, 7251807)
Rendering Start Time: [2016-05-19 16:28:39,715]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 1.37626, 0.00129, 0.00049, 0.00002, 1.37823
Extent #9 (XMin, YMin, XMax, YMax = 434807, 7252405, 434974, 7252505)
Rendering Start Time: [2016-05-19 16:28:41,119]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.54258, 0.00128, 0.00049, 0.00002, 0.54453
Extent Minimum: 0.356, Maximum: 1.378, Average: 0.677
Test iteration duration: 00:00:06.403
--- Iteration #2 ---
Extent #1 (XMin, YMin, XMax, YMax = 332183.623, 7187094.172, 463014.145, 7288313.999)
Rendering Start Time: [2016-05-19 16:28:41,712]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00017, 0.32611, 0.00131, 0.00047, 0.00002, 0.32809
Extent #2 (XMin, YMin, XMax, YMax = 388704, 7255838, 434159, 7282970)
Rendering Start Time: [2016-05-19 16:28:42,086]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.35109, 0.00129, 0.00049, 0.00002, 0.35304
Extent #3 (XMin, YMin, XMax, YMax = 432883, 7247799, 436236, 7249800)
Rendering Start Time: [2016-05-19 16:28:42,508]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00017, 0.31580, 0.00126, 0.00049, 0.00002, 0.31773
Extent #4 (XMin, YMin, XMax, YMax = 393566, 7226441, 448302, 7259111)
Rendering Start Time: [2016-05-19 16:28:42,866]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.33235, 0.00126, 0.00048, 0.00002, 0.33427
Extent #5 (XMin, YMin, XMax, YMax = 409338, 7229189, 441658, 7248480)
Rendering Start Time: [2016-05-19 16:28:43,241]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.34847, 0.00126, 0.00048, 0.00002, 0.35039
Extent #6 (XMin, YMin, XMax, YMax = 322807, 7245887, 377543, 7278557)
Rendering Start Time: [2016-05-19 16:28:43,631]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.32566, 0.00126, 0.00048, 0.00002, 0.32758
Extent #7 (XMin, YMin, XMax, YMax = 432344, 7248328, 435018, 7249924)
Rendering Start Time: [2016-05-19 16:28:44,021]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.31893, 0.00122, 0.00048, 0.00002, 0.32081
Extent #8 (XMin, YMin, XMax, YMax = 437788, 7251023, 439102, 7251807)
Rendering Start Time: [2016-05-19 16:28:44,380]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.32969, 0.00127, 0.00049, 0.00003, 0.33164
Extent #9 (XMin, YMin, XMax, YMax = 434807, 7252405, 434974, 7252505)
Rendering Start Time: [2016-05-19 16:28:44,770]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014_25cm, 0.00016, 0.35147, 0.00126, 0.00049, 0.00002, 0.35340
Extent Minimum: 0.318, Maximum: 0.353, Average: 0.335
Test iteration duration: 00:00:03.462
PerfQA - Nimble
---------------------------------------
RENDERING RESULTS
PerfQA Analyzer (Standalone) 10.3.0.168
---------------------------------------
Test Machine:
User:
ArcGIS Product: Desktop
ArcGIS Version: 10.3.1.4959
Map: H:\StorageSpeedTesting\NimbleSource.mxd
Date/Time: 19/05/2016 4:28:55 PM
---------------------------------------
Hardware Acceleration: Disabled
---------------------------------------
--- Iteration #1 ---
Extent #1 (XMin, YMin, XMax, YMax = 332183.623, 7187094.172, 463014.145, 7288313.999)
Rendering Start Time: [2016-05-19 16:28:55,206]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00048, 0.42842, 0.00132, 0.00048, 0.00002, 0.43072
Extent #2 (XMin, YMin, XMax, YMax = 388704, 7255838, 434159, 7282970)
Rendering Start Time: [2016-05-19 16:28:55,674]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00017, 0.15836, 0.00148, 0.00048, 0.00002, 0.16050
Extent #3 (XMin, YMin, XMax, YMax = 432883, 7247799, 436236, 7249800)
Rendering Start Time: [2016-05-19 16:28:55,877]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00017, 1.32930, 0.00134, 0.00048, 0.00002, 1.33131
Extent #4 (XMin, YMin, XMax, YMax = 393566, 7226441, 448302, 7259111)
Rendering Start Time: [2016-05-19 16:28:57,234]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00016, 0.17589, 0.00122, 0.00050, 0.00002, 0.17779
Extent #5 (XMin, YMin, XMax, YMax = 409338, 7229189, 441658, 7248480)
Rendering Start Time: [2016-05-19 16:28:57,437]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00017, 1.31651, 0.00138, 0.00047, 0.00002, 1.31855
Extent #6 (XMin, YMin, XMax, YMax = 322807, 7245887, 377543, 7278557)
Rendering Start Time: [2016-05-19 16:28:58,778]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00016, 0.20759, 0.00135, 0.00048, 0.00002, 0.20961
Extent #7 (XMin, YMin, XMax, YMax = 432344, 7248328, 435018, 7249924)
Rendering Start Time: [2016-05-19 16:28:59,028]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00016, 72.45965, 0.00133, 0.00048, 0.00002, 72.46164
Extent #8 (XMin, YMin, XMax, YMax = 437788, 7251023, 439102, 7251807)
Rendering Start Time: [2016-05-19 16:30:11,541]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00017, 62.00180, 0.00135, 0.00048, 0.00002, 62.00382
Extent #9 (XMin, YMin, XMax, YMax = 434807, 7252405, 434974, 7252505)
Rendering Start Time: [2016-05-19 16:31:13,582]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00016, 8.20153, 0.00143, 0.00049, 0.00002, 8.20363
Extent Minimum: 0.161, Maximum: 72.462, Average: 16.255
Test iteration duration: 00:02:26.628
--- Iteration #2 ---
Extent #1 (XMin, YMin, XMax, YMax = 332183.623, 7187094.172, 463014.145, 7288313.999)
Rendering Start Time: [2016-05-19 16:31:21,834]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00017, 0.13879, 0.00130, 0.00049, 0.00002, 0.14076
Extent #2 (XMin, YMin, XMax, YMax = 388704, 7255838, 434159, 7282970)
Rendering Start Time: [2016-05-19 16:31:22,021]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00017, 0.25686, 0.00129, 0.00049, 0.00002, 0.25882
Extent #3 (XMin, YMin, XMax, YMax = 432883, 7247799, 436236, 7249800)
Rendering Start Time: [2016-05-19 16:31:22,333]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00017, 0.31134, 0.00131, 0.00048, 0.00002, 0.31333
Extent #4 (XMin, YMin, XMax, YMax = 393566, 7226441, 448302, 7259111)
Rendering Start Time: [2016-05-19 16:31:22,692]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00018, 0.21799, 0.00136, 0.00048, 0.00002, 0.22003
Extent #5 (XMin, YMin, XMax, YMax = 409338, 7229189, 441658, 7248480)
Rendering Start Time: [2016-05-19 16:31:22,957]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00015, 0.15674, 0.00131, 0.00049, 0.00002, 0.15871
Extent #6 (XMin, YMin, XMax, YMax = 322807, 7245887, 377543, 7278557)
Rendering Start Time: [2016-05-19 16:31:23,160]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00016, 0.15897, 0.00133, 0.00048, 0.00002, 0.16096
Extent #7 (XMin, YMin, XMax, YMax = 432344, 7248328, 435018, 7249924)
Rendering Start Time: [2016-05-19 16:31:23,363]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00016, 0.31232, 0.00130, 0.00049, 0.00002, 0.31429
Extent #8 (XMin, YMin, XMax, YMax = 437788, 7251023, 439102, 7251807)
Rendering Start Time: [2016-05-19 16:31:23,737]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00015, 0.11939, 0.00140, 0.00049, 0.00002, 0.12145
Extent #9 (XMin, YMin, XMax, YMax = 434807, 7252405, 434974, 7252505)
Rendering Start Time: [2016-05-19 16:31:23,909]
Layer, Background, Geography, Annotation, Selection, GeoSelection, Total
BRC_2014, 0.00016, 0.09907, 0.00129, 0.00048, 0.00002, 0.10102
Extent Minimum: 0.101, Maximum: 0.314, Average: 0.199
Test iteration duration: 00:00:02.236
And here is the results of the raw disk benchmark between the two.
It quite cleary shows that there is some sort of issue, but just need to find where.
So hence trying to find anyone else using Nimble to see what their performance is like and how they have the device configured.
Regards
Steve