csny490

ArcGIS NVMe disk performance?

Discussion created by csny490 on May 8, 2017
Latest reply on May 17, 2017 by Dan_Patterson

Anyone running a machine with these newfangled NVMe disks?

 

We purchased a newer workstation with some of these, and while the "industry benchmark" tests we have run(CrystalMark) indicate the disks are indeed super fast and operating as expected, the real world test that we have run in ArcGIS and Python have not shown a significant performance increase over old school SATA HDD platter disks. 

 

Anyone know why this might be the case? My only assumption is that disk speed is not the bottle neck... If it helps, here's an excerpt of some of the tests we are running. Note that not all of these involve use of disk i/o and there are some others (not shown) that are just simple unions and dissolves and stuff:

 

#Process: Export raster to pnts (about 2 million points)
rasterPntsFC = os.path.join(fgdbPath, "raster_pnts")
time1 = time.clock()
arcpy.RasterToPoint_conversion(conRst, rasterPntsFC, "VALUE")
time2 = time.clock()
benchmarkDict["RASTER_TO_POINTS"] = time2 - time1
logMessage("RASTER_TO_POINTS = " + str(time2 - time1))

 

#Process: Build a large dictionary independent of disk
time1 = time.clock()
randomDict = {}
sum = 0
i = 0
for x in range(1,2001):
    for y in range(1,2001):
        i = i + 1
        randomDict[x,y] = [random.randint(1,1000)]
        sum = sum + randomDict[x,y][0]
        randomDict[x,y].append(sum / float(i))
time2 = time.clock()

 

del randomDict
benchmarkDict["BIG_DICTIONARY"] = time2 - time1

 

#Make another dictionary, but one sourced from disk
time1 = time.clock()
pntDict = {r[0]:r[1] for r in arcpy.da.SearchCursor(rasterPntsFC, ["OID@", "grid_code"])}
time2 = time.clock()
benchmarkDict["POINTS_TO_DICT"] = time2 - time1

 

#Sort something pretty big
time1 = time.clock()
sortList = sorted(pntDict.items(), reverse=True)
sortList.sort()
time2 = time.clock()
benchmarkDict["SORT_DICT_ITEMS"] = time2 - time1

 

#Write a big txt file
time1 = time.clock()
testTxtFile = os.path.join(benchmarkFolderPath, "test_text_file.txt")
f = open(testTxtFile, 'a')
for i in range(10000000):
    f.write(str(i))
f.close()
time2 = time.clock()
benchmarkDict["WRITE_BIG_TXT_FILE"] = time2 - time1

Outcomes