POST
|
I found this Testing with ArcPy: Isolation and Mocking· Notes from the Lifeboat Thanks folks.
... View more
11-22-2017
01:44 AM
|
0
|
0
|
366
|
POST
|
Does anyone out there have any good examples, tips and tricks for practising Acceptance Test Driven Development (ATDD/BDD) and Test Driven Development (TDD) with GIS in a Continuous Integration (CI) environment? In the most part when I develop software, I write Acceptance Tests first using Gherkin and Cucumber to execute them. I then make the code in whatever language is appropriate (usually Python) and use unit tests to ensure the code is working. The problem in a GIS environment is creating test cases that can be arranged, actioned and asserted quickly in order to reap the benefits of fast feedback in an Agile team environment. Much of the Continuous Integration documentation and online material suggest that the total execution time from check in of code to VCS to detecting a broken build (either failed compilation or failed tests) is around 10-15 minutes. With ArcGIS and GIS in general, I find it can take an age to even do the most basic tests - just loading arcpy in a python script can take upto 20 seconds which seriously puts constraints on your fast feedback loops. To overcome this in most programming environments, one could create a mock that would substitute the behaviour however I'm finding this difficult to achieve for all but the simplest scenarios or those that interact with ArcGIS Server's REST API (because you can mock service end point easily). If tests take too long to execute, developers put off the testing and check in of their code to VCS and CI, which delays integration, reduces transparency, wastes time, reduces quality and potentially increases time to market (TTM).
... View more
11-16-2017
02:45 AM
|
0
|
1
|
901
|
POST
|
Hi Dave, I think I've discovered the insanity that is causing this to perform so badly. During the publication process, ArcGIS appears to be copying (yes, copying!) each raster to a subfolder beneath C:\Users\user\AppData\local\ESRI\Desktop10.1\Staging\connection this is cleaned up at the end but it's a complete waste and (depending on your environment) can impede your ability to publish more than one service at once. Why it is doing this I do not know nor can I fathom! In my case, this is copying roughly 40GB of data one raster at a time from the shared network file location which is a) registered with the server and b) visible to the server. If you create a Service Definition without connecting to the server, it still takes as long but there doesn't appear to be any of the copying. I'd love to know what Esri say about this! Craig
... View more
10-20-2014
03:30 AM
|
0
|
1
|
870
|
POST
|
Hi Dave, I didn't get any response to this posting other than yours. I've just learned to accept that it is just going to take this long and I work it into my day somehow. Yesterday in fact, I republished the services after some updates and it took just over an 1.5 hours. I believe a Mosaic Dataset would be faster if you have AGS Advanced but we don't (yet) so I can't test this. My catalogues are all unmanaged but I might see what happens if I make them managed? It would be nice to get a commentary from Esri as to what it is doing when it publishes a raster catalogue. I believe that it's not just testing for the existence of each raster but opening each raster and testing it for something. Craig
... View more
10-14-2014
02:15 AM
|
0
|
3
|
870
|
POST
|
Hi William, I've just come across this post regarding slow publishing for map services with raster layers. I'm experiencing pitiful performance when I publish my map service that contains 4 layers based up different raster catalogues. The paths to the rasters in the catalog are to a UNC share which is visible to and registered with the server. The path to the actual catalog is on the same UNC as the rasters. The 4 raster catalogs have 56, 815, 2285 and 10951 rasters respectively. The process is still extremely slow even though analysis confirms that no files need to be copied to the server - the more rasters I have in the catalog the slower this is. Publishing the service takes several hours (2+ hours). The network is 10Gbps, server is a virtualised (VM Ware) 2-node configuration in a single arcgis server instance each with 40GB RAM and 4 processors. The workstation publishing the service has 128GB RAM, 48-processors and masses of free disk space (>1TB). Any insight you could lend would be useful. Craig
... View more
09-15-2014
08:05 AM
|
0
|
5
|
870
|
Online Status |
Offline
|
Date Last Visited |
11-11-2020
02:24 AM
|