Can't load more than 3 Local Map Service

[JeremieJoalland1]

I face limitation with the ArcGIS Runtime 10.1.1 samples "Add Shapefile" (issue is similar with "Add Raster").

I have just changed the source code in order to load a Local Tiled Package and used my own blank MPK to load Shpaefile/Raster.

When I run the sample application, if I try to load several Shapefiles, I always have the following Exception on the 4th file.
It's like my Local Server can't handle more than 3 worker processes at the same time...

...
2013-04-12 12:09:24,821: INFO  rid=31  'server'  mpk3 - Request handled.

2013-04-12 12:09:24,852: INFO  rid=32  'server'  mpk - Request handled.

2013-04-12 12:09:25,055: INFO  rid=33  'server'  mpk2 - Request handled.  -> Here we see that there are 3 localmapservice already loaded

2013-04-12 12:09:29,305: INFO  rid=34  'server'  AdminWebService - POST admin/createService -> Here I just select a 4th shapefile

java.lang.NullPointerException
 at com.esri.map.Layer.setUrl(Unknown Source)
 at com.esri.map.DynamicLayer.setUrl(Unknown Source)
 at com.esri.map.DynamicLayer.<init>(Unknown Source)
 at com.esri.map.ArcGISDynamicMapServiceLayer.<init>(Unknown Source)
 at com.esri.client.samples.datasources.LocalShapefileApp.addShapefile(LocalShapefileApp.java:103)
 at com.esri.client.samples.datasources.LocalShapefileApp.access$3(LocalShapefileApp.java:73)
 at com.esri.client.samples.datasources.LocalShapefileApp$3.actionPerformed(LocalShapefileApp.java:247)
 at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2018)
 at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2341)
 at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
 at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
 at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
 at java.awt.Component.processMouseEvent(Component.java:6505)
 at javax.swing.JComponent.processMouseEvent(JComponent.java:3321)
 at java.awt.Component.processEvent(Component.java:6270)
 at java.awt.Container.processEvent(Container.java:2229)
 at java.awt.Component.dispatchEventImpl(Component.java:4861)
 at java.awt.Container.dispatchEventImpl(Container.java:2287)
 at java.awt.Component.dispatchEvent(Component.java:4687)
 at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832)
 at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492)
 at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422)
 at java.awt.Container.dispatchEventImpl(Container.java:2273)
 at java.awt.Window.dispatchEventImpl(Window.java:2719)
 at java.awt.Component.dispatchEvent(Component.java:4687)
 at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:723)
 at java.awt.EventQueue.access$200(EventQueue.java:103)
 at java.awt.EventQueue$3.run(EventQueue.java:682)
 at java.awt.EventQueue$3.run(EventQueue.java:680)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
 at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
 at java.awt.EventQueue$4.run(EventQueue.java:696)
 at java.awt.EventQueue$4.run(EventQueue.java:694)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
 at java.awt.EventQueue.dispatchEvent(EventQueue.java:693)
 at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:244)
 at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:163)
 at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:151)
 at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:147)
 at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:139)
 at java.awt.EventDispatchThread.run(EventDispatchThread.java:97)
2013-04-12 12:09:35,071: INFO  rid=    ''  WebServer - Lost connection on port 50000  

Stopping server   -> I've closed the application from here
2013-04-12 12:09:37,790: INFO  rid=    ''  WebServer - Lost connection on port 50000

2013-04-12 12:09:37,790: INFO  rid=35  'server'  AdminWebService - POST admin/shutdown?f=json

2013-04-12 12:09:37,790: INFO  rid=    'server'  ServerMain - Shutting down

2013-04-12 12:09:37,790: INFO  rid=    'server'  ServerMain - Terminating web services

2013-04-12 12:09:37,790: INFO  rid=    'server'  AdminWebService - Terminating service blankMap.mpk

2013-04-12 12:09:37,790: INFO  rid=    'server'  mpk_host - Terminate

2013-04-12 12:09:37,790: INFO  rid=    'server'  AdminWebService - Terminating service blankMap.mpk2

2013-04-12 12:09:37,790: INFO  rid=    'server'  mpk2_host - Terminate

2013-04-12 12:09:37,790: INFO  rid=    'server'  AdminWebService - Terminating service blankMap.mpk3

2013-04-12 12:09:37,790: INFO  rid=    'server'  mpk3_host - Terminate

2013-04-12 12:09:37,790: INFO  rid=    'server'  ServerMain - Waiting for 3 worker processes to exit. Timeout=10s.

2013-04-12 12:09:37,899: INFO  rid=    ''  WebServer - Lost connection on port 50000

2013-04-12 12:09:37,899: INFO  rid=    'server'  ServerMain - Worker processes exited.

2013-04-12 12:09:37,899: INFO  rid=    'server'  ServerMain - Deleting data directory C:/Windows/Temp/arcgisruntime_6452

2013-04-12 12:09:37,899: INFO  rid=    'server'  ServerMain - 10 filesystem object(s) removed.

2013-04-12 12:09:37,899: INFO  rid=    'server'  ServerMain - Server process exiting.

Local server exited, code: 0

Exception occurs on addShapefile(LocalShapefileApp.java:103), and corresponds to :

// set up a local dynamic layer
        final ArcGISDynamicMapServiceLayer localDynamicLayer = new ArcGISDynamicMapServiceLayer(
                localMapService.getUrlMapService());

I've checked for Local Server parameters in the runtime tool "Local Server Utility 10.1.1", but nothing there...

Any ideas why I have limitations for 3 local map services ???

[JeremieJoalland1]

To continu previous discussion...

I would like to create a "blank mpk" with 10 sub layers (more than 1 is my concern), instead of only one today...
So when creating my dynamic map service layer, for each sub layer I can update the dynamic layer infos to point to my Raster datasource (10 rasters from same directory).

I was able to create a blank MPK with 1 sub layer, because it's just an empty map.
But how can I prepare additional empty sub layers in ArcMap ?

Thanks for your help...

[SachinKanaujia]

You can add the same blank layer as many times as you want. It should not matter as you are going to dynamically change the data source etc. I hope I understood your question correctly.

To continu previous discussion...

I would like to create a "blank mpk" with 10 sub layers (more than 1 is my concern), instead of only one today...
So when creating my dynamic map service layer, for each sub layer I can update the dynamic layer infos to point to my Raster datasource (10 rasters from same directory).

I was able to create a blank MPK with 1 sub layer, because it's just an empty map.
But how can I prepare additional empty sub layers in ArcMap ?

Thanks for your help...

[JeremieJoalland1]

In fact I was trying to add blank layers, when the blank map Package only need to contain blank Group Layer.
So If I add 10 "new Group Layer" and create my blank Map Package, then I can add 10 rasters by changing the 10 corresponding datasources. issue resolved !


In same time, thanks to ESRI support, I found another way to use only one process RuntimeLocalServer.exe, and I'm able to load multiple rasters from same or different locations (very useful in my case to directly load DTED files which are organized in subfolders).

for that, you just need to instantiate and start the Local Map Service before calling the addRaster() method, like this :

// create a local map service and enable dynamic layers
LocalMapService localMapService = new LocalMapService(resourceMPKFile);
localMapService.setEnableDynamicLayers(true);
localMapService.start();

// Iterate on all files to add them with new WorkspaceInfo
//  pass the Local Map Service here as parameter
for (File file: fileList) {
    addRaster(file.getName(), file.getParent(), transparency, localMapService);
}

Then in the addRaster() method, you just need to stop/restart the service, and it will add a new workspace info

private void addMultipleRaster(final String fileName, final String fileDir,
  final int transparency, final LocalMapService localMapService) {

 // First stop the service
 localMapService.stop();

 // get dynamic workspaces from service
 WorkspaceInfoSet workspaceInfoSet = localMapService.getDynamicWorkspaces();

 // an arbitrary unique string
 final String workspaceId = ""+count++;

 // create a workspace info via the static method according to data type
 // e.g. raster folder connection
 WorkspaceInfo workspaceInfo = WorkspaceInfo.CreateRasterFolderConnection(workspaceId, fileDir);
 // also:
 // --> WorkspaceInfo.CreateSDEConnection
 // --> WorkspaceInfo.CreateShapefileFolderConnection
 // --> WorkspaceInfo.CreateFileGeoDatabaseConnection

 // Add this one raster workspaceInfo to the set of workspaces
 workspaceInfoSet.add(workspaceInfo);

 // set dynamic workspaces for our local map service
 localMapService.setDynamicWorkspaces(workspaceInfoSet);

 // now (re)start the service...
 localMapService.start();

 ...

In this case, we can use the initial blank Map Package with only one default group layer....
seems to work fine, but it can take some time, because of the stop/restart issue for each file.
and I still need to perform stress test on this solution...

[CarlosColón-Maldonado]

.... seems to work fine, but it can take some time, because of the stop/restart issue for each file.
and I still need to perform stress test on this solution...

Consider the asynchronous operations of stopAsync with a addLocalServiceStopCompleteListener event to load the rasters, and the startAsync with a addLocalServiceStartCompleteListener event to keep your app responsive.

This is good work, Jeremie! Does this not work if you stop the service for all work space creation/file loading? Have you tried using the geodatabase connection method to persist loaded rasters on app restart?

[JeremieJoalland1]

Consider the asynchronous operations of stopAsync with a addLocalServiceStopCompleteListener event to load the rasters, and the startAsync with a addLocalServiceStartCompleteListener event to keep your app responsive.

You're right, and after some time, I was able to implement a solution with asynchronous operations ! It's much better for user as it can take some time to load DTED or CADRG Tiles...
my code allows user to select one folder, then all tiles contained in subfolder are loaded automaticaly by package of maximum X tiles, where X depends of number of layers/datasources in my Blank Map Package. Inconvenient is performance, as 1 local server process is running for each package, in my case 14 processes for 160 DTED tiles.

Does this not work if you stop the service for all work space creation/file loading? Have you tried using the geodatabase connection method to persist loaded rasters on app restart?

I also tried to use another solution : I definitly can only start a local service once with several workspace info, then on start completed add several dynamic layers and for each layer, define the datasource on layer initiliazed completed event, in order to add my tiles... however, it's not working all fine at this time, as I got error message when I try to load to much tiles ... I'm still looking at this, but hope it can be a better solution, as it definitly use less memory with only one process.

also, in my case I can't use geodatabase / ESRI specific data format, so no geodatabase. we persist our map configuration by serialization. (in fact, I would need to be able to create new geodatabase, then generate Mosaic Dataset with my local tiles - all this by Geoprocessing Package with ArcGIS Runtime... but I guess I need some Tools to be authorized for ArcGIS Runtime first)

[CarlosColón-Maldonado]

You're right, and after some time, I was able to implement a solution with asynchronous operations ! It's much better for user as it can take some time to load DTED or CADRG Tiles...

Glad you're able to get it done. This is how I start the local server and load map services for each of my raster types along with a base map for raster management/footprint viewing mode, all while not missing a beat at startup. At least, these process' instances do not take away from the java VM memory our app needs to fully function.

I also tried to use another solution : I definitly can only start a local service once with several workspace info ... I'm still looking at this, but hope it can be a better solution, as it definitly use less memory with only one process.

I've begged for more clarification (and documentation) on the proper use of the workspace API with no avail. Perhaps, we won't see this until an advanced course gets put together.

also, in my case I can't use geodatabase / ESRI specific data format, so no geodatabase. we persist our map configuration by serialization. (in fact, I would need to be able to create new geodatabase, then generate Mosaic Dataset with my local tiles - all this by Geoprocessing Package with ArcGIS Runtime... but I guess I need some Tools to be authorized for ArcGIS Runtime first)

You're not alone, brother. Hopefully, ESRI will realize this and get us what we need. We're at a stop because we can't add rasters to a file geodatabase where the mosaic datasets reside. Help this cause by giving me your vote here.

[TorbenWiggerich]

I have the same problem, that I cannot load more than one tif file at the same time. They are just loaded one after another...

So one question is about the following what Jeremie Joalland wrote:

"In fact I was trying to add blank layers, when the blank map Package only need to contain blank Group Layer.

So If I add 10 "new Group Layer" and create my blank Map Package, then I can add 10 rasters by changing the 10 corresponding datasources. issue resolved !"

What do you mean with corresponding datasource? Where do I need to change this? (And what exactly do I have to change?)

Second question you wrote that you have only 1 local server running, but 14 Processes. I'm not sure if it works by you or not. So this means you load 14 times a different tile at the same time?

But this only works with the mpk file changed to allow the number of processes (14)? Right?