topic Re: Can't load more than 3 Local Map Service in Java Maps SDK Questions

Can't load more than 3 Local Map Service

JeremieJoalland1 — Sun, 12 Dec 2021 16:44:07 GMT

I face limitation with the ArcGIS Runtime 10.1.1 samples "Add Shapefile" (issue is similar with "Add Raster").

I have just changed the source code in order to load a Local Tiled Package and used my own blank MPK to load Shpaefile/Raster.

When I run the sample application, if I try to load several Shapefiles, I always have the following Exception on the 4th file.
It's like my Local Server can't handle more than 3 worker processes at the same time...

...
2013-04-12 12:09:24,821: INFO  rid=31  'server'  mpk3 - Request handled.

2013-04-12 12:09:24,852: INFO  rid=32  'server'  mpk - Request handled.

2013-04-12 12:09:25,055: INFO  rid=33  'server'  mpk2 - Request handled.  -> Here we see that there are 3 localmapservice already loaded

2013-04-12 12:09:29,305: INFO  rid=34  'server'  AdminWebService - POST admin/createService -> Here I just select a 4th shapefile

java.lang.NullPointerException
 at com.esri.map.Layer.setUrl(Unknown Source)
 at com.esri.map.DynamicLayer.setUrl(Unknown Source)
 at com.esri.map.DynamicLayer.<init>(Unknown Source)
 at com.esri.map.ArcGISDynamicMapServiceLayer.<init>(Unknown Source)
 at com.esri.client.samples.datasources.LocalShapefileApp.addShapefile(LocalShapefileApp.java:103)
 at com.esri.client.samples.datasources.LocalShapefileApp.access$3(LocalShapefileApp.java:73)
 at com.esri.client.samples.datasources.LocalShapefileApp$3.actionPerformed(LocalShapefileApp.java:247)
 at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2018)
 at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2341)
 at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
 at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
 at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
 at java.awt.Component.processMouseEvent(Component.java:6505)
 at javax.swing.JComponent.processMouseEvent(JComponent.java:3321)
 at java.awt.Component.processEvent(Component.java:6270)
 at java.awt.Container.processEvent(Container.java:2229)
 at java.awt.Component.dispatchEventImpl(Component.java:4861)
 at java.awt.Container.dispatchEventImpl(Container.java:2287)
 at java.awt.Component.dispatchEvent(Component.java:4687)
 at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832)
 at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4492)
 at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422)
 at java.awt.Container.dispatchEventImpl(Container.java:2273)
 at java.awt.Window.dispatchEventImpl(Window.java:2719)
 at java.awt.Component.dispatchEvent(Component.java:4687)
 at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:723)
 at java.awt.EventQueue.access$200(EventQueue.java:103)
 at java.awt.EventQueue$3.run(EventQueue.java:682)
 at java.awt.EventQueue$3.run(EventQueue.java:680)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
 at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
 at java.awt.EventQueue$4.run(EventQueue.java:696)
 at java.awt.EventQueue$4.run(EventQueue.java:694)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
 at java.awt.EventQueue.dispatchEvent(EventQueue.java:693)
 at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:244)
 at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:163)
 at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:151)
 at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:147)
 at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:139)
 at java.awt.EventDispatchThread.run(EventDispatchThread.java:97)
2013-04-12 12:09:35,071: INFO  rid=    ''  WebServer - Lost connection on port 50000  

Stopping server   -> I've closed the application from here
2013-04-12 12:09:37,790: INFO  rid=    ''  WebServer - Lost connection on port 50000

2013-04-12 12:09:37,790: INFO  rid=35  'server'  AdminWebService - POST admin/shutdown?f=json

2013-04-12 12:09:37,790: INFO  rid=    'server'  ServerMain - Shutting down

2013-04-12 12:09:37,790: INFO  rid=    'server'  ServerMain - Terminating web services

2013-04-12 12:09:37,790: INFO  rid=    'server'  AdminWebService - Terminating service blankMap.mpk

2013-04-12 12:09:37,790: INFO  rid=    'server'  mpk_host - Terminate

2013-04-12 12:09:37,790: INFO  rid=    'server'  AdminWebService - Terminating service blankMap.mpk2

2013-04-12 12:09:37,790: INFO  rid=    'server'  mpk2_host - Terminate

2013-04-12 12:09:37,790: INFO  rid=    'server'  AdminWebService - Terminating service blankMap.mpk3

2013-04-12 12:09:37,790: INFO  rid=    'server'  mpk3_host - Terminate

2013-04-12 12:09:37,790: INFO  rid=    'server'  ServerMain - Waiting for 3 worker processes to exit. Timeout=10s.

2013-04-12 12:09:37,899: INFO  rid=    ''  WebServer - Lost connection on port 50000

2013-04-12 12:09:37,899: INFO  rid=    'server'  ServerMain - Worker processes exited.

2013-04-12 12:09:37,899: INFO  rid=    'server'  ServerMain - Deleting data directory C:/Windows/Temp/arcgisruntime_6452

2013-04-12 12:09:37,899: INFO  rid=    'server'  ServerMain - 10 filesystem object(s) removed.

2013-04-12 12:09:37,899: INFO  rid=    'server'  ServerMain - Server process exiting.

Local server exited, code: 0

Exception occurs on addShapefile(LocalShapefileApp.java:103), and corresponds to :

// set up a local dynamic layer
        final ArcGISDynamicMapServiceLayer localDynamicLayer = new ArcGISDynamicMapServiceLayer(
                localMapService.getUrlMapService());

I've checked for Local Server parameters in the runtime tool "Local Server Utility 10.1.1", but nothing there...

Any ideas why I have limitations for 3 local map services ???

Re: Can't load more than 3 Local Map Service

CarlosColón-Maldonado — Fri, 12 Apr 2013 15:08:19 GMT

I'm seeing similar behavior loading NGA Raster Product Format (RPF) raster types. The occurrence is random based on what is being loaded: Common Imagery Base (CIB) averages on the fifth attempt, Global Navigation Chart (GNC) goes no more than three attempts, and Jet Navigation Chart (JNC) takes no more than 4 files. I'm enclosing the Runtime local server utility log and eclipse console log files when trying CIB types.

The symptom is overall identical based on log entry "WebServer - Lost connection on port 50000."

Re: Can't load more than 3 Local Map Service

MarkBaird — Wed, 17 Apr 2013 14:51:38 GMT

Tried this out today and it's looking like it might be a bug.

We're looking at it... however we think we've got a workaround which we're testing out so all it not lost. Watch this space.

Mark

Re: Can't load more than 3 Local Map Service

MarkBaird — Thu, 18 Apr 2013 13:52:32 GMT

Looked at this some more.

I've found that there doesn't appear to be a limitation to the number of local services you can run (obviously I've not gone mad adding them as you'd just run out of memory).

I think the problem here appears to be around using the same MPK file multiple times. I modified the sample code so that is used a dedicated MPK file for each instance of the local server and I was able to add around 12 shapefiles without issue.

The other potiential option is to add multiple shape files to a single instance of a local server.

Mark

Re: Can't load more than 3 Local Map Service

JeremieJoalland1 — Fri, 19 Apr 2013 08:49:50 GMT

I think the problem here appears to be around using the same MPK file multiple times. I modified the sample code so that is used a dedicated MPK file for each instance of the local server and I was able to add around 12 shapefiles without issue.

Thanks Mark. I was able to load lot more shapefiles and raster using a different MPK for each new local map service... so I agree, the limitation seems to appear when using the same MPK file multiple times.
However it's not good news for me, as my application should stay open on the number of shapefile/raster the user can load (even if performance could be impacted), so I will have to copy my MPK template file before each time I want to create a new local map service... not so great !

The other potiential option is to add multiple shape files to a single instance of a local server.

So I'm definitly interested in your second idea :
- could you provide a sample code for this ? (multiple shp to a single instance of "local map service" ?)
- but will I still be able to manage layer order the same way ? (user can change layer order at any time in my app)
-> if exemple I have 1 TPK, 1 GraphicsLayer and 2 Shapefiles added to same MPK instance, could I manage order like 0-TPK 1-SHP n°1 2-GraphicsLayer 3-SHP n°2 ?

Re: Can't load more than 3 Local Map Service

CarlosColón-Maldonado — Fri, 19 Apr 2013 21:24:15 GMT

The other potential option is to add multiple shape files to a single instance of a local server.

Ditto! I'll be watching for some sample code. The multiple sets of data to be loaded at once (RPF), I plan to keep as a representation of a single raster layer, so I am not as concerned with the layer ordering of raster file of a set since they are related. Shape files, though, can be of concern.

Re: Can't load more than 3 Local Map Service

EliseAcheson1 — Tue, 23 Apr 2013 16:28:21 GMT

Hi,

For multiple shapefiles to one local map service, you need to have them in the same directory and then when you create your dynamic map service layer, for each sub layer you can update the dynamic layer infos to point to your shapefile datasource. The limitation here is in your available sublayers. With 3 sublayers you can add 3 shapefiles in one go, etc.

So I'm definitly interested in your second idea :
- could you provide a sample code for this ? (multiple shp to a single instance of "local map service" ?)
- but will I still be able to manage layer order the same way ? (user can change layer order at any time in my app)
-> if exemple I have 1 TPK, 1 GraphicsLayer and 2 Shapefiles added to same MPK instance, could I manage order like 0-TPK 1-SHP n°1 2-GraphicsLayer 3-SHP n°2 ?

The shapefiles at this point are like sublayers of an ArcGISDynamicMapServiceLayer, so they would move as a group and not separately.

I'll attach a modified version of the add shapefile sample that has the multiple file selection enabled on the JFileChooser and corresponding modifications to loop through each file selected.

~elise

Re: Can't load more than 3 Local Map Service

JeremieJoalland1 — Wed, 24 Apr 2013 06:23:52 GMT

When using your sample code, after selecting multiple shapefiles from same folder, they seem all to be added correctly as sub-layers to the ArcGISDynamicMapServiceLayer, through DynamicLayerInfoCollection...
😞 But each time only one (the first one) shapefile is displayed on the map !!

After the for {} which add all sub-layers and just before the setDynamicLayerInfos(dynamicLayerInfos), if I check the size of the DynamicLayerInfoCollection variable, result is 1 (should it not be 3 if I added 3 Shpaefiles ?) :
- dynamicLayerInfos.size() return "1" !
- dynamicLayerInfos.getVisibleList().size() return "1" !

Do you have this behavior ?

Re: Can't load more than 3 Local Map Service

EliseAcheson1 — Wed, 24 Apr 2013 08:27:37 GMT

That was what the first part of my post was about.
When I use USCitiesStates.mpk for example from the sample data that comes with the SDK, I can add 3 shapefiles as it has 3 sublayers. If I use the mpk_blank, I can add one at a time.

Note that the code I attached has a couple of build errors against 10.1.1 jars, from imports that have changed packages. A simple remove/re-import solves that but just to mention it.

~elise

Re: Can't load more than 3 Local Map Service

JeremieJoalland1 — Wed, 24 Apr 2013 09:51:29 GMT

ok, sorry I didn't undertand this sub-layers capacities from the blank mpk.
much clearer now 🙂
don't worry, your code is easy to use once we redo the import and change the TPK/MPK used.

Thanks for your help and code sample.

Re: Can't load more than 3 Local Map Service

JeremieJoalland1 — Mon, 08 Jul 2013 07:23:51 GMT

To continu previous discussion...

I would like to create a "blank mpk" with 10 sub layers (more than 1 is my concern), instead of only one today...
So when creating my dynamic map service layer, for each sub layer I can update the dynamic layer infos to point to my Raster datasource (10 rasters from same directory).

I was able to create a blank MPK with 1 sub layer, because it's just an empty map.
But how can I prepare additional empty sub layers in ArcMap ?

Thanks for your help...

Re: Can't load more than 3 Local Map Service

SachinKanaujia — Wed, 10 Jul 2013 14:37:36 GMT

You can add the same blank layer as many times as you want. It should not matter as you are going to dynamically change the data source etc. I hope I understood your question correctly.

To continu previous discussion...

I would like to create a "blank mpk" with 10 sub layers (more than 1 is my concern), instead of only one today...
So when creating my dynamic map service layer, for each sub layer I can update the dynamic layer infos to point to my Raster datasource (10 rasters from same directory).

I was able to create a blank MPK with 1 sub layer, because it's just an empty map.
But how can I prepare additional empty sub layers in ArcMap ?

Thanks for your help...

Re: Can't load more than 3 Local Map Service

JeremieJoalland1 — Sun, 12 Dec 2021 03:31:55 GMT

In fact I was trying to add blank layers, when the blank map Package only need to contain blank Group Layer.
So If I add 10 "new Group Layer" and create my blank Map Package, then I can add 10 rasters by changing the 10 corresponding datasources. issue resolved !

In same time, thanks to ESRI support, I found another way to use only one process RuntimeLocalServer.exe, and I'm able to load multiple rasters from same or different locations (very useful in my case to directly load DTED files which are organized in subfolders).

for that, you just need to instantiate and start the Local Map Service before calling the addRaster() method, like this :

// create a local map service and enable dynamic layers
LocalMapService localMapService = new LocalMapService(resourceMPKFile);
localMapService.setEnableDynamicLayers(true);
localMapService.start();

// Iterate on all files to add them with new WorkspaceInfo
//  pass the Local Map Service here as parameter
for (File file: fileList) {
    addRaster(file.getName(), file.getParent(), transparency, localMapService);
}

Then in the addRaster() method, you just need to stop/restart the service, and it will add a new workspace info

private void addMultipleRaster(final String fileName, final String fileDir,
  final int transparency, final LocalMapService localMapService) {

 // First stop the service
 localMapService.stop();

 // get dynamic workspaces from service
 WorkspaceInfoSet workspaceInfoSet = localMapService.getDynamicWorkspaces();

 // an arbitrary unique string
 final String workspaceId = ""+count++;

 // create a workspace info via the static method according to data type
 // e.g. raster folder connection
 WorkspaceInfo workspaceInfo = WorkspaceInfo.CreateRasterFolderConnection(workspaceId, fileDir);
 // also:
 // --> WorkspaceInfo.CreateSDEConnection
 // --> WorkspaceInfo.CreateShapefileFolderConnection
 // --> WorkspaceInfo.CreateFileGeoDatabaseConnection

 // Add this one raster workspaceInfo to the set of workspaces
 workspaceInfoSet.add(workspaceInfo);

 // set dynamic workspaces for our local map service
 localMapService.setDynamicWorkspaces(workspaceInfoSet);

 // now (re)start the service...
 localMapService.start();

 ...

In this case, we can use the initial blank Map Package with only one default group layer....
seems to work fine, but it can take some time, because of the stop/restart issue for each file.
and I still need to perform stress test on this solution...

Re: Can't load more than 3 Local Map Service

CarlosColón-Maldonado — Mon, 15 Jul 2013 13:02:53 GMT

.... seems to work fine, but it can take some time, because of the stop/restart issue for each file.
and I still need to perform stress test on this solution...

Consider the asynchronous operations of stopAsync with a addLocalServiceStopCompleteListener event to load the rasters, and the startAsync with a addLocalServiceStartCompleteListener event to keep your app responsive.

This is good work, Jeremie! Does this not work if you stop the service for all work space creation/file loading? Have you tried using the geodatabase connection method to persist loaded rasters on app restart?

Re: Can't load more than 3 Local Map Service

JeremieJoalland1 — Wed, 11 Sep 2013 14:48:04 GMT

Consider the asynchronous operations of stopAsync with a addLocalServiceStopCompleteListener event to load the rasters, and the startAsync with a addLocalServiceStartCompleteListener event to keep your app responsive.

You're right, and after some time, I was able to implement a solution with asynchronous operations ! It's much better for user as it can take some time to load DTED or CADRG Tiles...
my code allows user to select one folder, then all tiles contained in subfolder are loaded automaticaly by package of maximum X tiles, where X depends of number of layers/datasources in my Blank Map Package. Inconvenient is performance, as 1 local server process is running for each package, in my case 14 processes for 160 DTED tiles.

Does this not work if you stop the service for all work space creation/file loading? Have you tried using the geodatabase connection method to persist loaded rasters on app restart?

I also tried to use another solution : I definitly can only start a local service once with several workspace info, then on start completed add several dynamic layers and for each layer, define the datasource on layer initiliazed completed event, in order to add my tiles... however, it's not working all fine at this time, as I got error message when I try to load to much tiles ... I'm still looking at this, but hope it can be a better solution, as it definitly use less memory with only one process.

also, in my case I can't use geodatabase / ESRI specific data format, so no geodatabase. we persist our map configuration by serialization. (in fact, I would need to be able to create new geodatabase, then generate Mosaic Dataset with my local tiles - all this by Geoprocessing Package with ArcGIS Runtime... but I guess I need some Tools to be authorized for ArcGIS Runtime first)

Re: Can't load more than 3 Local Map Service

CarlosColón-Maldonado — Mon, 16 Sep 2013 11:28:27 GMT

You're right, and after some time, I was able to implement a solution with asynchronous operations ! It's much better for user as it can take some time to load DTED or CADRG Tiles...

Glad you're able to get it done. This is how I start the local server and load map services for each of my raster types along with a base map for raster management/footprint viewing mode, all while not missing a beat at startup. At least, these process' instances do not take away from the java VM memory our app needs to fully function.

I also tried to use another solution : I definitly can only start a local service once with several workspace info ... I'm still looking at this, but hope it can be a better solution, as it definitly use less memory with only one process.

I've begged for more clarification (and documentation) on the proper use of the workspace API with no avail. Perhaps, we won't see this until an advanced course gets put together.

also, in my case I can't use geodatabase / ESRI specific data format, so no geodatabase. we persist our map configuration by serialization. (in fact, I would need to be able to create new geodatabase, then generate Mosaic Dataset with my local tiles - all this by Geoprocessing Package with ArcGIS Runtime... but I guess I need some Tools to be authorized for ArcGIS Runtime first)

You're not alone, brother. Hopefully, ESRI will realize this and get us what we need. We're at a stop because we can't add rasters to a file geodatabase where the mosaic datasets reside. Help this cause by giving me your vote here.

Re: Can't load more than 3 Local Map Service

TorbenWiggerich — Thu, 18 Sep 2014 12:49:32 GMT

I have the same problem, that I cannot load more than one tif file at the same time. They are just loaded one after another...

So one question is about the following what Jeremie Joalland wrote:

"In fact I was trying to add blank layers, when the blank map Package only need to contain blank Group Layer.

So If I add 10 "new Group Layer" and create my blank Map Package, then I can add 10 rasters by changing the 10 corresponding datasources. issue resolved !"

What do you mean with corresponding datasource? Where do I need to change this? (And what exactly do I have to change?)

Second question you wrote that you have only 1 local server running, but 14 Processes. I'm not sure if it works by you or not. So this means you load 14 times a different tile at the same time?

But this only works with the mpk file changed to allow the number of processes (14)? Right?