topic Re: Accessing the JS API via a web worker in ArcGIS JavaScript Maps SDK Questions

Accessing the JS API via a web worker

RobertHewlett — Mon, 15 May 2017 21:51:56 GMT

I would like to offload some of the heavy lifting in a web app to a web worker.

It is a 2-3 second jank I would like gone.

Right now the web worker would still need access to parts of the API e.g. Extent and Polygon.

I have tried loading the init.js script via importScripts() but I still get errors.

Has anyone had any success with the ArcGIS JS-API inside a web worker ... not outside but inside.

Re: Accessing the JS API via a web worker

RobertScheitlin__GISP — Tue, 16 May 2017 12:09:35 GMT

Robert,

I have tried in the past and came to the conclusion that you can not interact with the API inside a web worker. I have been able to do simple or even complex math or JS operations but not use esri classes.

Re: Accessing the JS API via a web worker

ThomasSolow — Tue, 16 May 2017 13:11:56 GMT

It seems to me that importScripts is the right way to handle this. JavaScript doesn't allow you to share between threads currently.

I would imagine that there are limitations to what can be imported, and it may be difficult to import the entire JS API for various reasons. For instance the worker's global scope is different from the normal JavaScript global scope, and the JS API probably relies on the normal global scope in some places.

It may be possible to build the modules you need into a single file (Extent, Polygon, and all dependencies) and pull this in via importScripts, assuming there's no issues with those modules running in the worker's scope. I don't know much about the details here though.

Re: Accessing the JS API via a web worker

ReneRubalcava — Tue, 16 May 2017 15:07:22 GMT

If you are using 4x, there is a Worker framework in the API that we use for some of our internal processing of things like VectorTiles.

Connection | API Reference | ArcGIS API for JavaScript 4.3

There are limitations to what currently works in a Worker, for example esri/request works in a Worker, but in 4.3, Authentication does not, unless you authenticate on the main thread first. You also can't pass Geometries from workers to the main thread, you need to convert them to JSON first, but if your main thread used a typed Collection to load that JSON, it could work.

In the future we'll try to add a sample or two on workers, as we can make sure other parts of the API can be loaded in workers.

Re: Accessing the JS API via a web worker

RobertHewlett — Tue, 16 May 2017 15:38:58 GMT

Thanks for all the responses.

I am using 4.3 and I have been staring at the frame work but hesitate 'jumping down the rabbit hole' if what I am trying to do is not possible.

So, I have a couple of specific questions:

1.) Would I be able to do the following inside the worker:

require(["esri/geometry/geometryEngine", "esri/geometry/Extent", "esri/geometry/Polygon" ], someFunctionInTheWorker)

2.) Is the worker a true worker and what I mean here is, can I still use postMessage() and onmessage?

I would like to send a progress message back several times before I send the final result.

3.) If I serialize to JSON will I get everything back on the other side. For example, will a polygon object still have a centroid?

Rob

Re: Accessing the JS API via a web worker

RobertHewlett — Tue, 16 May 2017 16:32:37 GMT

In the sample for the framework there is a curios line that causes an error:

esriConfig.workers.loaderConfig

Re: Accessing the JS API via a web worker

KristianEkenes — Tue, 16 May 2017 16:42:33 GMT

Here are my responses to these three questions:

1. Yes you can reference API modules in the worker. I have definitely used geometryEngine, jsonUtils, and promiseUtils in a worker loaded from the JSAPI. But I would use define() instead of require().

2. I don't know. cc\ odoe ubatsukh-esristaff

3. Yes. I serialized Graphics in a worker (including the geometries) and got everything back.

Re: Accessing the JS API via a web worker

RobertScheitlin__GISP — Tue, 16 May 2017 16:49:23 GMT

Was this possible in 3.x?

Re: Accessing the JS API via a web worker

KristianEkenes — Tue, 16 May 2017 16:51:20 GMT

I'm not sure. I've only played around with this in 4.x.

Re: Accessing the JS API via a web worker

RobertHewlett — Tue, 16 May 2017 17:14:13 GMT

Actual progress (rabbit-hole-time)

Replace

The esriConfig.workers.loaderConfig stuff in the API sample code with the code below:

var local = window.location.href.substring(0, window.location.href.lastIndexOf('/'));
workers.open(this, local + "/<myJSdirectory>/<nameOfMyworkscript.js>")
.then(function(connection) {
return connection.invoke("max", {
numbers: [0, 1, 2, 3, 4]
});
})

That seems to bridge the sample code and gets the value 4 in the console.

Re: Accessing the JS API via a web worker

JohnGrayson — Tue, 16 May 2017 17:14:51 GMT

This allows you to resolve paths to your worker scripts relative to your app so you don't have to provide a full url when calling connection.open(...).

config | API Reference | ArcGIS API for JavaScript 4.3

Re: Accessing the JS API via a web worker

ReneRubalcava — Tue, 16 May 2017 17:19:54 GMT

The client doesn't have access to the created worker, so you can't add event listeners to listen to messages. I can check if there is some other route for that or if we can expose it in some manner.

Here is a sample using a worker to buffer the centroid of the view extent.

index.html · GitHub

Re: Accessing the JS API via a web worker

RobertHewlett — Tue, 16 May 2017 17:33:51 GMT

I was able to make an extent in the worker and send it back.

On my first attempt I sent the raw object back to the UI but the object seemed confused.

Then I used JSON.stringify() and got something that made sense.

See below:

{"xmin":1,"ymin":1,"xmax":10,"ymax":10,"spatialReference":{"wkid":4326}}

Inside the worker (BTW I flatten my code i.e. avoid fx nesting at all costs):

define(["esri/core/promiseUtils", "esri/geometry/geometryEngine", "esri/geometry/Extent", "esri/geometry/Polygon"], work);

function work(promiseUtils, ge, Extent, Polygon) {

....

env = new Extent( 1, 1, 10, 10);
return promiseUtils.resolve({data: JSON.stringify(env)});

.....

}

Re: Accessing the JS API via a web worker

JohnGrayson — Tue, 16 May 2017 17:44:34 GMT

No need to convert it to a string, but you do need to convert the JS object to JSON:

return promiseUtils.resolve({data: env.toJSON()});‍‍

You can then recreate the Extent in your main thread by passing in the JSON directly to the Extent constructor:

Re: Accessing the JS API via a web worker

ReneRubalcava — Fri, 10 Dec 2021 23:16:41 GMT

I just updated this gist to handle worker communication for things like progress updates

index.html · GitHub

Turns out you can pass an object as the first argument to workers.open(). You can then invoke any functions from the worker on this argument.

// client
workers.open({
  progress: (msg) => {
    console.log("progress:", msg);
    return promiseUtils.resolve(msg);
  }
 }, workerUrl)

// worker
const Request = function() {}
 Request.prototype.magic = function magic({ extent }, { proxy }) {
  proxy.connection.invoke("progress", "starting");
  const area = Polygon.fromExtent(Extent.fromJSON(extent));
  proxy.connection.invoke("progress", "polygon created");
  const centroid = area.centroid;
  proxy.connection.invoke("progress", "centroid found");
  const ptBuff = geometryEngine.buffer(centroid, 1000, "feet");
  proxy.connection.invoke("progress", "buffer created");
  const data = ptBuff.toJSON();
  proxy.connection.invoke("progress", "buffer serialized");
  return promiseUtils.resolve({ data });
 }
 return Request;‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Re: Accessing the JS API via a web worker

RobertHewlett — Tue, 16 May 2017 20:17:06 GMT

That worked!

Re: Accessing the JS API via a web worker

RobertHewlett — Tue, 16 May 2017 20:34:21 GMT

Hi Rene,

I am newbie on this part of the API so my conceptual thinking might be off but ....

Is proxy.connection.invoke basically the worker invoking one of its own methods/functions as this is usually done from the client side.

Is the there a list of client side objects you can send into the worker and from within the worker call these methods/functions back in the client.

Rob

Re: Accessing the JS API via a web worker

RobertHewlett — Tue, 16 May 2017 20:39:30 GMT

"You can then invoke any functions from the worker on this argument."

What if the function updated the client UI? BOOM?

Re: Accessing the JS API via a web worker

ReneRubalcava — Tue, 16 May 2017 21:30:43 GMT

It's not invoking a method on itself, the proxy.connection invoking a method on the object you are passing to workers.open(), that exist on the client. It can be anything you want, it just needs to have a method that returns a promise, because it's also a way for you to request information from the main thread from within the worker.

Re: Accessing the JS API via a web worker

RobertHewlett — Wed, 17 May 2017 23:07:17 GMT

Just a heads up for others. Once I started using the proxy.connection as shown in the gist-sample the variable names on the client-side and the web-worker side had to match.

This might be a chrome-thing, transferable object-thing or a AJS-connection framework-thing. Not sure as I am a newbie to this part of the API.

If I do this on the non-web-worker side

 return connection.invoke("someFX", { argsAsJSON });‍

Then inside the web-worker I had to make the variable names match:

       Request.prototype.someFX = function someFX({argsAsJSON}, {proxy}) {‍

If I used another variable name then that variable would be undefined:

       Request.prototype.someFX = function someFX({data}, {proxy}) {‍

The line above (in the chrome debugger) would report data as undefined.