Drew - It's great that ESRI listens constructively.
In that spirit, this is off the mark: "This is actually one of the reasons that Pro IS multi-threaded: it's ability to run geoprocessing in a dedicated thread so that other work can be done in the app while a tool is running."
"Multi-threaded" in parallel processing does not mean "two threads". Using one thread for a GUI and just one other thread for a single, non-parallel background process does not qualify as "multi-threaded" today. Today people want thousands of threads in CPU and GPU, not just two threads.
Regarding: "At the 2.0 release, ArcGIS Pro has more than 50 Desktop tools that leverage distributed processing to use multiple cores or spawn multiple processes to do parallel processing."
You need to do that but it only gets you to where parallel development was years ago. Not enough to catch up. I meant ESRI should implement ArcGIS Pro as a fully parallel application. Pro is brilliant. It is worth the effort to do the job right.
Using a non-parallel ArcGIS Pro together with a collection of parallel geoprocessing tools to be introduced in 2.0 does not make ArcGIS Pro a parallel application. It will remain a non-parallel application that can use some parallel external tools, basically in batch mode.
Adding parallel geoprocessing tools has benefits, I agree. At least those jobs that can be compartmentalized can go faster. But there are limitations to that approach that have caused it to be left behind by more advanced parallel developers:
- A mix of non-parallel and parallel code is error prone. All-parallel applications are more reliable.
- Separate parallel modules usually require more user expertise. Reading the Parallel Processing Factor link I get the impression 2.0 will be that way. Users do not want to learn or to think about parallelism. They just want the software to run faster and better. Users want parallelism invisibly and automatically integrated throughout an application.
- Calling parallel geoprocessing tools from non-parallel Pro is like having a one-lane road that connects to a segment of 16 lane highway that connects back to a one-lane road again. Modern architecture is 16 lane highway all the way.
- Users want parallel speed all of the time in the main application too. Nobody wants a slow interface because it is using one thread out of 16 or 32. I saw a Reddit video how a parallel package interactively triangulated 5 million points in one second. Pro should do that too.
- You must have state of the art GPU parallelism. Keeping GPU busy requires a fully CPU parallel host. A non-parallel main application can't do that.
- Adding 50 parallel modules is weak compared to thousands of built-in GPU and CPU parallel capabilities a strong package will have.
Regarding GeoAnalytics on clusters using Spark, that is worlds away from a parallel desktop application. Listen to what people are telling you about Pro and being tied to remote servers. They don't like it.
My comment about taking two or three seconds what takes Pro hours was my mistake. When I wrote "seconds" I was thinking of a demo where a parallel package did in nine seconds what Q could not do in many hours (Delaunay triangulation). For Pro I should have written "two or three minutes" not seconds. Still harsh.
I have seen parallel software demos doing CPU/GPU computations in SQL 50 to 500 times faster than anything else. It does in two or three seconds what Arc takes minutes, or with bigger data does in two or three minutes what Arc takes hours. That seems to be mostly GPU parallelism.
2.0 is necessary but not sufficient. What you should do is parallelize all of ArcGIS Pro so all of it runs with parallel CPU and also parallel GPU. Instead of spending years re-inventing the wheel ESRI should just buy somebody that has it. Don't make people wait like they had to wait for 64-bit.
maribeth
