Optimise and improve Creating Geoprocessing Cache

[RTPL_AU]

Quite often, when starting Pro (now 3.5.2) I will see "Creating Geoprocessing Cache".
This is the item that takes the most time during Pro start-up and prevents you from getting on with real work.

I have explored if what I am seeing is a bug with the local reseller, and the conclusion from them was that what I am seeing and the time it takes, is normal (10 - 30sec). 

This Idea:
Improve the management and processes related to the geoprocessing cache so that it does not get recreated so often, and when it does, it is much faster. 

Some context:
I have the Geoprocessing pane open at start-up with about 20 tools in my favourites. 

From some searches and conversations this behaviour does not seem uncommon.

[paleo78]

It's every bit of 30 seconds for me.  I can eliminate the wait by deleting the cache files in C:\Users\<username>\AppData\Local\ESRI\Local Caches, but the "Creating Geoprocessing Cache" delay just returns one startup cycle afterwards.

[DrewFlater]

@RTPL_AU @paleo78 What is described is a defect, as the Geoprocessing Cache is meant to be generated once per release (updating from Pro 3.5 to 3.6 for example), NOT each time you start Pro. The amount of time is also quite high and should complete in a few seconds under normal conditions with a system that matches requirements. Please report a BUG with Esri Support to describe your situation and debug the problem.

https://support.esri.com/en-us/knowledge-base/log-an-arcgis-bug-000034294

A few pieces of information I am curious about that will also help in your communication with Esri Support,

1. Do you have ArcGIS Pro language packs installed? 

2. When installing Pro, do you set the Optional Features to install the feature AI Models>Semantic Search? When semantic search is installed, additional caching needs to happen which impacts cache generation performance. Semantic search also has system requirements for a CPU that supports the AVX2 instruction set. https://pro.arcgis.com/en/pro-app/latest/get-started/arcgis-pro-system-requirements.htm#:~:text=A%20... . 

3. Do you install ArcGIS Pro per user or per machine? Related to this, the geoprocessing cache is generated in the folder C:\Users\<username>\AppData\Local\ESRI\Local Caches. You can delete the g*.cache files in this folder and restart Pro. Confirm the generation of g0, g1, g3, g4 files after you restart Pro. You could try renaming the whole ESRI folder and restart Pro to see if the problem is related to user profile files previously created. The ESRI folder will be recreated on restart. 

4. Do you have a custom cloned Python environment activated? Can you import arcpy in the Python window? 

[RTPL_AU]

Hi @DrewFlater 

After lodging  a case with our locals in Feb 2025, and with no clear info we (both myself & support) could find defining what is normal, this Idea was lodged in April.
My main issue to resolve at the time was to make Pro as fast to start and be ready as possible. Caching seems to still be a key detractor to a quick start. The workaround of deleting any & all caches at close or frequently to solve intermittent issues doesn't help this goal.
My questions to support were phrased around optimising anything possible around cache generation, therefore requiring documentation verbosely describing the caching process - none seems to exist.
At no point did support say that what I am seeing is a bug/abnormal and the case was closed with part of their response to lodge an Idea regarding documentation around this topic. Don't know where that went so can't link to it. 

"Geoprocessing Cache is meant to be generated once per release (updating from Pro 3.5 to 3.6 for example), NOT each time you start Pro"

Seeing the message has been my 'normal' on multiple computers over a long period of time, different networks, different servers, different user names - always installed per machine. What you are describing has 100% not been my experience in recent memory?  Are you sure? 

My unanswered questions to our local support were:

It will be great if you can try to answer these two questions:

• What triggers Pro to create the geoprocessing cache? (Having the Processing Pane open, certain toolbars, having an icon in your Quick Access Toolbar, etc)
• What causes the process to take different amounts of time if the startup project is the same every time? (content of the default fgdb, default toolbox, how many tools in favourites, etc)

If you can answer these, we can start looking for something that would cause a fresh install of Pro, on a new user account, with a new Windows install, on a new computer, to display this behaviour.

With changes in the way Windows and various apps do updates, could a metric that Pro uses to determine 'change' that triggers a cache refresh, be looking at something deprecated and therefore get false triggers?
I use tools such as winget etc to manage various other tools - maybe it resets a counter or something?  Saying that out load doesn't make sense as I get the caching happening multiple times a day as I open & close multiple instances of Pro as I work on different jobs. I'm not checking for app updates every few minutes....
With the 3.5.x Copy bug, I am closing/starting Pro A LOT more often that I used to.

Answers to your questions:

1: Not on the latest machine. Will check on others but pretty sure that the recent ones have tried to be as vanilla as possible to reduce support rabbit holes.

2: All installed - Pretty much all my recent machines except laptops have top-end CPUs with AVX2. Current 2 are Threadripper Pro 5000 & Threadripper 7000 models.

3: Tried that - ended up building a brand new machine to get away from potential user & Pro profile issues. Used a new user name (not domain joined nor a Microsoft account; pure local) with nothing brought over from my other accounts on other machines. No difference in Pro behaviour (over a spectrum of issues).

4: Nothing custom on the latest machine. I use stuff in Notebooks etc reasonably often and regular GP tools work as expected (except for known issues, different topic). 
arcpy loads.

Latest machine is running on Server 2025 (not vm, full local with the Desktop experience enabled etc) to remove any Windows 11-specifc variables with recent updates. Also running the Nvidia Studio drivers for the 4090 & RTX4000 gpus.  Many Pro issues such as the freeze on copy bug, staying in the background when clicking in empty space in the ribbon, and others, occur after a vanilla fresh install so, repeating myself, I'm pretty sure it isn't due to a dodgy Pro profile or Windows user account.

To remove another potential rabbit hole I used the new machine without XTools Pro for a while (no other plug-ins/add-ins) and the issue occurs frequently. Once XTools was installed & licensed the caching seems to take longer but I haven't noticed an increase or decrease in the overall occurrences.

@paleo78  Anything to add?

[paleo78]

1.  No.

2.  No.   AI optional search was not selected/active.

3.  Installed on machine, just one user (me).  When the cache is deleted in the folder you listed, upon starting up next time the files are generated and I do see "Creating Geoprocessing Cache" when I click on the Geoprocessing tab, but very briefly, as if is fixed/repaired.  The next time I use ArcGIS and click on the Geoprocessing tab, I get the message but this time the long 20-30 second wait.  It remains like this unless I delete the cache again.

4.  No.  No python scripting is used. 

As mentioned by @RTPL_AU,  I am also using the latest version of XTools Pro, but don't see a change in the behavior with or without it installed.

I have attached screenshots of the Diagnostic Monitor showing the elapsed time and log showing there is a CSharedQueue Stall timeout that repeats during this delay.    It's got me scratching my head.

[DrewFlater]

@RTPL_AU @paleo78 just to reinforce, this is not normal behavior and is a bug. This documentation is being added in Pro 3.6 to the Find a geoprocessing tool topic:

"The first time you access a tool or any interface that uses geoprocessing, the geoprocessing cache is automatically created. The cache is used by search and is created once per software release. Creating the geoprocessing cache typically completes in a few seconds on a machine that meets the ArcGIS Pro system requirements. If you have a language pack or Semantic Search installed, creating the cache may take longer, but it should still finish within one minute."

I am going to explore the impact of Xtools on cache creation. If they are injecting additional system and custom project tools, that will increase the time, and if it is failing in some way to generate a full cache, that may be attempted repeatedly for each Pro session.

[RTPL_AU]

@DrewFlater @paleo78 

Esri are aware of issues with XTools Pro that makes the problem worse. There is a workaround and I quote from our support:

We’ve recently also heard similar reports from other clients where XTools appears to be contributing to this behaviour. The XTools team is aware of the problem, and they have provided the following workaround:

Delete the XTools site package folder located at: C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\XTools Pro

This will remove the XTools toolbox while retaining the ribbon. I realise this isn’t an ideal fix, but at the moment it’s the best available option until XTools provides a permanent resolution.

As with any advice from strangers on the internet, make sure to understand the impact of deleting anything under Windows system folders!!!!

Drew - from an Esri Inc point of view, and as Esri are seemingly aware of issues in this context with XTools, can you please ensure that your relevant people add the Esri+XTools Bug to the central bug list under support so that it is discoverable.

Further to the base issue.
What you are quoting from the proposed docs is a bit lacking in the context of finding solutions when things don't behave as you describe. Please review the questions I listed and ask the tech writers to try to add a bit more actionable content in the docs. There has to be other triggers to cause it to run. Please list those. 
It is really hard to figure stuff out when minimal docs require you to figure out if they imply omitted details = bug, or omitted detail just means the docs are incomplete.

I understand that in Esri's basic testing the caching behaves as you say, but this has not been my normal in Pro for as long as I can remember. 
As I have seen it on many different computers, OSs, networks, etc there has to be other factors that trigger cache creation. 
I've asked my locals to re-open the case but we will need input from Esri Inc with regards to the actual triggers so we can follow a targeted approach for troubleshooting further. 

Somebody at Esri wrote a piece of code that goes IF A!=[B] THEN RUN 'Create Geoprocessing Cache'
To help you figure out the issue we need to know what's in A & B.

Thank you!