ArcGIS Notebook Installation and Configuration in Offline Environment

[BrandonCranfield]

Hello all!

Enterprise Specs:
Windows Server 2022 (Disconnected - Local Intranet)
Enterprise 11.3 (Notebook 11.3)
Mirantis Container Runtime 23.0.15

We are trying to configure ArcGIS Notebooks and have been successful all up until the point of creating a notebook. I am unclear if there are any additional configurations that need to happen for Docker/Notebook server in order for a Notebook to be created. Port 2375 is open. Notebook Server has been federated with Portal. Dockerd.exe is configured to run with tcp://localhost:2375. No firewalls are configured on the system yet so we have unfettered access. 

Things I have tried so far:

1. Changing Admin API System/Properties setting to reflect machine's IP and fqdn as the dockerConnectionHost
2. Changing daemon.json to include host 'IP':2375
3. Opening the communication to 0.0.0.0:2375 on both
4. Ensured permissions for the service account are applied to Notebook directories and Docker
   1. The C:\ProgramData\docker folder refused to allow permissions changed on it, so this one was left alone.

Symptoms

1. When trying to open a notebook, a new tab opens and begins the process of opening a notebook
2. Starts at 20%, then 50%, and hangs at 60% eventually leading to a timeout
3. A container is created, but the following errors are revealed in the logs
   1. Attempting to access https://127.0.0.1:30001/nbs/notebooks/*containerid*/ api sessions/ using httpClient from Notebook Server to the container is not yet successful.
   2. ArcGIS Notebook Server is not yet ready to receive requests for 'https://127.0.0.1:30001/nbs/notebooks/*containerid*/api/sessions/". Retrying...
   3. Docker container did not reach a ready state in the allocated amount of time when trying to access /api/sessions
   4. A Docker container '' exists in the configuration store but no longer exists in the Docker environment. This Docker container should be cleaned up from the configuration store.
      1. This error goes away once the container that was created is deleted...

If this could please get pushed up to the Product Owners of ArcGIS Notebooks.... that would be a life saver.

[CodyPatterson · Accepted solution]

Hey @BrandonCranfield 

Sounds about the same as I had when I was originally setting my organization's Notebook server up.

Have you tried temporarily disabling the firewall on the server since it's an intranet device? What leads me to think it's still a port issue is the api problems. Once disabled, I would check the status of the Notebook to see if it's able to run. I would also follow this chart and ensure all ports on your devices listed are opened:

Let me know if this doesn't end up helping and I can look further!

Cody

[BrandonCranfield]

So, I did find out that Windows Defender Firewall was turned on even though I changed that in the GPO. Apparently, a parent GPO overwrote my change and kept it enabled. Once I moved the machine into a No_GPO state, the application worked as intended. Thanks for the assist!

[BrandonCranfield]

@CodyPatterson 

Thanks for the reply! I will need to get with the Firewall folks on Monday to test that theory. 

I was able to use Test-Netconnection from my Portal VM and it succeeded. 

We have also provided them a list of all the ports used by ArcGIS Enterprise according to that document you attached to be included in the Allow list. So that is what is leading me away from thinking it's  a port issue.

[shikhar_deep]

Hi @BrandonCranfield ,

Can you please share the following details too?
 

1. Docker version & docker info

2. Heath check status of the NBS machine from its administrative api?

3. For starters, can we start with single site machine with directories & config-store defined locally in the same machine.

4. Please share the system properties from NBS administrative api and runtime memory values (in case any changes are made)

5. Please ensure all the steps in "Configure Mirantis Container Runtime" from https://enterprise.arcgis.com/en/notebook/latest/install/windows/install-mirantis-container-runtime-... are performed.

Thanks & Regards,

Shikhar Deep

[BrandonCranfield]

Docker Version - 23.0.15

Health check status - Ready

This is a single site machine with default directory and config-store location

All system properties have remained default, no changes have been made.

All steps from that site you mentioned were followed to the 'T'

[shikhar_deep]

Hi Brandon,

Will need more detailed info. 

1. Please share the screenshot of "docker info"

2. Please increase the websocket size (32MB) and in the runtimes try increasing the Max CPU (3) and Memory(8) too.

3. Set the log levels to Debug and create a new user with administrative privileges.

4. Login with this new user, and open the notebook. Now please share logs from administrative api and ALSO share container logs (admin->system->container-> logs)

5. Try using another browser.

6. What are systems specs? 

Thanks & Regards,
Shikhar Deep