Monitor 2024.0.1 - All monitored hosts are disconnected

[EricJohnson-HDR]

Monitor server is not connecting to any hosts.  Under Admin\Logs there are recent entries with "Token is Expired", but not details as to which or what token is expired. All systems use same service account to register back to server.  Another is "ArcGIS Monitor Agent is not ready"

Attempted to re-register with command line from host back to server, but stated validation exception.  Assume due to there already being an existing machine with same name?  If i disconnect the host, will i loose the historical data and start from ground zero?

I have restarted services, entire monitor server but still no luck.  Restarted Postgres service and the existing DB seems to be good and healthy.

Out of options at this point.  Let me know if you have some tricks.

Thanks

[EsriEvan]

@EricJohnson-HDR - Thanks for sharing the latest. Acknowledging the OS updates, I am a bit troubled the Agents did not heal their own connection after some time 🤔. Will definitely flag this for future dev investigation.

[SteveMcCarthy]

@EsriEvan , @EricJohnson-HDR , @GrahamWood-HWC 

My issue turn out to be caused by the Metric column getting to big for the Integer field and we need to convert it to BigInit.  That resolved the issue right away.  a big shout out to Premium Support, Mao did a great job on this.

[TylerTheobald]

Our 2024.1.1 AGM server was down for multiple weeks before we noticed.  (I've since put a heartbeat monitoring in place")

Afterwards, all agents stayed disconnected until I had to register them all again with the Agent ID parameter.  That was a bit of work for almost 100 agents.  

Definitely something to get fixed in future versions. 

[VirgilioPalmi]

Hi, I'm in the same situation. I created this script in the first version, and I was able to re-register the VMs with little effort.

I'm posting it here, hoping it can help you in the future (it works on Windows).

# Script: Register-MonitorAgents.ps1

# Mappatura host -> agent-id
$agents = @{
    "prsitagsgp01.domain.it"  = 52
    "prsitagsgp02.domain.it"  = 54
    "prsitagsimg01.domain.it" = 53
    "prsitagsimg02.domain.it" = 55
    "prsitagsm01.domain.it"   = 56
    "prsitagsm02.domain.it"   = 57
    "prsitagspub01.domain.it" = 60
    "prsitagspub02.domain.it" = 61
    "prsitagsun01.domain.it"  = 42
    "prsitagsun02.domain.it"  = 43
    "prsitagsun03.domain.it"  = 44
    "prsitagsun04.domain.it"  = 45
    "prsitagsun05.domain.it"  = 46
    "prsitagsun06.domain.it"  = 47
    "prsitagsun07.domain.it"  = 49
    "prsitagsun08.domain.it"  = 48
    "prsitagsun09.domain.it"  = 50
    "prsitagsun10.domain.it"  = 51
    "prsitdata01.domain.it"   = 58
    "prsitdata02.domain.it"   = 59
    "prsithsd01.domain.it"    = 40
    "prsithsd02.domain.it"    = 41
    "prsitprt01.domain.it"    = 38
    "prsitprt02.domain.it"    = 39
    "prsitwebadp01.domain.it" = 36
    "prsitwebadp02.domain.it" = 37
}

# Parametri comuni
$serverUrl = "https://AGSPRMONITOR.domain.it:30443/arcgis"
$username  = "portaladmin"
$password  = "PortalPassword10@"
$exePath   = "C:\Program Files\ArcGIS\Monitor\Agent\bin\arcgis-monitor-agent.exe"

foreach ($vm in $agents.Keys) {
    $agentId = $agents[$vm]
    Write-Host ">>> Registrazione agente su $vm (Agent-ID: $agentId)" -ForegroundColor Cyan

    Invoke-Command -ComputerName $vm -ScriptBlock {
        param($exePath, $serverUrl, $username, $password, $agentId)

        & "$exePath" admin:system:register `
            --server-url $serverUrl `
            --username $username `
            --password $password `
            --agent-id $agentId
    } -ArgumentList $exePath, $serverUrl, $username, $password, $agentId
}

Virgilio

[DerekLaw]

Thanks for sharing your work @VirgilioPalmi !!