How to setup an ArcGIS counter for a cloud formation stack

MikeSchonlau · ‎10-09-2019

I have deployed an AWS elastic load balancer (ELB) CloudFormation template along with a stand-alone ArcGIS Server CloudFormation template (10.7.1). My ArcGIS Server site has a file server and two ArcGIS Server nodes. I am hoping to setup an ArcGIS counter in Monitor to look at this new ArcGIS Server site.

I'm having trouble figuring out which EC2 Security Group rules I need to enable so Monitor can see this new site. I have this setup in Monitor for two other ArcGIS Server sites, but they were not built with CloudFormation templates.

I've tried adding inbound All TCP rules for both my Monitor security group and the elastic ip of my Monitor server to the CloudFormation stack security group, but that hasn't worked. I tried doing the same for the ELB security group, but that didn't work either.

Anyone have thoughts on how this should be setup? Thanks

Anonymous User · ‎10-09-2019

Hi Michael,

Have you configured outbound rules too?

what TCP ports you had added, are they include these ports - TCP 135, random port number between 1024 - 65535?

It bit tricky, I will suggest work with your cloud system administrator and figure out the correct random ports.

Hope this will help you.

Good Luck!

Manoj

MikeSchonlau · ‎10-21-2019

All outbound ports are open to all traffic on both the CloudFormation stack's security group and the Monitor server's security group.

Anonymous User · ‎10-21-2019

Hi Michael Schonlau @mschonlau

,

Let me dig my email archives and check for you what I did previously to resolve such type of issues. I am sure I had documented this somewhere in my repository.

-Manoj

ShreeRajagopalan · ‎10-09-2019

Hello Michael Schonlau‌ - Your question warrants a holistic approach where an expert will need to go over the whole system with you. In order to do that, please contact Esri Technical Support at your earliest convenience.

MikeSchonlau · ‎10-10-2019

I don't believe Tech support will be able to answer this question. I already have an established Monitor setup and I am now trying to add one new component that has a slightly different configuration than my existing ArcGIS Servers. I was hoping someone who has worked with the Cloudformation templates and ELB's might have some thoughts. I'll keep investigating...

Anonymous User · ‎11-25-2019

Are you have to successfully login to the ArcGIS Server Manager via your web browser (see generic link below) launch from your laptop or from the ArcGIS Monitor server?

https://gisserver.domain.com:6443/arcgis/manager

If you can get this link to work, then the Site URL and Token URL that you specific in ArcGIS Monitor should work.

MikeSchonlau · ‎11-25-2019

It is not that simple. Yes I can get to my Manager URL, however Monitor is trying to resolve the underlying private IP address of the server for some reason. That won't work because an auto-scaled ArcGIS Server environment uses different machines every day with different IP addresses, so even if it registers the right IP when I create the collector, that may not work the next day because that machine IP is no longer joined to the site. I also cannot use elastic ip's with an auto-scaling group.

This is not a standard scenario. Our ArcGIS Server uses an AWS EC2 auto-scaling group with an elastic load balancer and does NOT use the ArcGIS Web Adaptor. I have an ArcGIS collector setup in Monitor with my other ArcGIS Server that works fine. That server that also uses an elastic load balancer, but is not an auto-scaled server.

ESRI must be doing the same thing as I am trying to do with their internal systems. ESRI staff has told me they use AWS auto-scaling and ArcGIS Monitor to monitor their systems. How is their setup different?

Also, please don't paste instructions from the standard documentation and then mark the answer as correct. That is very presumptive.

AboladeAfolabi2 · ‎01-17-2020

Hi Michael! We worked together on this case alongside other internal resource to arrive at a work around pending a more permanent resolution as noted in the bug :

[BUG-000124066:Improve our ArcGIS Enterprise Cloud documentation so that users know that they can set the machineSuspendThreshold and suspendedMachineUnregisterThreshold properties for Auto-Scaling deployments]

and the enhancement

[ENH-000118085: After an Auto Scaling rule is invoked the machine is not deleted from the ArcGIS for Server site leaving it orphaned within the machines section of the site. Because of the orphaned machines publishing takes a long time]

I write this so that others could benefit from the work around, we configured the following properties:

machineSuspendThreshold (At 10.7, machines record their status once per minute in the configuration store if they are active. This property defines the time period of inactivity after which a machine will be suspended from receiving administrative and publishing requests. Once a suspended machine records an active status, it will be reinstated to receive such requests. The default value is 60 minutes.) and suspendedMachineUnregisterThreshold (At 10.7, machines record their status once per minute in the configuration store if they are active. This property defines the time period of inactivity after which a machine will be unregistered from the site. This property is disabled by default, with a value of -1. If used, this threshold should be higher than the machineSuspendThreshold property).

The information required to modify these properties can be found at https://developers.arcgis.com/rest/enterprise-administration/server/serverproperties.htm

The machineSuspendThreshold and suspendedMachineUnregisterThreshold is a JSON addon that can be added in the Arcgis Server Admin API https://server:port/arcgis/admin/system/properties

As of version 10.7.x, these are the two properties whose values can be configured to automatically unregister machines from the ArcGIS Server site after a certain amount of time. It was good to see ArcGIS Monitor Counter connect as expected after the above configuration.

MikeSchonlau · ‎01-21-2020

A quick follow-up to this. The ArcGIS Monitor collector worked initially because it found the ip address of one of the site's active machines. Once the auto-scaling group had replaced that machine though, the Monitor collector stopped working. The moral of this story is that the ArcMonitor 'ArcGIS' collector doesn't handle AWS auto-scaling groups well because it tries to register the ip address of the machine instead of using the site's url. The machines in that scenario are variable so the collector breaks. I can understand why this would not work with an ArcGIS Monitor 'system' collector (it can't dynamically monitor changing machines), but I would hope it could recognize an ArcGIS Server site, regardless of the underlying machines participating in the site.