GEP 10.2.1 Datastore - Cannot register AGS w/ Windows AD & non-public services

[DennisGeasan]

This is related to an earlier thread where I cannot register an ArcGIS server data store. 

The ArcGIS Server data store I'm trying to register is configured as follows:
Machine: Windows Server 2008 R2
ArcGIS Server: 10.2.1
AGS Security: Windows for Users and Roles with the 'Web' (Web Adapter) used for the Authentication Tier and Mode;
Map Services Permissions:  Root level - set to 'Private'.

With the above configuration on the AGS machine I cannot register it in 10.2.1 GEP.  If I change the root level permissions on the AGS machine to 'Public' then I can register it in GEP.  If I set the root level to 'Private' and then change the permissions on a folder to public, then I CAN register the AGS machine in GEP

It is required at this site that all map service permissions be private.  How do I accomplish that with the new version of GEP? I tried defining tokens but that doesn't work.

Dennis Geasan

[JavierDelgadillo]

Dennis,

The only mode of authentication supported by GEP at 10.2.1 for Servers and Portals is token based authentication.  We do not support any of the other various forms of authentication protocols supported by Web Browsers (like NTLM, Basic, Digest, or PKI).  If your WebAdaptor installation requires that (as it seems to), then you cannot configure GEP access to go through the WebAdaptor.

If both GEP and your ArcGIS Server sit behind the firewall, then you should configure GEP to talk directly to your ArcGIS Server using a token obtained using the browser.  The ArcGIS datastore registration dialog will show you the link to visit and get a token.  You'll need to ensure you copy the referrer exactly as it appears in the registration dialog (beware of copying/pasting whitespace), otherwise GEP will not be able to talk with your ArcGIS Server.

-Javier

[DennisGeasan]

Hello Javidel,

I did have success defining the AGS data store by generating a token using the ArcGIS Server administrators account user ID/password.  For using windows Authentication on the AGS machine I guess this is my only option.

However, I am able to use the web adapter when defining a GEP data source.  In GEP 10.2 I ran a service with a web adapter connection for several weeks.   It still works in 10.2.1 except now I have to set the ArcGIS Server feature access service permissions to 'public access' and I have to include port number 6080 in the GEP data source URL.  In 10.2 I did not have to do that.  In both versions of ArcGIS Server I am using Windows authentication (users and roles) with the Web Adapter installed.  In IIS I have the Web Adapter site configured with 'Anonymous Authentication' disabled and 'Windows Authentication' enabled.

I've generated tokens as you described but the data source fails to connect when I use a windows user account.  Attached is more description.

For the customer site I'm working with it is a requirement that all map services be private.   And the web adapter is also required for Windows Authentication with the corporate domain.  For GEP 10.2.1 it looks like I now have to register the AGS data store using the AGS admin account.

Not sure if that is a good thing or not.  Comments?

Also, as a suggestion, a little more description in the help on this subject would be useful.  If it is required that GEP connect to AGS using a token then explain that in the help and also don't provide the option to not use a token in the GEP data source dialog.
DG

[JavierDelgadillo]

Will try to answer some of the questions from your PDF here.

We have the option to use a token because some ArcGIS Servers require tokens while others don't.  We don't know that when registering the connection.  Furthermore, some servers may have some folders/services that are publicly viewable while some are restricted.  In these cases, users need the ability to use or not use a token to access the services on their ArcGIS Server.

When choosing the ID/password to enter on the token generation page, enter the credentials for an account that can access the service you want to consume or update.  Additionally, set the Expiration to the length of time to the duration you want your DataStore to be valid.


If you mouse-over the red icon in the ArcGIS data store window, what message do you see?  This should give you a hint as to why GEP is having trouble accessing the services.

-Javier

[DennisGeasan]

Hello Javidel,
The tokens I am creating are lasting only one day.  Yesterday I registered the server using a token that was created using the AGS admin account.  After the registration process there was a green check mark in the status column.  This morning there is a red circle with a horizontal white bar.  When I mouse over the icon the message is "Invalid Token".  I am selecting the option in the token generator to allow the token to last one year.

In the token generator, you have the option of setting the response to HTML or JSON.  I have been using the JSON response which includes a UNIX time format string for the expiration date.  Running that value thru a Unix time format converter it is 24 hours after creating the token, even if I have chosen the one year option.

So yesterday I created two AGS registrations, one based on the JSON response and the second on the HTML response.  My thought was that maybe the HTML response provided the correct token.  No joy.  Today, the status of both entries is now the red icon/"Invalid Token".  For both of those entries I used the AGS admin account for the user ID/password.

I tried again to use a windows domain account in the token generator.  I get the following response:

{"error":{"code":200,"message":"You are not authorized to access this information","details":"Invalid credentials"}}

The AGS machine I am registering is using Windows for managing roles and users and the Web Adapter is installed and working properly. I tried both styles for the user id (domain\userID and userID alone).  Same response to both.

I checked the security section of AGS and I see the User and Role store is actually set to 'ASP.NET'.  That is a configuration (one recommended by ESRI) I had to do in 10.2.0 to accommodate issues with AD roles.  After the upgrade to 10.2.1 from 10.2.0 this read as "Windows" for the User/Role store but apparently it has reverted to the previous configuration.  I will pursue getting this set back to strictly Windows but:

Should I be able to use a Windows user ID/Password in the token generator?

I would rather not have to use tokens.  There is the obvious problem of remembering to regenerate the token and if we can't get a token to last more than 24 hours, well.......  Using a Windows AD account should be sufficient.  Access to folders and services in AGS is defined by the Win AD roles assigned to those folders/services so the token approach is not really needed.  But if the GEP code base can only connect to AGS via tokens then I should be able to utilize a Windows AD domain account to generate the token.

DG

[AndreasEspersen]

Hi Javier,

There is one thing i do not understand when adding a data store with a token.

We have a setup with a server holding GEP and ArcGIS Server installed and wanting to register a data store on another server with ArcGIS Server containing feature services.. In order to be able to cluster the server with feature services. We are registering the data store the server with feature services via another server with the web adaptor installed as shown below.

The web adaptor gives access to the arcgis server with feature services to outside our internal network. Therefore we have disabled administrative access on the Web adaptor for security reasons.

The data stores is successfully added but from the log I can see that a call is being made to

https://<Web adaptor URL>/arcgistest/admin/data/findItems?f=json&token=XXXXXXX&types=egdb: GET Request failed(HTTP/1.1 403 Forbidden).

Why is the GEP trying to access an admin URL?

Regards

- Andreas

[RJSunderman]

Hello Andreas -

My apologies that no one has gotten back with you in response to your question.

GeoEvent was designed to rely on GIS-tier communications with an ArcGIS Server site using Server generated tokens. Support for web-tier authentication and authorization using SAML and IWA is being developed, but is limited, and is being introduced beginning with the 10.3.1 product release.

I need to clarify your question. Are we working with two different GIS Server machines, both with ArcGIS Server installed, which are participating in different ArcGIS Server sites? I ask because you mentioned the term "cluster" and I want to make sure that the GIS Servers we are considering are not part of a single ArcGIS Server site cluster.

If the two GIS Servers were participating in the same ArcGIS Server site cluster with the GeoEvent Extension installed on one GIS Server but not on the other we would be working with an unsupported configuration. All of the GIS Servers in a site's cluster must be provisioned exactly the same - this is important when using the GeoEvent Extension.

Next question - is it an option for your first ArcGIS Server, the one which has the GeoEvent Extension installed, to reach the second ArcGIS Server without going through the Web Adapter? Design assumptions built-in to the GeoEvent 10.3 release (and previous releases) assume that GeoEvent will be able to discover services using the ArcGIS REST Services Directory and that the directory will be accessed through either the open REST endpoint http://server‑name.domain:6080/arcgis or the secure endpoint https://server‑name.domain:6443/arcgis. The preference would be that GeoEvent not attempt to go through the web adapter to discover a server's services.

Some support for IWA was implemented in the 10.3.1 product release for registering ArcGIS Server connections as GeoEvent Data Stores. For example, you can specify that GeoEvent should use a token to access the data store and specify the secure endpoint (https://server‑name.domain:6443/arcgis) as the URI to be used. When the token expires, you will have to refresh your Data Store configuration with a new token in order to continue using the data store. You can also, beginning with the 10.3.1 release, select to use web-tier authentication and supply a recognized IWA username and password when registering a data store.

My understanding is that, even when using web-tier authentication, that access must still be made through fully-qualified server-name / port URI ... either the http://server‑name.domain:6080/arcgis endpoint or the secure endpoint https://server‑name.domain:6443/arcgis. I will try to get someone to confirm this.

Hope this information helps -

RJ

[RJSunderman]

Andreas -

Some feedback specifically on your question:  "Why is the GEP trying to access an admin URL?"

https://<Web adaptor URL>/arcgistest/admin/data/findItems?f=json&token=XXXXXXX&types=egdb: GET Request failed(HTTP/1.1 403 Forbidden).

The GeoEvent Extension is using the admin API to determine if a managed geodatabase has been registered with ArcGIS Server. If there is a managed geodatabase, when configuring a new 'Send Features to a Stream Service' output and selecting to publish a Stream Service, you will have the option to check the 'Store Latest' checkbox and publish a "buddy" feature service which will cache the most recent observation for each received TRACK_ID. If you do not have a managed geodatabase registered with ArcGIS Server, the 'Store Latest' option is not available.

- RJ

[AndreasEspersen]

Hi RJ,

Thanks for getting back - Much appreciated.

A little background information about our setup i might provide some insight why we asked the question.

We are using one ArcGIS for Server with GeoEvent Extension - Lets called this server for Server 1. We have another ArcGIS for Server holding feature services (FS) and map services (MS) (Server 2). The GeoEvent Extension outputs to a FS on server 2.  From what I heard from UC and what one of my colleagues was told in Palm Spring this is best practice.

Server 2 make MS and FS available both internally and externally. Some of the applications require to have security enabled. One of them is a service that the GeoEvent Extension updates.

Sometime in the future we would like to add more ArcGIS For Server to Server 2 and create a cluster still holding all FS and MS. We have prepared for this are using the web adaptor in front of the ArcGIS Server holding FS and MS as a load balancer but in the future we might use a physical load balancer. We have followed the architecture described in ArcGIS Help (10.2, 10.2.1, and 10.2.2).

Server 1 is not accessible from outside our domain and only run GeoEvent Extension.

We raised the initially question when we had problems updating FS with Security enabled  on server 2 (GIS-Tier with users LDAP). The update would run about 4-6 hours and then stop.

The Geoevent Extension would say everything was still alright. The data store could still be vailidated but the Feature class being updated did not receive any updates. The only thing that caught my attention in the log was was:

'https://<Web adaptor URL>/arcgistest/admin/data/findItems?f=json&token=XXXXXXX&types=egdb: GET Request failed(HTTP/1.1 403 Forbidden).'

If we connect to server 2 with the directy URL then we the error below: properly due a certificate error.

Invalid URL: Please provide a valid url for example: http://<hostname>:<port>/arcgis

So why is it that the geoevent Extension stops after 4-6 hours?

- Andreas

[DennisGeasan]

Andreas,

I found Version 10.2.2 a little more stable.  I experienced a similar problem with the extension just stopping after random amounts of time when using a Web Socket "listening" connection as the input.  The inputs seem to be more reliable if they poll the data source rather than waiting (listening) for the data source to send the input data.

DG