Portal Best Practices for setup?

[PaulDavidson1]

Surprisingly, I didn't find any prior posts about combining Portal with Server:

I am starting to setup an internal Portal  w/ Federated/Hosting servers

And a new server farm setup based on 10.3.1

I want to tie the whole shebang into our AD servers so that users should be able to do single sign in.

This will all be behind our firewalls on an intranet.

I'm in a virtual server environment, VMware

I've found two recent write ups rather helpful.

Portal for ArcGIS 101   - also published as a pdf somewhere....

http://www.esri.com/library/brochures/pdfs/arcgis-server-functionality-matrix.pdf

I'm curious as to how folks are doing some of the setup:

Are you dedicating a server strictly to Portal?

Putting Portal and ArcGIS Server on the same box?

Federating or Hosting?

http or https?  Am I correct in assuming that it's better to just start with https and go from there.

Seems like I've read that for something to work correctly, you need to be using secure services.

My reading indicates the best thing to do perhaps is to have an ArcGIS Server, with Portal and Data Store on the same box?

Is best to then only lightly load this ArcGIS server and let other Servers do the heavy lifting?

How does IIS play into this?  Separate server?

That's my plan.

Right now we have a server running ArcGIS Server, lots of map services and IIS and well....

It's not the most stable box.

How about when you toss a GeoEvent processor into the mix?

Should you dedicate a Server strictly for GeoEvents?

I'd imagine this depends on how many events are coming in and at what frequency?

What's the installation order that is most successful?

Build up your Server farm and then bring Portal into the mix?

Or stand up Portal first and then go from there?

I can always just build virtual boxes and toss them away if they seem to be wrong.

But without putting the setup into production, it's hard to load it properly for a real-time test.

Now I know a lot these potential answers are really dependent on the organization and it's level of activity.

But any comments or experience will be helpful.

[JonathanQuinn]

Something to consider regarding the all-in-one approach vs a distributed approach, if that one machine goes down, you lose your Portal, Server, and Data Store.  At 10.4, a new tool called the Web GIS Disaster Recovery tool was introduced to backup your Web GIS, (Portal and all of the content, users, groups, etc, Server and all of the services but not any referenced data within those services, and Data Store, with all of the data in the Data Store.).  Using the tool, you can take periodic backups of your whole Web GIS, so, if your all-in-one machine were to go down, you can easily stand it back up and import the backup.  If you're interested in uptime and redundancy, an all in one approach with single instances of Portal, Server, and Data Store probably won't do it.

[PaulDavidson1]

Good points Jonathan, which of course is why the HA approach is out there.  Eventually I will try to setup the SA site.

In our case, we have a DR site at a separate location where the primary site is replicated (snapshot) to DR on some defined backup basis, often hourly, depends on the application.

If primary goes down, we bring up the DR with very little effort.  That at least is theory.  The Primary/Secondary DR setup and a trial DR has not yet been tested in action yet.

However, we did just have to recover both Test and Production from our VM backups (nightly)  This took about ten minutes.

But you lose what work was not in the backup.  This is not the HA setup.  But a benefit of a good VM environment.

We are definitely looking into incorporating the Disaster Recovery tools into the mix.

Regarding distributed vs. single server setups, consider this: in a distributed setup, you are now maintaining 3 or 4 individual servers, any one of which can go down.  And then what?  Can Portal really continue to function if you lose anyone of the components?

It will of course depend on your setup, redundancies, etc... but if you lose your Portal server, your portal is down.  Maybe you have some apps running off AGS that could continue but most Portal setups need Portal.  Lose your AGS and there go all your services, what are you left with that is of any use?  Losing the Data Store could be the least painful IF most of your data is not hosted on there.

All scenarios have plus and minuses that need to be carefully considered relative to each site's needs.

I'm waiting to see how the single box works in terms of performance once it gets into heavy use.

Certainly for uptime, an HA failover setup, is probably about the most solid.  But I'm sure that introduces issues.

For example, I had setup our Production Portal with an eye toward an HA setup.  This means the config stores are on a shared file server.  Our shared file server had a NIC lockup which took everything Portal related down.  I think even in an HA Portal environment, now everything is down in that scenario.  Unless of course, you have a failover file server.

And failover typically requires a NLB.  So if that goes down...  A truly failsafe setup can be a real rabbit hole.  Fortunately, most GIS shops can handle a bit of downtime.

I guess we just always need to make sure our paper map books are still available for the absolute worst case scenario  🙂

[JonathanQuinn]

A truly failsafe setup can be a real rabbit hole

Nailed it.  True HA means everything is HA, web tier, portal tier, server tier, data tier, and file server tier.

Regarding distributed vs. single server setups, consider this: in a distributed setup, you are now maintaining 3 or 4 individual servers, any one of which can go down.  And then what?  Can Portal really continue to function if you lose anyone of the components?

It will of course depend on your setup, redundancies, etc... but if you lose your Portal server, your portal is down.  Maybe you have some apps running off AGS that could continue but most Portal setups need Portal.  Lose your AGS and there go all your services, what are you left with that is of any use?  Losing the Data Store could be the least painful IF most of your data is not hosted on there.

In regards to those questions, they might be rhetorical but I'll respond anyway.  If you have a distributed setup of 4 ArcGIS Servers all participating in one site, and that's federated to Portal acting as the hosting server, 3 of those machines can go down and you'll still have a functional Portal, as Portal should communicate with the Server site through a mechanism that can detect failures and route traffic to healthy machines.  For example, you'd federate Portal and Server using where the Admin URL, (the URL Portal communicates with Server through), and Services URL go through a load balancer.  In Portal HA, you'd also set the private portal URL for Portal, (the URL used by Server to talk to Portal) to go through a load balancer as well to make sure that even if one of the Portals go down, you can still reach the Portal.  Finally, you'd have a primary and standby data store, as those are configured to detect failures and failover when necessary.

Of course there's the added cost and trouble of maintaining a fully HA environment, so it's a line to balance; more redundancy and resiliency vs cost of maintenance.

[PaulDavidson1]

Good stuff Jonathan.  And no, those were not rhetorical questions.

I believe the design and layout of a Portal environment is actually a fairly meaty question and like all things Esri (in my experience anyway,) there is often not a single best way to do something.

It does boil down to all sorts of issues that I think make it difficult to come up with a best practices scenario.

I believe discussions like this are the best way to help one (myself included here) come up with an optimal solution.

Knowing how others do it, knowing about the different ways to do it, knowing something about the commitment in time and resources for the various setups, etc...  all can help one make a meaningful decision.

Plus, it's an ever changing game in the IT world.  And even for Esri it's a learning experience.  The past year alone has seen major shifts in thinking about BP for setups.

The VM world is a real game changer and at the same time, brings in it's own set of new issues that can be counter intuitive.

For example, we recently discovered that adding memory to our Maximo application server (big IBM asset management and work order system) actually slowed it down.  Turns out VMs have a sweet spot related to the # of cores, ram size, etc... and if you miss it, you can cause what I would say is analogous to disk thrashing but in this case is RAM thrashing.

That's very unintuitive because I've always looked at RAM ops as being very fast, even if going over boundary lines.  At least in this one case, the reality is otherwise.

The following might be a reasonable set of questions to answer:

• Do I really need Portal?  Can I do this with AGOL? - Our tech rep tried really hard to sell me on AGOL and in fact convinced me that not having to deal with supporting Portal was well worth it.  Unfortunately, other corporate issues meant we had to go down the Portal road.  It is nice to have finer grained control over our Portal system.  But it's also very tempting when having to troubleshoot issues to think about how AGOL would make a lot of headaches go away.  Take two AGOL and see me in the morning...  I strongly recommend that potential Portal users really think through this question.  It's very tempting for IT types to want to control everything and be in charge.  But Portal/AGOL is a lot about letting go of control.  You're empowering your users to create their own mapping solutions.  I suspect that at the end of the day, AGOL is actually much cheaper for an organization than Portal is.  Credits are quite cheap.  IT resources are not.  Just something to consider.
• What sort of uptime does my situation require?   Do I need HA or just DR or neither.  What level of HA do I need?  How far do I need to go to hit a reasonable balance of availability vs. complexity.  The more complex the setup, the more opportunities for failures right?  Even with a full HA setup, you still would need a fully independent DR site to handle the worst case scenario.  For example, if your full HA GIS setup was in the twin towers and you didn't have an offsite DR...
• What are my resources?
  • This is both hardware and personell.  If you don't have a solid IT crew, Portal is probably not for you.  Even with a solid crew, it's a non-trivial setup.  It looks fairly simple but as you get into it, it's a real beast.
• What are my companies needs?  How many named users?  What sort of data are we pushing around?  Our field guys love to see imagery so we'll need a fat pipe and a good VPN setup for that.
• What is my current setup?  We're lucky in the sense we have a mapzilla Silverlight product that folks are very happy with so we can take our time dialing in Portal because we have that to fall back on to.

I'm sure there are other critical questions to ask.  I should dig out my notes from Dave's class about Enterprise setups.  😉

I hope others will add to this list and use their experience to share with us the plethora of experiences, pain points, solutions, etc....that the different type of setups have provided them.

[SteveCole]

Great stuff, Paul, thank you!

[MichaelFlanagan]

FYI, there's a new Esri Training course that speaks to many of the pieces of your initial question. Might be of interest to some of you.

Esri Training | Deploying Portal for ArcGIS

Cheers,

Mike

[PaulDavidson1]

Excellent, looks like it's also available online which is always easier to get approval for than travel.

Thank you Michael!

Edit: 2017:  I took this class in mid 2016 and even though I had already setup a Dev/Test/Portal scenario, I found it very helpful and often use the course literature & my notes as a reference.

[YeonKim]

Paul,

This thread is really helpful as I try to figure out an answer to the same question. We are planning a migration from 10.3.1 to 10.5.1.

Are you still happy with your all-in-one approach?

Thanks!

Yeon

[PeterHanmore]

I'm hoping someone on this thread might be able to answer this question for me as the documentation I've seen is somewhat ambiguous to me.

When installing Portal and federating with an AGS site that has two or more servers, can you select one of the AGS servers as the hosting server or do you just pick the site ?  In our dev environment we only have one server but in production we have a multi-server site.  I'm curious about this because in dev, when installing the relational datastore, it puts it on a local drive but on a multi-server site will it put a copy of the datastore on all server's local drives?  Do you need to specify a UNC path for the relational datatore?

Sorry if these have been answered elsewhere but I haven't been able to find direct answers to these question yet.

Peter

[JonathanQuinn]

If you have a multi-machine site, you federate Portal with the site. The admin URL you use to federate should be a URL that checks the health of both Server machines and detects failures, (such as the web adaptor or a load balancer). If server 1 in your multi-machine site goes down, Portal should still be able to talk to server 2 through the admin URL.

For the relational ArcGIS Data Store, you can install it on a separate machine from your ArcGIS Server machines. When you register it to your site, Server doesn't communicate to the Data Store through the file system, but through database connections. The Data Store does use the file system to store the database files, so you should install and configure the Data Store on a local drive, (for example C:\arcgisdatastore), and when you register it with Server, Server and Data Store will handle the communication between themselves.