topic Re: "Lift and Shift" AWS Migration Strategy Question in ArcGIS Enterprise in the cloud Questions

"Lift and Shift" AWS Migration Strategy Question

RyanUthoff — Fri, 29 Oct 2021 13:58:49 GMT

I am currently looking at migration strategies for migrating our on-premise ArcGIS Enterprise deployment to AWS. I know there are a couple migration strategies we could use, but the one I am looking at is the "lift and shift" strategy. This is where we just copy our VMs from on-premise and put them in the AWS environment.

My question is if this is a recommended or viable migration strategy we should use? Or would we be better off using other migration strategies, such as using webgisdr to do a backup and restore of our servers? We have a 4 machine deployment, along with the config store also stored on a server, all of which will be migrated into AWS. Since it is a "lift and shift", all hostnames will be staying the same. The only thing that will probably change are the IP addresses. But we used hostnames when configuring ArcGIS Enterprise, so I think we will be okay with that.

The "lift and shift" strategy is the most simplest and straightforward for us, but I haven't seen much documentation on if this is a viable strategy for migrating ArcGIS Enterprise into AWS. Or if this is just a bad idea with disaster written all over it. Are there any reasons why we shouldn't use this migration strategy? Or anything to look out for if we choose this migration strategy?

Re: "Lift and Shift" AWS Migration Strategy Question

ChristopherPawlyszyn — Tue, 02 Nov 2021 14:48:28 GMT

We've definitely seen customers use the lift-and-shift approach to migration and experience success with doing so. The biggest issues I've seen were changing of FQDNs during the process or loss of access to underlying registered data sources. Keep in mind that even if you have a VPN tunnel to your VPC, you won't want to route your DBMS traffic across that connection due to the increase in latency.

Migration via WebGISDR is convenient because it allows you to keep the production system up-and-running while making any configuration changes on the duplicated deployment, so that may be worthwhile to look further into as well. If you can tolerate an outage window for the lift-and-shift approach and have a rollback strategy in place, then that may be the path of least resistance for your organization.

Re: "Lift and Shift" AWS Migration Strategy Question

RyanUthoff — Tue, 02 Nov 2021 16:45:43 GMT

Thanks for your response, I appreciate it! We're doing a full migration, so our GIS and DB servers will all be migrated at one time. We won't have any VPN tunnels linking AWS GIS to on-prem DB. We know that latency would probably be a nightmare.

Since everything is being migrated, all hostnames, DB servers, etc. will be staying the same, so I'm thinking there won't be any FQDN or registered data source issues. I'm not aware of any modifications that will need to be made (besides the DNS pointing to AWS instead of on-prem, but that's beyond my scope of responsibilities).

We considered WebGISDR, but it's kind of our backup method in case list-and-shift doesn't work. The lift-and-shift method is a little more straight forward for us to use.

Anyway, thank you for your response! I was just making sure we aren't setting ourselves up for failure, and I'm glad to hear that other people have used this migration strategy before.

Re: "Lift and Shift" AWS Migration Strategy Question

stevetaylor_perth — Wed, 12 Apr 2023 08:39:54 GMT

Hi Ryan - did you manage to successfully implement the lift and shift approach?

We are looking at doing the same.

Regards

Steve

Re: "Lift and Shift" AWS Migration Strategy Question

RyanUthoff — Wed, 12 Apr 2023 13:42:58 GMT

Yeah, we did implement the lift and shift approach. I'm only the GIS Server Admin, not the IT admin, so there was a lot of other stuff happening behind the scenes that I wasn't involved with or aware of. But the lift and shift approach did work successfully for us with no issues. I wasn't the one who was configuring our AWS environment, moving the VMs, etc. I just verified that everything worked after it was done.

Re: "Lift and Shift" AWS Migration Strategy Question

JonSwoveland — Wed, 04 Oct 2023 20:56:12 GMT

Hi Ryan,

Hoping to get some clarification on a couple of things:

Did you use some kind of "lift-and-shift" utility that automates moving the VMs, or did you build out a new environment and use WebGISDR to move everything?
Did your GIS environment include an Enterprise Geodatabase system, and if so, did you move this as well, or did that remain on-prem? If it remained on-prem, are you using a site-to-site VPN to support this?

Thanks!

Re: "Lift and Shift" AWS Migration Strategy Question

RyanUthoff — Wed, 04 Oct 2023 22:30:05 GMT

We did a lift-and-shift of the VMs. We had a third party do the lift-and-shift for us, so I'm not 100% sure what the tool was, but I believe it was an AWS tool that did the migration and kept the VMs in sync until we did the cut over.

Yes, it did include multiple instances of SQL Server and we migrated those to AWS as well. I 100% do NOT recommend doing any sort of site-to-site VPN where your ArcGIS Enterprise environment is hosted in the cloud while your EGDBs are hosted on-prem. I will guarantee you will have horrible latency and your maps will suffer. Your DBs need to be as close to Esri as possible (at min, the same region and preferably the same availability zone as well).

One thing to note is that you will also have horrible latency if your EGDBs are hosted in the cloud, but have VPN on your local machine where you are using ArcMap/Pro. The performance is terrible. If your EGDBs are moving to the cloud, then you will need virtual desktops to use in the cloud as well. It's not required......but the amount of productivity you will lose if you don't do that will be crazy high.

Re: "Lift and Shift" AWS Migration Strategy Question

JonSwoveland — Wed, 04 Oct 2023 23:25:22 GMT

Thanks for the quick response! Regarding the use of a VPN for EGDB connections, you've provided some good reinforcement of my position on that, so thank you for that as well!

Re: "Lift and Shift" AWS Migration Strategy Question

DevinBartley2 — Fri, 08 Dec 2023 22:59:34 GMT

First of all - thanks to Ryan and Christopher for this discussion - this thread has become an important reference to me for our own lift & shift project.

I asked the ESRI cloud team at the UC about keeping your ArcGIS pro install on-prem and connecting to your cloud hosted EGDB. They also echoed that is is technically possible but that the user experience will be "terrible" and thus is is definitely not recommended. This is a key point that any organization should consider in their cost scoping for a lift and shift, as many probably expect that they can use their current desktop work machines to run Pro to connect to their cloud system but that is not the case.

As I am scoping out our own lift & shift strategy I am looking at various partial-GPU machines to connect to to do my ArcGIS Pro work. I hope to report back on what works for me, and hopefully I can save a significant amount of cost by not going with a full-GPU machine. More info on the Azure VM's that are available here:
ArcGIS Pro on Microsoft Azure Cloud—ArcGIS Pro | Documentation

Re: "Lift and Shift" AWS Migration Strategy Question

RyanUthoff — Sat, 09 Dec 2023 00:23:59 GMT

Glad this discussion was helpful for you! You are right that organizations should consider the costs of hosting a desktop solution in the cloud. I'm not sure about MS Azure, but there are a couple options for AWS. One is the AWS Workspace, which is essentially just a virtual PC (make sure it is hosted in the same region as your DBs, otherwise it will be slow (I know from experience)).

However, depending on your organizational needs, you could also use a Windows Server EC2 instance (or Azure equivalent) and have your users connect to it instead of providing them with a dedicated workspace. The machine would be shared, but each person would have their own user account to log into it. You might need to pay for more "seats" (how many people that can concurrently RDP into the EC2 instance), but otherwise you just have to pay for the EC2 instance. It could be a more cost effective solution for people who only use Esri periodically instead of providing them with a dedicated workspace.

Re: "Lift and Shift" AWS Migration Strategy Question

DevinBartley2 — Wed, 17 Apr 2024 15:34:30 GMT

We successfully completed a Lift and Shift migration from our on-prem servers into the Azure cloud last month. The project went quite well, thanks in a large part to doing our research beforehand and having a good detailed understanding of our system. If anyone is considering doing this I'd say that you must be able to do the migration during a full system outage (if you have to keep things live then Lift and Shift is probably a bad choice), have a good understanding of your system and its dependencies, have a test environment to test the deployment, a solid working relationship between IT and GIS, a good understanding of Cloud offerings and concepts, and the ability to troubleshoot the inevitable problems that may occur.

A few things to note:
-The only major problem we ran into was our hosted feature layers broke when we first tested the migration. The solution was running the allowconnection.bat tool which modifies pg_hba.conf to allow connections between the data store machine and the arcgis server machine. More info: ArcGIS Data Store utility reference—Portal for ArcGIS | Documentation for ArcGIS Enterprise

-ESRI is correct that if you try to connect to your Enterprise Geodatabase in the cloud from a local desktop install of ArcGIS Pro it simply will be slow to work. I was skeptical but yes, it really does not work. You must deploy a GPU enabled Azure virtual desktop in the same cloud region as the database to run ArcGIS Pro. For me personally the smallest Nv_V4 machine with 1/8 GPU worked fine. However, the low amount of RAM on that machine didn't work when I also had a bunch of large PDF files open alongside ArcGIS Pro. In the end I bumped up to the 1/4 GPU Standard_NV8as_v4 and I am happy with the performance. I shut this machine off while not using it to save costs. More info on these machines here: NVv4-series - Azure Virtual Machines | Microsoft Learn

- Be aware that file storage speed is an important consideration, especially when working in ArcGIS Pro and I found especially so when working with file geodatabases. We went with Azure Files, and it works okay, but NetApp Files is recommended for optimal performance.

Overall the only reason we went with Lift and Shift is because that is how our IT scoped out migrating all of our organizations systems and because we had a very short timeline and limited budget to complete the project. We will soon be doing a complete system build from scratch with the Enterprise Cloud Builder and migrating/rebuilding our content there. This will allow for easier updates or expansions to the system in the future.

I used the System Log Parser tool to collect benchmark metrics on the on-prem system performance, and then ran it again once we migrated so that we could compare the systems. Overall our Enterprise deployment in Azure is actually slightly faster than on-prem in terms of how the ArcGIS Portal services perform. This is because we used slightly better machines in Azure than we had on-prem. Also, we have a 70ms latency to the Azure data center and I don't notice any issues there in terms of how things perform.

Re: "Lift and Shift" AWS Migration Strategy Question

DevinBartley2 — Mon, 13 May 2024 21:44:01 GMT

Just an update to this - one issue we have ran into is file geodatabases are incredibly slow when stored in Azure Files. ArcGIS Pro projects stored in Azure Files that connect to our Enterprise geodatabase seem to work fine, but if you have to do geoprocessing or field calculations on a file geodatabase in Azure Files it is not ideal. To be fair, we were warned about this. I am currently looking into Azure NetApp files or another solution.

Re: "Lift and Shift" AWS Migration Strategy Question

DevinBartley2 — Tue, 12 Nov 2024 20:33:57 GMT

The solution that we found for the above issue of the file geodatabases that are created within ArcGIS Pro Projects running slow on Azure files was to set up a scratch database instance within SQL Server. I realize that this solution may not be practical for larger organizations with more users (in that case use NetApp files!) but for us it worked just fine to set this up. We set all ArcGIS Pro projects to default to this database instance for scratch and working files, and then still maintain our separate production database for all of our authoratative feature classes. We plan on creating a new one of these scratch database instancess annually to manage the number of feature classes created within the geodatabase.