10 Critical Cloud Migration Planning Factors
This post covers the 8th topic in the series > “10 Critical Cloud Migration Planning Factors”.
A recent CIO survey by JP Morgan projects “41.6% of corporate workloads at big companies are expected to be running in the public cloud within the next five years, up from 16.2% today.” This level of cloud migration marks a MASSIVE shift. Enterprises face a daunting challenge to migrate their on-premises workloads into the cloud. However, unlike the famed and perilous wildebeest migration on the Serengeti plains, enterprise workloads can smoothly and securely migrate to the cloud, if Enterprise IT considers the following critical planning factors:
1. Why Migrate Your Applications To The Cloud?
2. Three Different Paths to Cloud Migration: Different Time/Cost/Skills
3. Selecting Applications: Good Candidates for Migration
4. Selecting Applications: Bad Candidates for Migration
5. Pitfalls in Migrating Complex Apps to the Cloud
6. Discovery versus Assessment for Cloud Migration
7. Software Licensing: Can I run this app in the cloud?
8. Cloud Migration: Manual versus Automated: What's The Difference?
9 Cost Analysis: On Premises vs Cloud
10. Critical People Factors: Right Stakeholders? Right Skills?
In this post, we will cover
#8: Cloud Migration: Automated vs Manual Migrations - What's The Difference?
Over the last 12 months, enterprise conversations regarding cloud migration have accelerated. However, much to my dismay, the majority of the volume migrations have been performed using manual/VM import-export/image conversion tools - whether it is by enterprises or cloud partners/cloud systems integrators.
I can only explain this because of the lack of awareness of automated solutions. Why perform manual migrations when automated solutions exist?
Now, if you are a small to mid-size company, using an image conversion solution to migrate 5-15 VMs/servers could make sense. Especially since some of these tools provided by the cloud providers are free. You could grit your teeth and migrate one VM at a time and do the prep work to make these apps run in the cloud. And as you will learn, it’s more than converting an image; the cloud is not simple or friendly.
If you are considering “volume” or “mass migrations”; i.e. 50 to 500 to 1000 VMs/apps or more, you need to absolutely evaluate a solution that can automate this task to minimize errors, reduce project delays and accelerate your time to value.
Let’s review both manual and automated approaches to cloud migration by breaking them into 3 stages:
- Prepping the Apps/VMs for migration
- Replicating the Apps/VMs in the destination cloud
- Prepping the migrated Apps/VMs to run in the cloud
For purposes of discussion, we will consider the following
> 5 VMs
> Manual tool uses VM import/export
> Automation tool from CloudVelox that uses “Application Blueprinting”
- Prepping the Apps/VMs for Cloud Migration: Manual vs Automated
- Identifying the servers/VMs that are to be migrated
- Collecting information on the server, processor, memory configurations
- Collecting information on the storage devices/volumes in use by the selected applications
- Collecting information about the subnets, IP addresses of the various systems
- Collecting information about the security policies (firewall rules etc.) for the selected VMs/servers.
- Some enterprises may find some of this information in a Configuration Management Database (CMDB). Others may use an application discovery tool. Nonetheless, the collection of this information is through various tools and is likely to be manual. The aggregation of this information is manual and likely on separate reports that will need to be reviewed and input manually in steps 2 and 3 below.
- Cost calculations for running the apps in the cloud will need to be done manually for each of the systems using the AWS Cost Calculator
Automated Prepping for Migration:
CloudVelox Sentinel software (installed on servers/VMs in data center) and One Hybrid Cloud Director software (installed in the cloud) automate the prepping of the VMs/Apps for cloud migration:
- The Sentinel software installed on either the physical server or VM automates the identification of the infrastructure characteristics of the application environment. The installation of the Sentinel software on selected VMs in the migration wave can also be automated (e.g.: using systems management tools such as Systems Center Configuration Manager etc.)
- Since Sentinel software runs in “user space”, no reboot of VMs is required. Production apps can continue to run during migration
- Sentinel software performs the following functions in seconds to identify the application environment (this step is the creation of the “Application Blueprint”, a process that is continuous, wherein any changes to the application environment are automatically updated in the blueprint):
- Automated identification of compute characteristics: server, CPU, memory
- Automated identification of storage characteristics: storage devices in use by the apps, file systems, shared storage over NFS, etc.
- Automated identification of network characteristics: network structure of applications including subnets and IP addresses of VMs,
- Automated identification of security characteristics: open/closed ports on the system (for example, on a web server, ports 80 and 443 may be open, but on a database server, typically these ports will be closed)
- The 5 VMs in the application can be grouped into an “application group”. All of the steps below are performed for the group rather than for individual VMs having a significant impact on productivity and the reduction of manual errors.
- Automated remapping of “application infrastructure environment” (based on the “Application Blueprinting” completed in the previous stage) by Sentinel software running in data center and One Hybrid Cloud Director software running in the cloud:
- Automated selection of an appropriate matching compute instance(s) in AWS EC2 - can be overridden for CPU or memory options
- Automated identification of storage devices/volumes used by the apps in the datacenter, including the identification of the file systems in use. The provisioning of the storage volumes in the destination cloud is automated based on the storage volumes in the data center. In addition, there are automated mechanisms built in to overcome the 1 TiB limitation on magnetic EBS volumes. The matched storage volumes are presented and can be overridden for higher performance storage services (eg: Provisioned IOPS SSD volumes). In the future identification of IOPS characteristics in the datacenter will be matched to the appropriate storage services in the cloud.
- Automated remapping of data center network characteristics to a cloud network structure. Two options are possible a) automated creation of new VPC and either recreating the same subnets or variant and remapping to the data center network design on the fly, or b) automated remapping into a specified existing VPC in AWS. Networking characteristics such as how instances get IP addresses can also be selected and overridden
- Automated remapping of security characteristics to security groups in the cloud – security groups can be customized
- The above automated remapping of the application environment is completed for the entire group of applications that were selected. In the current example, the set is 5 applications but there is no technical limitation on selecting more apps.
- Having completed the mapping to the appropriate compute, storage, and network services, the user can now look into the cost-calculations for running the application group in the cloud. The cloud costs per hour are displayed and the user can do a “What-If” analysis for different scenarios: for example selecting different EC2 instances (higher performance, more memory) or various storage services (magnetic vs SSD vs higher provisioned IOPS). The cloud costs per hour are dynamically and automatically calculated giving the user a real-time estimate of the what it would cost to run their workloads in the cloud..
All of the above steps for prepping the apps/VMs for the cloud are performed in minutes through automation and without requiring any specialized cloud knowledge or skills.
- Replicating the Apps/VMs To The Cloud
- Select the VM
- Shut down the VM (for production apps, this requires downtime for migration) before exporting it to a format (For example, OVA, VMDK, VHD, or raw) accepted by VM Import/Export
- Use VM Import/Export or Image Conversion tool to convert VM (say vSphere VMDK format) to cloud provider image format. e.g.; in case of Amazon Web Services the VMDK needs to be converted to an Amazon Machine Image (AMI)
- Many of the tools are cloud-based, so the import of the VM uploads the VM image from the source (data center) to the cloud where it is converted and then stored. And in most cases, the VM image will be stored in AWS S3.
- This procedure will need to be manually repeated for each of the 5 VMs.
- Some of the tools for Import/Export may allow for more than the boot volume to be replicated from the data center to the cloud. If not, you have to replicate the other volumes manually. Database copying and synchronization maybe complex and require specialized users who can perform this task so data arriving in the cloud is not corrupted and synchronized appropriately
- In addition, in a scenario, where the manual image conversion was performed on day 1 of the week and then the source application environment was changed on day 4 (installed patch, security update etc.) before the replication finished; then most likely the image conversion may need to be restarted from the beginning, delaying the migration project
- Finally, it will be important to check if the image conversion tool is resilient to network interruptions i.e. if an unplanned network failure occurs while the replication is ongoing, can the image conversion tool sync incrementally or will it need to be initiated from the beginning.
This approach makes it very difficult to keep the cloud cutover window to a minimum. Typically the window for doing a manual replication and sync is days or weeks
Automated Replication & Sync:
- The selected applications in the group are automatically replicated and sync-ed through continuous replication
- Everything in the application group is included in the replication: the application, the operating system, the patches, the kernel, the database and the data.
- The replication “traffic” is encrypted while in transit and encrypted when stored in the cloud; in addition the applications are replicated using data compression techniques to speed up the replication as well as reduce the amount of bandwidth consumed
- Any changes in the source environment are dynamically replicated in the cloud
- The replication process is resilient to network interruptions and will dynamically and incrementally resume from the last point of sync.
This approach makes it easy to keep the cloud cutover window to a matter of hours, not days.
- Prepping the Migrated VMs/Apps To Run in the Cloud
- The user has the 5 VMs/apps now available as AMI’s in the cloud. The adventure begins!
- The right EC2 instances need to be selected. Manually.
- The storage volumes need to be provisioned. Manually
- If the storage volumes were replicated in S3 buckets, they need to be converted to EBS volumes
- The cloud networking needs to be done. Manually.
- If mapping the data center network, sub-nets, and IP addresses for the VMs, this needs to be done. Manually. The VPC needs to be created. Manually.
- If mapping the data center network, etc to an existing VPC, this entire process needs to be done. Manually.
- The security rules and groups need to be configured and managed. Manually.
- Be brave and launch the application in the cloud. If it doesn’t run, then be prepared for troubleshooting and isolating the root cause – this could take hours or days.
- Once the instance are up and running, Instance Tags need to be applied and IAM policies need to be assigned. This is all standard practice for a mature AWS environment
Automated Prepping Migrated Apps To Run in the Cloud:
- A majority of the work to remap and provision and orchestrate the infrastructure services was done via automation. The continual replication of application blueprint and data sync ensures that any changes in the source environment, including data changes, are reflected in the cloud environment
- It is important to note that the replicated applications are “clones” of the application in the data center. For example, if the application was using RHEL 6.2 with certain patches applied, the replicated system in the cloud did not use a Red Hat OS AMI from AWS; it is the actual replica (clone) of the RHEL 6.2, including the OS, patches, binaries, libraries et. al from the data center. A very faithful replica. The reason for this approach is to reduce the risk to running the enterprise applications in the cloud and preserve the “qualified” application environment in the data center and reproduce it in the cloud
- At this point, the cloud automation software will run a fix-up process to address any differences in the virtualization environment between the source (data center) and the destination (the cloud). For example, the VMs in the application group in the data center were running in a vSphere/vCenter virtualized environment. As you know, AWS uses a modified Xen virtualization environment, the replicated system needs to be modified so that it can run in AWS. For example, OS configuration files in the replicated/sync volumes need to be modified to reflect devices allocated in the AWS environment, Xen drivers may need to be injected, etc.
- Instance Tagging and IAM Policy Assignment can be configured
- The apps are now ready to be launched with the right environment in the cloud
As you can see, the process of migrating apps to the cloud requires attention to detail. When using manual scripts and tools, the likelihood of errors increases. Migrated apps may not operate correctly creating doubt about whether it’s the application or the migration process or the persons involved or about the cloud services. This can lower confidence for the overall project.
By contrast, automation that leverages an understanding of the application environment and automates the selection of the appropriate cloud services; preserves the critical characteristics of application from the data center while remapping to the cloud can accelerate being able to run the app in the cloud. Each successful migration boosts the confidence while also accelerating time to value.
If you are considering taking your existing/”brownfield” apps to the cloud, I urge you to evaluate cloud automation solutions like CloudVelox. The impact on your project can be dramatic.
The next blog in the series will be: Software Licensing: Can I run this app in the cloud?
Stay tuned or register for the Cloud newsletter.
Please provide your comments about this post to firstname.lastname@example.org. And if you like it, please share.