Information Gathering

There are three main methods for information gathering in relation to application migration. It’s preferable that you perform the information gathering methods in the following sequence: questionnaires, tooling, and then workshops. We discuss each method in the follow sections.

Questionnaires

It’s necessary to build a robust questionnaire to obtain the data you require from an application. This helps you make decisions about that application—for example, whether you should migrate it at all and which R scenario best fits with the application today and its future state.

In Table 5-1 we show you a list of basic sample questions (and some example answers) that you could use as the start of your questionnaire.

Table 5-2 highlights the basis of some business driver questions, which further help you to prioritize and select the best type of applications to start with.

Table 5-3 includes some sample questions that are more aligned to the development of the application and its evolution.

Table 5-4 shows sample questions relating to regulatory requirements.

These questions are the basics. You can go deeper and gather more information relating to the application and the surrounding infrastructure. Some additional areas for which you may want to build questionnaires include

• Compute

• Networking

• Storage

• Database

Questionnaires on these topics give a more robust picture than the information we provide in the tables.

Baselining

Application baselining collects data on the normal state of the application and how it operates during normal and off-peak business hours so that you can establish a pattern and provide a comparison when you move to the cloud. Baselining really should be an ongoing thing rather than being focused on a point-in-time migration because it will help you understand application characteristics that will be useful for determining problems during operations.

Gathering this data enables you to make accurate predictions of the types of services you consume within your cloud provider and helps you build budgeting and financial models from that data. If you don’t understand how the normal state looks then you can greatly affect the IT budget by making inaccurate decisions.

Baselining requires collecting two main types of data: performance and configuration.

Performance data Performance data gives you a view on how the application is responding to the workload it{{#}}8217;s being put under. Reviewing this data for a wide period rather than using a single time capture of only a couple hours will give you a more accurate profile of how the application works. Ultimately you can compare the performance data when you move the application to the cloud.

You can handle collecting performance data in a variety of ways:

• Performance monitor Performance monitor is natively built into Windows and helps you capture and visualize performance data for any performance counter available in Windows and its applications.

  Logman is a command-line utility that allows you to build performance counter sets (a collection of performance counters in a file) and invoke them to collect performance data.

  Here’s a simple way of creating a file and invoking a timed capture for performance data.

  1. Open Notepad and copy the following content into it:

     Click here to view code image

     "SystemProcessor Queue Length"
     "MemoryPages/sec"
     "MemoryAvailable MBytes"
     "Processor(*)\% Processor Time"
     "Network Interface(*)Bytes Received/sec"
     "Network Interface(*)Bytes Sent/sec"
     "LogicalDisk(C:)\% Free Space"
     "LogicalDisk(*)Avg. Disk Queue Length"

  2. Save the file and call it windowsperf.conf.

  3. Open an administrative command prompt and copy the following code into the window. This code requires you to change the data and time windows between the -b parameter and the -E parameter:

     Click here to view code image

     logman create counter baseperf -f bin -b 03/04/2018 09:40:00 -E 03/04/2018
     09:45:00 -si 05 -v mmddhhmm -o "c:perfaseperf" -cf "c:perfwindowsperf.conf"

  4. Open Performance Monitor (type perfmon and press enter). As shown in Figure 5-7, you will observe the data set after its created on the left-hand menu. Notice the data set baseperf.

     

  5. Go back to the command prompt windows and type the following:

     logman start baseperf

     This will begin the data collection for the prescribed time period you entered in the previous steps.

     After the data is collected, you can use Performance Monitor to view the data. The same procedure can be used across multiple machines, and you could wrap these in scripts to further automate the process.

• PowerShell PowerShell can help automate the collection of performance counter data using the Get-Counter cmdlet. Here is a sample script for collecting performance counter data via PowerShell:

  Click here to view code image

  $CtrList = @(
          "SystemProcessor Queue Length",
          "MemoryPages/sec",
          "MemoryAvailable MBytes",
          "Processor(*)\% Processor Time",
          "Network Interface(*)Bytes Received/sec",
          "Network Interface(*)Bytes Sent/sec",
          "LogicalDisk(C:)\% Free Space",
          "LogicalDisk(*)Avg. Disk Queue Length"
          )
      Get-Counter -Counter $CtrList -SampleInterval 5 -MaxSamples 5 | Export-Counter -Path C:PerfExample.blg -FileFormat BLG -Force

• SCOM Microsoft System Center Operations Manager (SCOM) is traditionally an on-premises monitoring tool created to give deep insight into your IT environment. You can use SCOM to collect long-term data about your environment, (using management packs and/or scripts) and it gives you insight on what the application’s normal state is.

• OMS&S Microsoft Operation Management Suite and Security (OMS&S) is a cloud-based tool that can span multiple environments (cloud and on-premises) via agent-based collection to aggregate large amounts of data—including performance data—into a single repository for analysis. You can collect any performance counter on Windows and Linux and determine an application’s normal state from this and a variety of other data.

  OMS&S can collect multiple sources of data and provide service mapping to help identify interdependencies and/or communication paths you may not be aware of. Figure 5.8 shows an example of adding Windows performance counters to Windows.

• Third-party tools

  There are many third-party tools beyond the scope of this book. Here’s a small sample:

  • Splunk

  • Nagios

  • Spiceworks

  • HP OpenView

Configuration data Configuration data enables you to look at how the application is configured at a point in time. This point-in-time view enables you to correlate its normal state of operations in conjunction with the captured performance data. Application vendors often have many settings that you can tweak for improved performance depending on the environment where the application is deployed.

When you are capturing the configuration data you can review it before moving to the cloud so you can determine any potential problems. A simple example involves authentication. If the configuration of the application requires Kerberos authentication but you haven’t planned your network around support for a Kerberos authentication method in the cloud, the application will fail.

Additionally, if you choose to step a little into the modernization of the application—for example, containers—you need to ensure you can allow the containers to connect securely to other systems using some identity model. If it is a Windows container and you haven’t stepped into modernizing the authentication methods to something like OAuth2.0, then the application still requires Kerberos and the supporting infrastructure to allow Kerberos to operate successfully.

You can collect configuration capture in a variety of different ways. Manually capturing the configuration data is an option, of course. However, using automate tools, such as capturing performance data, simplifies the process. We discuss some of these tools in the next section because some of the discovery tools or migration assistants also capture the configuration data you need.

Service Mapping

When looking at applications, you need to gather as much information as possible about the application to have a successful migration project. However, the information needs to be validated, as is always the case when you’re collecting information.

Applications that are candidates for migration usually have the original architecture diagram and protocol flows drawn up at the start, but as the development process occurs this information tends to not be updated as regularly as you require. (For application developers who do make updates, that’s great!) But the lack of information updates or inaccurate updates still can lead into the scenario in which the app was “designed” to do X, but it does Y. This becomes more important when you’re about to migrate an application.

Take a simple example of a LOB application. The vendor has described in the architecture and supporting documentation that the app uses a built-in authentication process. When you move it to the cloud, even in a rehost scenario, the application breaks! No one can log on.

You troubleshoot and discover that the application requires Kerberos authentication. Now you must either change the network configuration to support hybrid connections to your on-premises network or you must use Azure Active Directory Domain Services (if your application can support the associated Kerberos versions!).

Service mapping is the technique of taking the architecture and protocol flows and discovering whether your application is indeed operating in the capacity the specifications have defined. This will help you identify how your application is working, what processes it instantiates, how it communicates with other systems; you essentially have almost all the information you need to understand which way the application is working on the network.

Not only will service mapping help you with your application and how it communicates, you can use it to help you design your network infrastructure in a Cloud environment. From the information you can determine if you need to open additional ports or firewall rules, or you can find out if you can move the application to Cloud.

During your service mapping, if you discover an application that communicates using multicast technologies or broadcast technologies, then you will have problems moving to a cloud platform like Azure. because at the time of writing, Azure doesn’t support those technologies.

Oracle RAC is an example of one of these database platforms whose technology currently uses multicast technology in its clustering mechanisms. So, for now at least, you can’t bring it through application migration in the rehost scenario.

You can handle service mapping in a variety of ways, and it builds on the information you have collected in the previous sections of this chapter. You could use network traces in tools like Network Monitor or Wireshark. Figure 5-9 shows a sample network trace in network monitoring. We show the communications that happen normally (you will see a lot), and then we show that we can map this to a process.

Figure 5-10 shows Service Map that’s built into OMS. It shows the application server, where it’s talking, and on what ports it’s talking. On the right side we drill into process information and event details down to the DLL versions. Service maps, which has a helper agent on it, collects this data alongside the OMS agent and injects it into our log analytics workspace, so we can produce these views!

Data

Data is usually the most important part of the application. If we lose the application, we can restore the front-end components and connect back to the data stores. More importantly, we need to profile our data in the correct way. Identifying our data stores and understanding the sizing requirements, its security requirements, and its performance requirements will help us build an appropriate map to the cloud.

For example, an Azure Standard Storage account can hold up to 500 TB of data and has a 20,000 IOPS limit. If we have an application that has 400 GB of data then the storage account is perfect, but if the IOPS count required is 30,000 IOPS then the storage account won’t meet the needs. Also, an application that requires an NFS interface to the storage also discounts the Azure Storage Account. Finally, if the data stored in that data store has been classified as High Business Impact or Personal Information, we potentially need to enable encryption or more to host the data in that storage account even if its other needs have been met.

Similarly, if the data happens to be a database, the same information about performance, sizing, and so on is still required, but support also comes into play. Does the vendor support the datastores that are available in the cloud, and what are its conditions on items like blob storage and latency, for example.

Tools

An organization needs to select the correct tools to gather the depth information related to an application to aid in their decision-making process. In this section, we discuss some of the free tools available today. These tools will help you gather a lot of the information we have talked about in your environment.

Microsoft Assessment and Planning Toolkit

Microsoft Assessment and Planning Toolkit (MAPS) provides a powerful tool to collect an inventory. It’s an assessment and reporting tool to gather the information required to help you migrate your applications to the cloud.

The installed tool collects an inventory of the machines you want to target. Targets are called scenarios, and they include the following:

• Windows

• Linux/Unix

• VMWare

• SQL

• Oracle

An inventory contains the operating system, configuration, and performance data for the machine to be able to assess it for scenarios later. One of the scenarios we are most interested in is the Azure Migration Platform Scenario, which collects information to build an inventory that consists of the hardware information, OS information, IIS instances, SQL information, and web applications. Figure 5-11 shows the inventory discovery process in MAPS.

After the inventory is collected, the next step is to collect the performance data. You can run some reports prior to doing this, but collecting the performance data gives you a complete picture of how your system performs over a period of time The performance data collection should be left for as long a period as possible during normal business operations to get an accurate representation of how it performs and also to contribute to the sizing report for an Azure VM. Table 5-5 shows the output of a sizing report after the inventory and performance data have been collected.

Azure Migrate

Azure Migrate is a cloud-based service used to discover, assess, and migrate from n-premises networks to the cloud. This tool is targeted toward a virtual machine migration or a database migration and leverages existing Microsoft tools within Azure Site Recovery and the Database Migration Service. As of March 2019, this tool is scoped only to VMware environments with a planned roadmap to support additional environments. Hyper-V support is currently planned.

Azure Migrate runs an appliance that you can download via the Azure Marketplace. It connects to vCenter and scans the environment to build an inventory and assess it for migrating to the cloud.

After analysis, Azure Migrate guides you to the process of migrating your applications to the cloud.

Azure App Service Migration Assistant

Although it’s not an official Microsoft tool, the Azure App Service Migration Assistant looks at a variety of information related to an on-premises web app and determines its suitability for the cloud, specifically the Azure App Service. The Azure App Service Migration Assistant inventories and collects configuration data around an app’s binding, authentication, extensions, and so on. You receive an assessment and report based on the findings from the inventory and determine if you can easily move to the Azure App Service. The tool also performs the migration if you want.

This tool is not officially supported by Microsoft, but it’s a valuable tool in the arsenal.

Third-party tooling

There are plenty of third-part tools available today. It’s outside the scope of this book to highlight all available tools or provide in-depth guidance about them, but here is a short list of some of the major tools available. You can visit their websites for further information.

• BitTitan (https://www.bittitan.com/)

• Movere (https://www.movere.io/product/)

• CloudPhysics (https://www.cloudphysics.com/)

• Cloudamize (https://www.cloudamize.com/)

Workshops

The workshop is considered the final point in the chain of the discovery phase. It requires the participation of all the teams—infrastructure, support, operations, security, and development. Using all the information collected in the previous stages of the discovery phase, you can review each application and have a complete end-to-end discussion to determine each app’s viability for the cloud. You can use the information to agree on the appropriate R scenario to map to for migrating the application to the cloud.

These workshops also can cover the assessment work that we cover in more detail in the next section, but that assessment work is not the end purpose of the workshop.

By Type

Breaking applications into their respective types can help you discover whether there are SaaS alternatives or native PaaS frameworks that you can leverage. Figure 5-12 shows a breakdown of how you might segment applications by type.

By Business Value or Criticality

Understanding the business value and how critical it is to the organization also helps you prioritize an application for migration. If it is a mission-critical application which the business cannot operate without and requires absolute stability, this might lead to a delay in bringing the mission-critical application to the cloud until lessons are learned.

Figure 5-13 shows an example of how you might segment the applications by business value.

By Complexity and Risk

Finally, you can break applications into segments of complexity and risk. If an app has low risk and low complexity for the migration, it receives a high score for being approached. Figure 5-14 shows how you might segment based on complexity and risk.

You can score each area and bucket and use the cumulation of those scores to build a prioritization table. Figure 5-15 shows a sample prioritization table derived from the segments and the discovery data. It also includes other potential factors you may weigh in the decision of prioritization. The weight factor is from 1 to 5: 1 is not important or low, and 5 is very important or high.

This table holds a list of all the applications in an organization. You use one like it to determine the path and a timeline of events to migrate the application estate to the cloud.

Functional Tests

In the functional testing phase of the migration factory, you validate that the application operates as expected. You can do a variety of tests, including synthetic transactions with validate data queries or logon procedures. Other elements including cross-platform integration points and reports. You also would test for component upgrades of an application. The tests are constructed with IT operations and end-user scenarios in mind, and the scores must be 100%. Any failure should be fed back into the remediation phase of the migration factory. Table 5-6 shows a sample definition for a functional test for an application.

Performance Tests

In the performance testing phase of the migration factory, you validate that the application operates equally or better than the on-premises infrastructure. Testing performance covers standard end-user scenarios, including reporting, login times, querying for new data, and creating records. Testing also should cover internal application metrics that aren’t traditionally visible to a user but can be used to correlate reported events. The performance testing can be validated against the baselines previously performed in the discovery phase. Application architectures may change because of the performance testing during the remediation phase.

Performance tests can be defined similarly to functional tests. They should focus on performance metrics for the application. Table 5-7 shows a sample performance test you could start with to document and then build the automation test from.

Performance testing also will describe load scenarios and expected performance. Here are some sample expectations under load testing of a web app with a load of 500 users.

• Throughput 100 requests per second (ASP.NETRequests/sec performance counter)

• Requests Executing 45 requests executing (ASP.NETRequests Executing performance counter)

• Avg. Response Time 2.5-second response time (TTLB on 100 megabits per second [Mbps] LAN)

• Resource utilization thresholds

• Processor\% Processor Time 75 percent

Hopefully by now, you might begin to understand that these tests can be duplicated from the original application build and testing process. This is the whole point of DevOps—to simplify and automate while maintaining the same or better coverage as before. You may also find that the testing from the migration factory feeds back into the original testing scenarios and improves them at the initial build so that each functional upgrade undergoes a rigorous end-to-end test.

Teams

Pods form teams. Although there will be a core migration team to support the effort, there also will be a variety of other teams to support the overall migration factory. Figure 5-19 shows some of the additional teams (which can also be made up from pods).

Our approach is to have a core migration team that will run our factory and get our application estate to the cloud in the quickest but best way possible. Our app team will be responsible for discovery and profiling our applications. Our advance team is responsible for ensuring the migration team knows what needs to change in its processes to support any new types of applications, especially if the advance team has predicted or identified a pattern in the applications still to come.

The exception team handles the one-offs in the applications, which have pretty unique circumstances, or applications that have entered the factory and have been identified as having unique characteristics that don’t conform to a standardization process. The QA teams ensure that the applications have been validated correctly in the cloud and are the guardians of the cutover.

The teams shown in Figure 5-19 work closely together. As in the DevOps life cycle, they feed into each other with valuable feedback to improve the overall migration factory.

Backup

When moving your application to the cloud, especially in a rehost scenario, you have to consider how you back up the data. Traditionally, in an on-premises network you have a backup server or farm that deploys agents out to the application hosts you want to back up and then create a schedule. These backup hosts are connected to storage and some archival tape. These hosts also need relatively good network connectivity between the backup server and the application host, especially if there are large amounts of data that need to be backed up.

To a degree, we could replicate this setup in the cloud by building a backup server virtual machine and attaching lots of data disks. Then install a virtual tape library drive that emulates the functionality and have it backed up on some other storage type.

Virtual machines consume resources in Azure and have sizing guidelines that directly affect the CPUs, RAM, and disk it can support (among other things) and heavily impacts the cost.

For example, the max size disk supported today is 4 TB in Azure. We can attach up to 64 disks to a virtual machine, which allows us to have a maximum space of 256 TB (that’s quite a lot). If we match this to a virtual machine size, such as Standard_L16s, to ensure we get the IOPS and throughput needed to meet our RPO and RTO for our backup, our estimated monthly cost runs into $33,000 USD per month. What’s interesting is that this is just for the data; it’s not a complete virtual machine backup!

That solution isn’t cost-effective, and we shouldn’t implement it. If we take an optimized approach to backups, we would look at a native cloud service. Azure has an integrated backup platform called Azure Backup. If we want to back up the virtual machines with all the data and applications installed in a snapshot consistent method, the cost for 256 TB of storage for Azure Backup is estimated to be $11,000 USD per month. If we still require guest OS–level backup, the cost rises because we must include a virtual machine to run Azure Backup Server. Figure 5-20 shows a sample virtual machine blade with Azure Backup integrated directly into it. You can see job information and the recovery points available. This shows the default policy of one backup per day.

Another example scenario involves SQL backup. SQL can export the backup data directly to an Azure storage blob, so we could use the Azure Backup service to get a full virtual machine backup and then run a SQL backup job to export the data to Azure Storage. SQL also can restore from Azure Storage as well.

Backup also changes when you consume Azure PaaS services. If you use Azure SQL Database, for example, you don’t have an agent-based backup available; you do not get access to the “host” server. You select the backup options that the PaaS service makes available with the appropriate SLAs.

In the case of Azure SQL, the native backup runs every five minutes and creates a restore point automatically. Depending on the tier of service you select, you can retain data for 35 days and then integrate it into a recovery vault for longer-term retention.

Figure 5-21 shows a sample of the restore recovery point blade for an Azure SQL Database. Notice that we have the oldest recovery point available and then a restore point selected. You can modify the date and time of the restore point to any date going back to the oldest recovery point.

Monitoring

Monitoring can be different when it comes to the cloud, with different technologies in the cloud, with different cost structures (i.e. charging for egress traffic). You need to spend time thinking of what you need to know and how to achieve it in cloud. If you choose the standard rehost scenario then you can use existing tooling to monitor the virtual machines you have migrated as long as you have the appropriate connectivity between the virtual machines and the monitoring system. If you choose a refactor scenario then the monitoring system that’s currently in place has to be examined to determine whether it’s still fit for the purpose.

The deeper and more cloud-native your network becomes, the less effective the existing tool becomes. For example, if you have an agent-based monitoring system and you move away from virtual machines to a serverless architecture, where do you deploy the agent? How do you determine the health state and provide alerts when problems occur?

Another problem with trying to maintain an existing monitor system is that you’re not receiving the latest updates of the latest services that are happening in cloud. If you think at the pace at which the cloud evolves and look at the vendor’s update cycle, it’s generally significantly behind where the services are today. Even with the updates you can’t take advantage of the powerful machine learning and AI algorithms that cloud monitoring solutions are implementing to detect more sophisticated threats.

Figure 5-22 and 5-23 show two different security-related dashboards that are built from Microsoft rulesets; those rulesets have been built from the knowledge that Microsoft has attained while running global cloud services.

OMS&S is an example of a cloud monitoring tool that can collect data native from Azure virtual machines and native PaaS services.

Microsoft takes the information it has obtained from running its global services and creates rules and patterns to detect erroneous events. You can build customer-specific rules and export data to PowerBI to build visually striking dashboards. This tool can collect data from any internet-connected source via a variety of methods, including the agent-based collection.

Disaster Recovery

When you migrate your application to the cloud, how you recover the system in the event of a critical failure has to change. Traditionally you would define a recovery point and recovery time objective, and that information would dictate the type of system you had to build. This could be a costly endeavor because you may have to duplicate the hardware of the production system and maintain the space for this duplicate hardware to exist.

If you move the application with a rehost scenario, what do you duplicate in terms of hardware? How do you replicate the virtual machine to a different location?

Azure Site Recovery is a cloud-native tool that replicates a virtual machine to a different region and enables you to perform a recovery of the virtual machine in the event of a disaster. Azure Migrate—which is a discovery, assessment, and migration tool—is built upon Azure Site Recovery because it also contains features that enable you to have Azure as your secondary site for disaster recovery for your on-premise systems. There is no duplicate hardware required, and in most cases network connectivity needs to be put in place with VPN technology (although we highly recommend a network assessment to determine the bandwidth required). Figure 5-24 shows an Azure-to-Azure disaster recovery scenario being configured.

Azure to Azure Site Recovery reduces the cost of having to maintain secondary datacenters and secondary hardware. If you use the “playbook” scenarios available in Azure Site recovery, you can greatly reduce the human touchpoints required to recover from a disaster.

Azure Web Apps

If you have an application that has a web-tier front end, an easy way to optimize the application is to move the web app to an Azure Web app. This ensures you can benefit from the scalability, reliability, and availability of the Azure App Service platform. It also removes another virtual machine that you must manage.

The authentication methods and frameworks the application consumes may need to be updated before it can be hosted on the platform. You can integrate Azure Web app deployment mechanisms into your DevOps frameworks, so you can make changes and deploy and test rapidly.

Containers

If the application you migrated has a web tier, you also could migrate it to a container to reduce the overall footprint. With containers, you can integrate the packaging and deployment into a DevOps pipeline and have end-to-end deployment. You can use a managed container service (AKS) to run your container estate. Containers give you the chance to optimize in a lift-and-shift manner from a virtual machine to a container.

When you use containers, you become less worried about some management concepts like guest OS monitoring. Your monitoring footprint changes, and you can gather telemetry from the managed service rather than having hooks into a custom-built platform.

Azure Functions

Azure Functions provide even more opportunity for shrinking the size of a deployment and using native PaaS systems and serverless compute as much as possible. If your application had background web jobs or you had a “processing tier” in the application, you could migrate in source code these jobs to Azure functions that would execute the jobs. The job is only charged for the length of runtime, and Azure Functions provide all the necessary resources to execute that job.

In comparison, if you had to run a virtual machine to execute these jobs, you would constantly be paying for a virtual machine even when it had no work to do. You could also run into scale issues if the virtual machine ran out of resources.

Powerbi

This is one of the simplest optimizations you could do, and the benefits that an application would receive are also unquantifiable! You could use PowerBI to generate reports for your application or your business. This would reduce the need for a reporting server. PowerBI scales on demand because it’s a cloud service, and it can integrate directly with Azure SQL as a data source (as an example, it can also connect to other data sources).

Azure Sql Managed Instances

If your application requires a SQL database, rather than provisioning a dedicated virtual machine you could optimize and use a SQL Managed Instance in Azure. Managed instances provide all the functionality of a virtual machine with SQL installed but have none of the management overhead because it’s handled by the platform.