In this blog we define the process for earning AWS customer credits when migrating Data and AI workloads to Databricks on Amazon Web Services (AWS) with the AWS Migration Acceleration Program (MAP). We will show you how to use AWS MAP tagging to identify new migrated workloads such as Hadoop and Enterprise Data Warehouses (EDW), in order to ensure workloads qualify for valuable AWS customer credits. This information is helpful for customers, technical professionals at technology and consulting partners, as well as AWS Migration Specialists and Solution Architects.
Databricks is the data and AI company. More than 7,000 organizations worldwide — including Comcast, Condé Nast, H&M and over 40% of the Fortune 500 — rely on the Databricks Lakehouse Platform to unify their data, analytics and AI. Founded by the original creators of Apache Spark™, Delta Lake and MLflow, Databricks is on a mission to help data teams solve the world’s toughest problems. Databricks is recognized by Gartner as a Leader in both Cloud Database Management Systems and Data Science and Machine Learning Platforms.
The Databricks Lakehouse on AWS unifies the best of data warehouses and data lakes in one simple platform to handle all your data, analytics and AI use cases. It’s built on an open and reliable data foundation that efficiently handles all data types and applies one common security and governance approach across all of your data and cloud platforms.
The AWS Migration Acceleration Program (MAP) is a comprehensive and proven cloud migration program based upon AWS’s experience migrating thousands of enterprise customers to the cloud. Enterprise migrations can be complex and time-consuming, but MAP can help you accelerate your cloud migration and modernization journey with an outcome-driven methodology.
MAP provides tools that reduce costs and automate and accelerate execution through tailored training approaches and content, expertise from AWS Professional Services, a global partner network, and AWS investment. MAP also uses a proven three-phased framework (Assess, Mobilize, and Migrate and Modernize) to help you achieve your migration goals. Through MAP, you can build strong AWS cloud foundations, accelerate and reduce risk, and offset the initial cost of migrations. Leverage the performance, security, and reliability of the cloud.
Migrated resources must be identified with a specific map-migrated tag (tag key is case sensitive) to ensure AWS credits are provided to customers as an incentive and to reduce the cost of migrations. The tagging process explained below should be used for Hadoop, Data Warehouse, on-premises, or other cloud workload migrations to AWS.
The following infographic provides an overview of the seven-step process:
Set up your Databricks workspace via Cloud Formation or the Databricks account console in less than 15 minutes.
Provide the Migration Program Engagement ID (MPE ID is received after signing an AWS MAP Agreement with your AWS representatives) on the CloudFormation stack to be used to create the dependent AWS objects. This will create Cost and Usage Reports (CUR) and generate a server ID to be used by the AWS Migration Hub for migrations.
After the AWS CloudFormation is run successfully, copy the migration hub server IDs generated from the output and tag them as a value to the map-migrated tag set on the Databricks clusters used as the target clusters for migration. In addition to Databricks clusters, follow the same tagging mechanism across other AWS resources used for the migration, including the Amazon S3 buckets and Amazon Elastic Block Store (EBS) volumes.
Spin up the Databricks clusters for migration and tag them with map-migrated tags one of three ways: 1. the Databricks console, 2. the AWS console, or 3. the Databricks’ API and its cluster policies.
Be sure to tag the associated Amazon S3 buckets
Once all Databricks on AWS resources are tagged appropriately, perform the migration and track the usage via AWS Cost Explorer. Organizations who have signed an AWS MAP Agreement and performed all the required steps will see credits applied to their AWS account. Remember to activate the MAP tags in the Cost Allocation Tags section of the AWS Billing Console. The map-migrated tags may take up to 24 hours to show up in the Cost Allocation Tags section after you have deployed the CloudFormation template.
Services > Billing > Cost & Usage Reports.
Summary
In this blog we explained how to successfully tag migrated workloads to Databricks on AWS using the AWS Migration Acceleration Program (MAP). Using tags to identify migrated workloads will benefit customers through AWS credits. The steps involved include generating server IDs on the AWS Migration Hub, setting up cost allocation tags, using MAP tags to target Databricks clusters, automatically delivering cost and usage reports, and tracking usage via Cost Explorer.
Questions? Email us at [email protected].
Additional Resources
AWS Migration Acceleration Program (MAP)
Hadoop Migrations
SAS Migrations
Data Warehouse Migrations
