Before you start
Make sure you have the following setup and resources to successfully install and use LiveData Platform for Azure.
Supported Versions#
Filesystems supported as sources:
- HDP 2.6.3, 2.6.5, 3.1.0
- CDH 5.13, 5.15, 5.16, 6.2, 6.3
- CDP 7.1.4
Note: All HDFS versions from Hadoop 2.6 should work, however, if you need additional support, contact WANdisco Support.
Filesystems supported as targets:
- ADLS Gen 2
Supported Hive Migrator metastores:
- Azure HDI 3.6/4.0 Hive External Metastore
- Azure SQL DataBase
Azure CLI:
- Minimum version - 2.26.0
Source environment#
Provide your own Hadoop cluster or use the HDFS Sandbox if you want to test LiveData Platform for Azure.
info
The HDFS Sandbox is available for both LiveData Migrator for Azure and LiveData Plane for Azure.
For your own Hadoop cluster, the following prerequisites are required:
- Root access to the Hadoop cluster.
- Network requirements must be met.
- For example, LiveData Migrator for Azure requires that ports 5671 and 443 are open for outbound traffic.
Azure resources#
Make sure your selected region is supported and consistent across all your resources.
Resource group to contain the Azure zone resources.
- Owner or Contributor level access to the resource group.
Create an ADLS Gen2 storage account and container with hierarchical namespace enabled.
Create VNet/Subnet that is accessible from your intended on-premises environment, or use an existing one.
Access control for your Azure resources. We recommend Managed identities. For help using a VM managed identity to access Azure Resource Manager, see the Microsoft guidance for Linux or Windows.
When using managed identities, you must assign the appropriate roles to your managed identity as described in security details.
You can also use storage account access keys for access control.
Resources for LiveData Migrator metadata migrations#
To migrate metadata, you will require the following:
- An Azure SQL Database in the same Azure resource group as LiveData Migrator.
Supported regions#
The following regions are supported.
- Australia East
- Canada Central
- East Asia
- East US 2
- East US
- Korea Central
- North Europe
- Southeast Asia
- South Central US
- Sweden Central
- UK South
- West Europe
- West Central US
- West US
- West US 2
- West US 3
note
LiveData Migrator now supports the Australia East, Canada Central, Korea Central, Sweden Central and UK South regions. If you'd like to use these regions, but don't see them in the dropdown list in Azure Portal, you might need to re-register the resource provider.
Next steps#
As an additional prerequisite to use LiveData Migrator for Azure, register the LiveData resource providers.