Download and install LiveData Migrator
Follow these steps to install and deploy the LiveData Migrator service.
#
Get your install keyThe install key is a string that is unique to your migrator resource and will need to be provided when running the installation script.
- Get your install key in the Azure Portal
- Get your install key with the Azure CLI
In the Azure Portal, navigate to the LiveData Migrator resource page.
From LiveData Migrator -> Get Started, click Get Install Key.
The Install Key is provided as a Base64 encoded string, for example:
This string will be used in the install steps.
--resource-group
- the name of the resource group where your Migrator resource is located.--migrator-name
- the name of your Migrator resource.
A Base64 encoded string is provided as the output from the command:
This string will be used in the install steps.
#
Install the LiveData Migrator serviceUse SSH to sign in to your HDFS cluster edge node or HDFS Sandbox, depending on your setup.
HDFS edge node
Log in to HDFS edge nodeHDFS Sandbox
Log in to HDFS Sandbox hostLog in to HDFS Sandbox container
Download the installation script to your own environment (HDFS edge node) or the HDFS Sandbox by running the following command:
Make the install script executable and run it using the following command:
Red Hat/CentOSWhen prompted, provide your install key:
ExampleThe installation will now continue, wait for it to complete before moving on to the next step.
Enable the LiveData Migrator service:
CentOS 6 onlyDebian/Ubuntu/Red Hat/CentOS 7Check the LiveData Migrator service is running:
CentOS 6 onlyRed Hat/Debian/CentOS 7Check the Hive Migrator service is running:
CentOS 6 onlyRed Hat/Debian/CentOS 7
If you need help, run az livedata migrator create --help
for more options.
The LiveData Migrator and Hive Migrator services are run using the following user/group settings:
Service | Default User/Group | Configuration file | Configuration properties |
---|---|---|---|
LiveData Migrator | hdfs:hdfs | /etc/wandisco/livedata-migrator/vars.env | USERNAME=hdfs and GROUPNAME=hdfs |
Hive Migrator | hive:hadoop | /etc/wandisco/hivemigrator/vars.sh | HIVE_MIGRATOR_SERVER_USER=hive and HIVE_MIGRATOR_SERVER_GROUP=hadoop |
To run services with different users or groups, edit the noted configuration properties in their corresponding configuration file.
If you're running a systemd
-based operating system, create an override service file:
#
Next stepsOnce you've installed LiveData Migrator, configure Kerberos if the HDFS cluster you are using is running Kerberos.
If the HDFS cluster is not running Kerberos, you may skip this step and create a target for your data migrations and/or create a metadata target for metadata migration.
#
TroubleshootingIf you are having problems downloading and installing LiveData Migrator, you can find solutions in the troubleshooting guide.
If you are running Kerberos on the HDFS cluster you have installed LiveData Migrator to and the dashboard displays either an Install State or Meta Install State of "InstalledNoSourceDetected", configure Kerberos for LiveData Migrator.