Download and install LiveData Migrator

Follow these steps to install and deploy the LiveData Migrator service.

Get your install key

The install key is a string that is unique to your migrator resource and will need to be provided when running the installation script.

Get your install key in the Azure Portal

1. In the Azure Portal, navigate to the LiveData Migrator resource page.

2. From LiveData Migrator -> Get Started, click Get Install Key.

3. The Install Key is provided as a Base64 encoded string, for example:

   Qc-GfyyhsofHJUosKKghsoHUNOPPSAPLhyTRDCju-ID

   This string will be used in the install steps.

Get your install key with the Azure CLI

az livedata migrator get-install-key --resource-group <resource_group> --migrator-name <migrator_name>

• --resource-group - the name of the resource group where your Migrator resource is located.
• --migrator-name - the name of your Migrator resource.

A Base64 encoded string is provided as the output from the command:

Example output

Qc-GfyyhsofHJUosKKghsoHUNOPPSAPLhyTRDCju-ID

This string will be used in the install steps.

Install the LiveData Migrator service

1. Use SSH to sign in to your HDFS cluster edge node or HDFS Sandbox, depending on your setup.

   1. HDFS edge node

      Log in to HDFS edge node

      ssh <user>@<hdfs_edge_node_hostname>

   2. HDFS Sandbox

      Log in to HDFS Sandbox host

      ssh <user>@<hdfs_sandbox_hostname>
      Log in to HDFS Sandbox container

      docker-compose exec -w /tmp sandbox-hdp-vanilla bash

2. Download the installation script to your own environment (HDFS edge node) or the HDFS Sandbox by running the following command:

   wget https://cdn.wandisco.com/ldma/install_ldma.sh

3. Make the install script executable and run it using the following command:

   Red Hat/CentOS

   chmod +x install_ldma.sh && ./install_ldma.sh

4. When prompted, provide your install key:

   Example

   Install key: Qc-GfyyhsofHJUosKKghsoHUNOPPSAPLhyTRDCju-ID

   The installation will now continue, wait for it to complete before moving on to the next step.

5. Enable the LiveData Migrator service:

   CentOS 6 only

   ln -s /opt/wandisco/livedata-migrator/livedata-migrator.jar /etc/init.d/livedata-migrator
   Debian/Ubuntu/Red Hat/CentOS 7

   systemctl enable livedata-migrator

6. Check the LiveData Migrator service is running:

   CentOS 6 only

   service livedata-migrator status
   Red Hat/Debian/CentOS 7

   systemctl status livedata-migrator

7. Check the Hive Migrator service is running:

   CentOS 6 only

   service hivemigrator.service status
   Red Hat/Debian/CentOS 7

   systemctl status hivemigrator.service

If you need help, run az livedata migrator create --help for more options.

The LiveData Migrator and Hive Migrator services are run using the following user/group settings:

Service	Default User/Group	Configuration file	Configuration properties
LiveData Migrator	hdfs:hdfs	/etc/wandisco/livedata-migrator/vars.env	USERNAME=hdfs and GROUPNAME=hdfs
Hive Migrator	hive:hadoop	/etc/wandisco/hivemigrator/vars.sh	HIVE_MIGRATOR_SERVER_USER=hive and HIVE_MIGRATOR_SERVER_GROUP=hadoop

To run services with different users or groups, edit the noted configuration properties in their corresponding configuration file.

If you're running a systemd-based operating system, create an override service file:

systemctl edit livedata-migrator

Next steps

Once you've installed LiveData Migrator, configure Kerberos if the HDFS cluster you are using is running Kerberos.

If the HDFS cluster is not running Kerberos, you may skip this step and create a target for your data migrations and/or create a metadata target for metadata migration.

Troubleshooting

If you are having problems downloading and installing LiveData Migrator, you can find solutions in the troubleshooting guide.

If you are running Kerberos on the HDFS cluster you have installed LiveData Migrator to and the dashboard displays either an Install State or Meta Install State of "InstalledNoSourceDetected", configure Kerberos for LiveData Migrator.