Skip to main content

Troubleshooting

This article details some issues you may face when you install or use LiveData Migrator for Azure, as well as steps to help you resolve them.

Make sure you meet the Prerequisites to avoid common errors.

Registering the LiveData resource provider​

The following section provides details on errors you may encounter when registering the LiveData resource provider.

Insufficient authorization​

To register resource providers, you must have the Contributor role (or greater).

If you don't have the relevant Azure Active Directory (AAD) permissions, you'll see an error like this:

The client 'user.name@example.com' with object id 'ba878430-1a00-4056-acf3-5678b8k0e3de' does not have authorization to perform action 'Wandisco.Fusion/register/action' over scope '/subscriptions/9087fget-7697-4e7d-g908-a5a3ae908654' or the scope is invalid. If access was recently granted, please refresh your credentials.

You will need to get in contact with the team that manages your subscription and either get the appropriate permissions or ask them to register the LiveData resource provider for you.

Installing the LiveData extension with the Azure CLI​

The following section provides details on errors you may encounter when installing the LiveData Extension for the Azure CLI.

AZ: command not found​

If you don't have the Azure CLI installed when you try to add the LiveData extension, the command line returns this error:

az: command not found

Please check the Azure CLI is installed and try again. To check if the Azure client is installed and running, enter az.

See Install the Azure CLI on Linux in the Microsoft documentation for installation instructions.

LiveData extension is not compatible​

The following errors may occur if you enter an incorrect --source URL when adding the extension:

Example 1
az extension add --source https://cdn.wandisco.com/az-cli/extensions/frp/0/fusion-0-py2.py2-none-any.whl

Are you sure you want to install this extension? (y/n): y
The 'None' extension is not compatible with this version of the CLI.

You have CLI core version 2.3.1, and this extension requires a min of 2.0.67 and max of 2.1.0.

Please install a compatible extension version or remove it.
Example 2
az extension add --source https://wandiscopublicfusion.blob.core.windows.net/wandiscopublicfusion/az-cli/extensions/frp/0.1.0/fusion-0.1.0-py2.py3-none-an.whl
Are you sure you want to install this extension? (y/n): y
Request to https://wandiscopublicfusion.blob.core.windows.net/wandiscopublicfusion/az-cli/extensions/frp/0.1.0/fusion-0.1.0-py2.py3-none-an.whl failed with 404

See the LiveData extension for Azure CLI section for the correct URL.

Installation prerequisites not met for the LiveData extension​

In certain cases and on certain operating systems, the LiveData extension may fail to install due to missing prerequisites, often with an error about the missing dependency.

Examples of the error:

Example 1
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.
Example 2
cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.

If you encounter this problem, make sure you have all of the prerequisites installed.

LiveData Migrator​

Download command is not found​

Wget may not be pre-installed on your distribution. If you get a wget: command not found error, install wget by running yum install wget on an RPM-based operating system, or apt-get install wget on a Debian-based operating system. Try the download command again after.

Access is denied when installing LiveData Migrator​

Ensure that you make the LiveData Migrator install script install_ldma.sh executable and run it as the root user. For example:

chmod +x install_ldma.sh && sudo ./install_ldma.sh

Resource group not found when creating a migrator instance​

When you run the command to create a LiveData Migrator resource in the Azure CLI, you'll get a "resource group not found" error if your resource group doesn't exist in the subscription you're using.

To fix it, create a new resource group in the subscription you're using and use its name in the migrator resource creation command.

You'll also receive this error if you're using the wrong subscription. To fix this, switch your subscription to the one with the correct resource group.

DeploymentFailed error when creating a LiveData Migrator resource​

When you create a LiveData Migrator resource in Microsoft Azure, you need to have the LiveData resource provider registered in your subscription.

If you do not, you may get the following error:

"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
  {
    "code": "Conflict",
    "message": "{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"ResourceCreationFailed\",\r\n        \"message\": \"S2S Pipeline Failed\"\r\n      }\r\n    ]\r\n  }\r\n}"
  }
]

To fix this issue:

  1. Register the LiveData resource provider again.
  2. Create a new LiveData Migrator resource.

DeploymentFailed error when creating an Azure SQL metadata target​

Before you create an Azure SQL metadata target, you need to create a firewall rule in your Azure database to allow access from your LiveData Migrator instance.

If you don't, the following error occurs:

"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
  {
    "code": "ExecutionFailure",
    "message": "HiveMigratorClientApi Error: Cannot open server `mysqlserver` requested by the login. Client with IP address '192.0.2.0' is not allowed to access the server. To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range. It may take up to five minutes for this change to take effect. ClientConnectionId:1eXaMpLe-0a0a-0000-a000-1cOnNeCtIoN1"
  }
]

To fix this issue:

  1. Select your Azure SQL database resource in the Azure Portal.
  2. In the navigation menu, select Networking under the Security section.
  3. Select Add a firewall rule.
  4. Create a firewall rule that allows traffic from the same client IP address that the error returned (192.0.2.0 in the above example).
  5. Save the rule.

LiveData Migrator for Azure can now connect to the Azure SQL database. You may need to wait 5 minutes for the changes to take effect.

Source filesystem can't be found​

If your Hadoop client isn't available within the systemd environment, LiveData Migrator won't be able to detect your source filesystem and you'll get an error when you try to create a migration. To resolve this, export the path to system variables to make the Hadoop tool available to the service with the following command:

sudo systemctl set-environment PATH=$PATH

ResourceCreationFailed: Failed to Authenticate with ADLS​

When attempting to create a MigratorTarget, the above response can indicate an error authenticating the credentials provided.

Possible reasons for this error include:

  1. Your entered credentials are incorrect.
  2. Your shared secret has expired. In Azure, this may happen automatically if an exp (expiry time) is configured for the secret.
  3. Your shared secret has been revoked.
  4. Clock drift on the host has caused the client's time to be out of sync with the server, rendering calls invalid due to illogical time values specified in the call. For example, the call may claim it is from the future or claim it is from the past (queries from the past are considered replays by the server).

'Migrator is down' error when adding a target or creating an exclusion​

If you see a Migrator is down error in Azure Portal when adding a target, it means LiveData Migrator for Azure couldn't communicate with the LiveData Migrator instance on your filesystem. This can happen in several cases:

  1. Your filesystem isn't running.
  2. The LiveData Migrator service isn't running on the filesystem.
  3. Connection to your filesystem is lost.

Check your filesystem is active and run the following command to start the LiveData Migrator service: systemctl start livedata-migrator. Configure your network to allow communication between your LiveData Migrator instance and the Azure Portal.

The service has not been started​

Ensure the service is running. Run the following command: systemctl start livedata-migrator.

ResourcePurchaseValidationFailed​

If your Microsoft Azure purchase state fails to validate with Microsoft Azure, you'll get the following error:

"code": "ResourcePurchaseValidationFailed",
"message": "User failed validation to purchase resources. Error message: 'Offer with PublisherId: 'wandisco', OfferId: 'ldm' cannot be purchased due to validation errors. For more information see details. Correlation Id: '9ab979d4-2bb2-4bac-b45c-fdb1a2552978' The 'unknown' payment instrument(s) is not supported for offer with OfferId: 'ldm', PlanId 'metered-v1'. Correlation Id

You need a paid Microsoft Azure subscription to use LiveData Migrator. Make sure you're using a Microsoft payment plan that's tied to a credit card.

LiveData Migrator's install state/meta install state show as "InstalledNoSourceDetected" in the overview panel​

During installation, LiveData Migrator automatically sets its HDFS host as the source filesystem. However, the installation will fail if the filesystem runs Kerberos and you haven't provided the Kerberos security details.

If you're running Kerberos on your HDFS cluster, you must configure Kerberos for data migration and metadata migration to allow LiveData Migrator to access and transfer data.

Access Rights Manager (ARM) request for 'Kerberos Settings' fails with a 'Couldn't connect to Hive metastore database' error​

This error occurs when LiveData Migrator can't reach the JDBC connection URL for your Hive metastore. This could mean your database host is unreachable, or that the value of javax.jdo.option.ConnectionURL in your hive-site.xml configuration file is incorrect.

To fix this issue, ensure that the Hive metastore database can be accessed from your LiveData Migrator for Azure host filesystem through the JDBC connection URL. If you can't connect to the database, check the javax.jdo.option.ConnectionURL value in hive-site.xml is correct and follows the format jdbc:{database type}://{Host}:{Port used by database type}/hive.

LiveData Migrator's install state shows as "NotInstalled" in the overview panel​

The clock on your HDFS node may be running several minutes slow. This causes messages sent to the Azure Portal to be wrongly considered to have expired already, since the send time appears to have been several minutes before the information arrived at Microsoft Azure.

To fix this issue, use the NTP Pool to synchronize your node's clock.

Validation Failure: Your CLI LiveData Extension version is no longer supported​

You'll get the following error if you're running an Azure CLI LiveData extension version lower than 0.14.1:

ERROR: ValidationFailure: Your CLI LiveData Extension version is no longer supported

To fix this, install the Azure CLI LiveData extension again to upgrade it to the latest version. You can use the az extension show command to check you're running a higher version than 0.14.1:

az extension show --name livedata

Data transfer agents​

Restore connection to data transfer agent​

If an agent's configuration properties or credentials are moved or become corrupt, the connection from Data Migrator to the agent is lost. The agent is displayed as unhealthy.

The following notification is displayed:

Couldn't restore connection to data transfer agent <DTA_NAME>, <HOSTNAME>, <PORT NUMBER> at <DATETIME>.
Remove and add the agent again. View agents.

We recommend you remove the agent that is listed as unhealthy and add it again. This should resolve the problem. If the problem persists, contact WANdisco Support.

Setting up a Databricks metadata target fails due to missing driver​

You may have downloaded an unsupported version of the Databricks JDBC driver.

  • If you're running Hive Migrator 1.11 or earlier, select and download version 2.6.22 from the JDBC Drivers Archive.
  • If you're running Hive Migrator 1.12 or later, select and download version 2.6.25 from the JDBC Drivers Archive.

Migration exists on LiveData Migrator host system but not in the Azure Portal​

If a migration you deleted in the Azure Portal still remains in your on-premises LiveData Migrator installation, follow these steps to fully remove it:

  1. Recreate the migration in the Azure Portal with the same Name and Path.
  2. Delete the migration in the Azure Portal again.

Your local copy of the migration will reflect the changes you made in the portal.

Migrations are slow or stop automatically​

Some default HDFS NameNode configuration values can create a performance bottleneck for inotify clients like LiveData Migrator for Azure, which leads to migrations stalling. Use the values we recommend on the Configure your HDFS Cluster page to optimize migration performance.

How do I get more help?​

If you have a technical query about LiveData Migrator for Azure and can't find help on this page, see WANdisco's community knowledge base for LiveData Migrator for Azure or WANdisco's support resources.