Create path mappings
Create path mappings when you want to define an alternative migration path on your target filesystem.
Path mappings will automatically apply to any new migrations.
tip
If path mappings are not required for your migrations, you can start to migrate your data.
#
Why use path mapping?When path mapping is not used, the source path on your source filesystem will be created on your target filesystem. This may not always be suitable for the purposes of your migration.
The following example shows a common use-case where the default Hive metastore warehouse directory for Hortonworks is /apps/hive/warehouse
, and the default for Azure HDInsight is /hive/warehouse
.
Example for Hortonworks to HDInsight | Source path | Target path |
---|---|---|
Without path mapping | /apps/hive/warehouse | /apps/hive/warehouse |
With path mapping | /apps/hive/warehouse | /hive/warehouse |
Using path mapping enables any migrated data (relating to managed Hive databases and tables) to be stored in the equivalent default location on the target.
- Create a path mapping in the Azure Portal
- Create a path mapping with the Azure CLI
#
Add a path mappingOnce a path mapping has been created, any new migrations will automatically apply them (if the path being mapped is within the migration boundary).
Add a path mapping by following these steps:
- In the Azure Portal, navigate to the LiveData Migrator resource page.
- From the LiveData Migrator menu on the left, click Path Mappings.
- Click Create.
- Complete the Basic details:
- Name: A reference name for your path mapping.
- Path for Source Filesystem: The directory on your source filesystem to map from.
- Target Filesystem: Your target filesystem name (ADLS Gen2 storage account and container).
- Path for Target Filesystem: The directory on your target filesystem to map to.
- Click Review + create.
- Check that the path mapping details are correct and then click Review and create.
#
View a migration to see the applicable path mappingsView the path mappings applied to a migration by following these steps:
- In the Azure Portal, navigate to the LiveData Migrator resource page.
- From the LiveData Migrator menu on the left, click on Data Migrations.
- Select the migration name that you want to view.
- View the path mappings that have been applied to the migration under the Path Mapping field in Essentials.
#
Remove a path mapping- In the Azure Portal, navigate to the LiveData Migrator resource page.
- From the LiveData Migrator menu on the left, click on Path Mappings.
- Select the path mapping that you want to delete (using the checkbox).
- Click Delete and confirm the deletion.
note
When a path mapping is deleted, any migrations using the path mapping will not be affected. They will continue to function as if the path mapping were still in place.
#
Add a path mappingOnce a path mapping has been created, any new migrations will automatically apply them (if the path being mapped is within the migration boundary).
Run the following command to create a path mapping:
See the Azure CLI LiveData Extension - Path Mapping Create section for help with these parameters.
#
Remove a path mappingRun the following command to remove a path mapping:
See the Azure CLI LiveData Extension - Path Mapping Delete section for help with these parameters.
note
When a path mapping is deleted, any migrations using the path mapping will not be affected. They will continue to function as if the path mapping were still in place.
#
Path mapping limitationPaths on the source filesystem will not be migrated if they conflict with target paths that are created by the path mapping rules.
An example would be if you had a path mapping rule such as below:
Source path | Target path |
---|---|
/repl/folder/old | /repl/folder/old/archive |
In this case, if a new directory is created on the source /repl/folder/old/newdir
, it is migrated to the target as /repl/folder/old/archive/newdir
.
However, if the following directory is created on the source /repl/folder/old/archive
, this would not be migrated as it conflicts with the path mapping rule.
#
Next stepsOnce you have defined any path mappings, you are ready to migrate your data.
#
TroubleshootingIf you are having problems creating path mappings, you can find solutions in the troubleshooting guide.