Migrate your data
Once you have installed LiveData Migrator for Azure and created a target, you're ready to migrate data.
info
Use exclusions to prevent certain file sizes or file/directory names (defined using GLOB patterns) from being migrated to your target storage. You can also exclude files and directories by their last modified date.
- Migrate data with the Azure Portal
- Migrate data with the CLI
- In the Azure Portal, navigate to the LiveData Migrator resource page.
- From the LiveData Migrator menu on the left, select Migrations.
- Select Add.
- Complete the Basic details:
- Target storage: your previously defined ADLS Gen2 target storage.
- Path: the directory on your source filesystem to migrate data from.
- Select Next: Migration settings.
- Complete the Migration settings:
- Target Action Policy: forcibly migrate everything, overwriting even existing files of the same file size as the source files ("Overwrite") or only overwrite content that differs in filesize ("Skip if size match").
- Automatically Start Migration: start the migration right away, or start it manually later.
- Enable Live Migration: create a live migration, which will continue to migrate data from the source when it receives changes ('Yes'), or a one-time migration, which will only migrate all data once ('No').
- Exclusions: select the exclusions that you want to include for this migration.
- Select Review and create.
- Select Create.
#
View the migration- In the Azure Portal, navigate to the LiveData Migrator resource page.
- From the LiveData Migrator menu on the left, select Migrations.
- Select the migration to view it.
#
Start and stop migrations- In the Azure Portal, navigate to the LiveData Migrator resource page.
- From the LiveData Migrator menu on the left, select Migrations.
- Select the migration and select Start or Stop as required.
#
Create a migrationThe following command will create a migration using the target you created. You can have multiple migrations per migrator. During the creation of a migration you can start a migration right away, or start it manually later.
Defining exclusions (--exclusions
) are optional when creating a migration.
Run the following command to create a migration to start right away:
Run the following command to create a migration but start it later. Use the start migration command to start the migration.
#
View a migrationIf you want to see how a migration is progressing, you can check its status by running the following command. See the migrationStatus property in the response for the status of the migration.
#
Start a migrationRun the following command to start a migration that has been created but not started automatically. You can also use this command on a stopped migration.
#
Stop a migrationRun the following command to stop a migration.
Resume a migration again by starting it.
#
Add exclusions to an existing migrationRun the following command to add an exclusion or multiple exclusions to an existing migration:
Adding exclusions to an existing migration will apply immediately to the migration, but will not affect previously migrated content.
#
Remove exclusions from an existing migrationRun the following command to remove an exclusion or multiple exclusions from an existing migration:
Removing exclusions from an existing migration will apply immediately to the migration, but will not affect previously migrated content.
#
Data migration statesMigrations can be in one of six states:
SCHEDULED
: A scheduled migration is a migration currently starting.NONSCHEDULED
: A non-scheduled migration is a migration that has been created but isn't going to start automatically. Create a migration in this state by excluding the--auto-start
parameter on creation.RUNNING
: A running migration is scanning through source content and transferring content to the target, as well as responding to change notifications from the source if applicable.LIVE
: A live migration has completed its initial scan through source content. It continues to respond to change notifications from the source, and will transfer content to and make changes in the target as required.COMPLETED
: A completed migration is a one-time migration section that has finished transferring all applicable data from the source filesystem to the target, and won't migrate any further data changes.PAUSED
: A paused migration has been instructed to stop transfer, and is not transferring content or making other changes to the target. This state is applied by stopping migrations.
#
Create a one-time migrationOne-time migrations do not track activity in your data during a migration. Create a one-time migration if you do not want LiveData Migrator for Azure to scan your source filesystem for updates to the data and update the files in your target filesystem as the migration progresses.
A one-time migration does not write a marker file to the source storage. Once the scan of the source completes (to determine which files and directories are to be migrated), the migration will enter a COMPLETED
data migration state and won't perform any further scans.
- Create a one-time migration with the Azure Portal
- Create a one-time migration with the CLI
During creation of a migration, change the value of the Enable Live Migration option from the default 'Yes' to 'No'.
Your created one-time migration will display the parameter Live Migration: No
in the migration overview.
Specify the scan-only
flag when you create the migration:
#
Next stepsIf you want to migrate your metadata (for example: Hive databases and tables), create a metadata migration target next.
#
TroubleshootingIf you are having problems migrating data, you can find solutions in the troubleshooting guide.