Migrate your data

Once you have installed LiveData Migrator for Azure and created a target, you're ready to migrate data.

info

Use exclusions to prevent certain file sizes or file/directory names (defined using GLOB patterns) from being migrated to your target storage. You can also exclude files and directories by their last modified date.

  1. In the Azure Portal, navigate to the LiveData Migrator resource page.
  2. From the LiveData Migrator menu on the left, select Migrations.
  3. Select Add.
  4. Complete the Basic details:
    • Target storage: your previously defined ADLS Gen2 target storage.
    • Path: the directory on your source filesystem to migrate data from.
  5. Select Next: Migration settings.
  6. Complete the Migration settings:
    • Target Action Policy: forcibly migrate everything, overwriting even existing files of the same file size as the source files ("Overwrite") or only overwrite content that differs in filesize ("Skip if size match").
    • Automatically Start Migration: start the migration right away, or start it manually later.
    • Enable Live Migration: create a live migration, which will continue to migrate data from the source when it receives changes ('Yes'), or a one-time migration, which will only migrate all data once ('No').
    • Exclusions: select the exclusions that you want to include for this migration.
  7. Select Review and create.
  8. Select Create.

View the migration#

  1. In the Azure Portal, navigate to the LiveData Migrator resource page.
  2. From the LiveData Migrator menu on the left, select Migrations.
  3. Select the migration to view it.

Start and stop migrations#

  1. In the Azure Portal, navigate to the LiveData Migrator resource page.
  2. From the LiveData Migrator menu on the left, select Migrations.
  3. Select the migration and select Start or Stop as required.

Data migration states#

Migrations can be in one of six states:

  • SCHEDULED: A scheduled migration is a migration currently starting.
  • NONSCHEDULED: A non-scheduled migration is a migration that has been created but isn't going to start automatically. Create a migration in this state by excluding the --auto-start parameter on creation.
  • RUNNING: A running migration is scanning through source content and transferring content to the target, as well as responding to change notifications from the source if applicable.
  • LIVE: A live migration has completed its initial scan through source content. It continues to respond to change notifications from the source, and will transfer content to and make changes in the target as required.
  • COMPLETED: A completed migration is a one-time migration section that has finished transferring all applicable data from the source filesystem to the target, and won't migrate any further data changes.
  • PAUSED: A paused migration has been instructed to stop transfer, and is not transferring content or making other changes to the target. This state is applied by stopping migrations.

Create a one-time migration#

One-time migrations do not track activity in your data during a migration. Create a one-time migration if you do not want LiveData Migrator for Azure to scan your source filesystem for updates to the data and update the files in your target filesystem as the migration progresses.

A one-time migration does not write a marker file to the source storage. Once the scan of the source completes (to determine which files and directories are to be migrated), the migration will enter a COMPLETED data migration state and won't perform any further scans.

During creation of a migration, change the value of the Enable Live Migration option from the default 'Yes' to 'No'.

Your created one-time migration will display the parameter Live Migration: No in the migration overview.

Next steps#

If you want to migrate your metadata (for example: Hive databases and tables), create a metadata migration target next.

Troubleshooting#

If you are having problems migrating data, you can find solutions in the troubleshooting guide.