Version: 1.18.1 (latest)

Manage migrations

Manage your migrations with LiveData Migrator using either the UI or CLI.

caution

Don't write to target filesystem paths when a migration is underway. This could interfere with LiveData Migrator functionality and lead to undetermined behavior.

Use different filesystem paths when writing to the target filesystem directly (and not through LiveData Migrator).

Manage migrations with the UI#

The Dashboard displays an overview of migrations and their status, showing what pre-existing data has been moved and data added since the migration started.

Select View migration to see more detail.

note

You must stop a migration before you can delete it.

Migration metrics#

Use metrics to view details about your migrations to better understand progress, including their bandwidth usage, migration events, and remaining scanned paths.

In the Dashboard, select a migration.
You will be taken to the Status of the migration tab by default.

Under Migration Metrics, observe the following:

Events yet to be processed: this graph displays the size of the event manager queue across a time period selected in the Filter by dropdown list.
Events yet to be migrated: this graph shows the size of the scheduled actions queue across a time period selected in the Filter by dropdown list.
Paths to be scanned: this graph displays the size of the queue for paths yet to be scanned across a time period selected in the Filter by dropdown list.

Assign exclusions to an existing migration#

Adding exclusions to an existing migration will change the future actions performed for that migration, but will not affect previously migrated content.

In the Dashboard, select a migration.
Under the sidebar that appears, select Exclusions.
Select Add and the appropriate exclusion template from the dropdown list.

Remove exclusions from an existing migration#

In the Dashboard, select a migration.
Under the sidebar that appears, select Exclusions.
Remove any of the exclusions one at a time.

Start a migration#

To start a migration and initiate data transfer, select Start.

Stop a migration#

When you start a migration, the Start button will be replaced with a Stop button. Select it to stop the migration and suspend all current file transfer.

Resume a migration#

When you stop a migration, the Stop button will be replaced with a Resume button. Select it to resume the migration, which will continue from where it left off.

Delete a migration#

note

Migrations must be stopped before they can be deleted.

When viewing a migration, select on the Details panel. Select the option to Delete the migration.

Reset a migration#

Reset a migration to set it to the state it was in before it started. This deletes the migration and creates a new migration in its place with the same settings as the previous.

When you stop a migration, a Reset button becomes available next to the Resume button. Select it to open the Reset Migration panel. Configure the following:

Reset Path Mappings: Select this box to erase the migration's path mapping configuration and automatically use your most recent path mapping configuration for LiveData Migrator. Clear it to continue using the same path mapping configuration from before the reset.
Skip or Overwrite Settings: Select Overwrite if you want the migration to re-migrate any and all files that had already been migrated before the reset, regardless of whether the file sizes are the same at the source and target filesystems. Select Skip If Size Match to skip migrating any files that already exist in the target and have the same file size as the source.

Select Reset to confirm your selection and reset the migration.

Pending regions#

LiveData Migrator uses pending regions to keep your directories up to date if they change during data migrations. If directories are updated while being migrated, the changed paths on the source filesystem are tracked by the migration so they can be re-scanned for updates when the migration finishes.

LiveData Migrator collects pending regions automatically during a migration, but you can manually add them if you want the directories to be re-scanned after further updates. You can also re-run the entire migration by making the root directory the pending region.

Add pending regions in the UI#

To add a pending region in the UI:

Select the migration you want to add a pending region to in the dashboard.
Under the sidebar that appears, select Rescan.
Type the path to the directory in Add rescan directories.
Choose which action policy to use (Overwrite or Skip If Size Match).
Select +Add.

Add pending regions in the CLI#

Add a pending region to a migration by running the migration pending-region add command.

Example

migration pending-region add --migration-id myFirstMigration --path /dir1/userA --action-policy com.wandisco.livemigrator2.migration.SkipIfSizeMatchActionPolicy

Data migration states#

Migrations can be in one of the following states:

NONSCHEDULED : A non-scheduled migration has been defined but not yet started. Create a migration in this state by not specifying the --auto-start parameter on creation.

RUNNING : A running migration is scanning through source data and transferring data to the target, as well as responding to change notifications from the source, if applicable.

LIVE : A live migration has completed scanning through source data, continues to respond to change notifications from the source, and transfers data and makes changes to the target as required.

COMPLETED : A completed migration has scanned through all source data and finished transferring all applicable data to the target filesystem, and doesn't migrate changes to source data. See One time migrations.

Monitor failed operations#

The Failed Operations panel on the Migration page in the UI provides a counter of any failed operations, including paths that failed to migrate. For a breakdown of the latest failures, select View details. This opens a list of failures that includes the following information for each failure:

Date and time: Date and time of the failure.
Example
```
2020-01-07 13:30:40 AM UTC
```
Path: Path that failed to migrate.
Example
```
/path/to/migration/files/000000_1
```

Failure: Error message generated by the failure.

Example

java.io.IOException: Error rescanning [/path/to/migration/files/part-1234] [not found]

Check your migration#

Check the status of a live migration by going to the migration's screen and selecting Migration Check from the Migration menu.

The Migration Check panel provides a Migration Check Report that lists failed migration operations. The report table lists each failed operation with columns for the Timestamp, Operation type, and Path and Failure, which provides the path to the failed element, along with a description of the failure.

Filter and search for failed operations#

You can filter or search for failed operations by using the Operation type dropdown on the Migration Check Report panel. The filter contains the following operations:

Operation	Description
Create	Create operations set up a new data migration. For more information, see Create a migration.
Migration (Transfer file)	Migration operations handle the transfer of data to be migrated.
Unlink (Delete path)	Unlink operations remove files under migration.
Rescan (retry)	Rescan operations restart an earlier source filesystem scan.
Rename	Rename operation handles name changes in files or directories under replication.
Metadata (Change permissions)	Metadata operations change file metadata, such as file ownership.
Marker (Internal to LiveData Migrator)	Marker operations are part of LiveData Migrator's internal migration mechanics. Marker file operation failures may be evidence of a serious problem with your LiveData Migrator deployment. Contact WANdisco Support with details of the errors.

Advanced search#

Select Advanced search ^ to display Filter Rules for additional search options. Use the following entry fields to create an advanced search rule:

Field: Select Path to search all available migration paths or Failure to search only failed migration paths.
Operator: Select Contains or Does not contain to apply to your search.
Enter Value: Enter the value you want your search to match, according to your chosen operator. If you select the "Does not contain" operator, your search will match with paths or failure paths that don't include the value.

Select + Add Rule to add additional search criteria. Select Apply to view the search results. You can continue to add rules to your current search, but you must select Apply to update the search results.

note

Historical failure information may be garbage collected, so it may not be possible to show all failures. In this case, a warning appears:

"We're unable to show detailed results of failed operations. We only have detailed results from the most recent [number of available failures] results"

Manage migrations with the CLI#

Command	Action
`migration stop`	Stop a migration
`migration reset`	Reset a migration
`migration resume`	Resume a stopped migration
`migration delete`	Delete a migration
`migration exclusion add`	Add an exclusion to a migration
`migration exclusion delete`	Remove an exclusion from a migration
`migration list`	List running and active migrations
`migration start`	Start a migration
`migration show`	Get migration details
`migration pending-region add`	Add a pending region to a migration
`status`	Get migration status