Release notes
| Product Version | LiveData UI | LiveData Migrator | Hive Migrator | CLI |
|---|---|---|---|---|
| 3.4.0 | 14.15.2 | 3.4.0 | 2.11.11 | 3.4.2 |
Release Highlights
Live Ceph
The Ceph filesystem is built on top of Ceph's distributed object store. It is designed to provide high performance, scalability, fault tolerance, and flexibility. Common use cases include cloud infrastructure, data lakes and enterprise storage for AI/ML.
With this release, we extend support for Ceph filesystems, enabling seamless data migration to or between Ceph clusters. Real-time replication ensures your target environment stays in sync, even as users actively interact with the data. This release introduces the ability to use Keycloak authentication.
Take a look at Configure a Ceph source for more information.
HVM Metadata Verification - Phase 3
In this release we have improved metadata verifications by adding support for environments where the source and target have different Hive versions, such as migrations from Hive 1 or Hive 2 to Hive 3.
This release also introduces the ability to run multiple verifications in parallel, allowing you to trigger jobs across different migrations simultaneously rather than waiting for them to run sequentially.
Furthermore, we have extended these capabilities to include the verification of Hive Views to ensure consistency between your source and target. Additionally, we have enhanced mismatch reporting to explicitly identify the specific discrepancy, displaying the conflicting values from both the source and target environments.
For more information on metadata verification see here.
HVM Metadata Repairs - Phase 2 [Preview]
The Metadata Verification Repair capability, currently in preview, has been updated with several enhancements. Users can now repair Hive Views and have the flexibility to target specific objects for repair rather than processing an entire verification. To streamline the workflow, we have introduced repair queuing, which allows multiple repairs to be triggered and executed sequentially as previous tasks complete.
This release also includes UI support for repairs and integrated notifications to provide status updates on repair tasks. These features are intended for evaluation and testing in non-production environments while we continue to expand supported repair scenarios.
For more information on how to enable and use see here.
CDP 7.3.1 Support - Phase 2
We have expanded our support for Cloudera Data Platform 7.3.1 to include the migration of single file tables. This release allows for the seamless migration of single file tables between CDP 7.3.1 sources and targets. This builds upon our existing CDP 7.3.1 support, which already enables full metadata and data migrations between clusters running this version.
Other Improvements
CLI & UI Enhancements
The UI enhancements in this release are focused on boosting performance and we also introduce the ability to reset recurring migrations. There have been several improvements to the CLI too, one of which involves making it easier for a user to obtain the last completed verification report for a data migration.
Resolved Issues
Data Migrator Core
LM2-9097 Ceph: Swapping from an SSL keystore with a password to a pem one without causes issue
Hive Migrator
HVM-5867 - HVM migrations can continue dropping objects after they have been stopped
HVM-5849 - Failed to migrate object
HVM-5842 - OwnerName can cause discrepancies between Hive 1 (HDP 2.6.5) and Hive 3 (CDP 7.3.1)
HVM-5840 - Indexes present in older Hive versions cause migration errors in HVM
HVM-5839 - OwnerType causes discrepancies in Verifications between Hive 1 and Hive 2
HVM-5838 - VerificationObject needs toString() for logging
HVM-5822 - HVM table already exists error in the logs can cause spam
HVM-5818 - Logging in Hive agent never reaches the log file
HVM-5812 - Drop partition race condition leads to NPE
HVM-5788 - HVM Trace logging logs wrong object in fetchDatabases
HVM-5787 - Datalake Agent return Agent type and version
UI
ONEUI-8599 Correction to the display of the state of metadata migrations
ONEUI-8632 Make H2 upgrade compatible with later Java versions
CLI
LDMC-744 Migration reset preserves target match restrictions for filesystems
Planned for Retirement
Please note that the Data Transfer Agents feature is planned for retirement in the next release. Therefore you should not use this going forward.