Skip to main content
Version: 2.3 (latest)

Configure an ADLS Gen2 target

You can migrate data to an Azure Data Lake Storage (ADLS) Gen2 filesystem by configuring it as a target filesystem for Data Migrator.

You can authenticate to an ADLS Gen2 filesystem with OAuth 2.0 or a shared key. Select an option below to get started:

note

You can find more info on maximum ingress rates, capacity, request rates, as well as, scalability and performance targets for Azure Storage here.

info

The default configuration for ADLS gen2 storage allows a maximum file size of 400GB, see the following knowledge base article for steps and guidance to increase the maximum file size.

Configure an ADLS Gen2 target with OAuth 2.0

Prerequisites

You need the following:

  • A service principal with either the Storage Blob Data Owner role assigned to the ADLS Gen2 storage account, or an access control list with RWX permissions for the parent of a migration path. The required Access Control list is RWX for the parent of a migration path. If you have many migration paths, the ACLs will need to be at the parents of each path, or on one common parent. For more information, see the Microsoft documentation.
  • Your OAuth 2.0 credentials. See more information on credentials below.

Configure an ADLS Gen2 target filesystem with OAuth 2.0 in the UI

  1. From the Dashboard, select an instance under Instances.

  2. In the Filesystems & Agents menu, select Filesystems.

  3. Select Add target filesystem.

  4. Enter the following details:

    • Filesystem Type - The type of filesystem target. Select Azure Data Lake Storage (ADLS) Gen2.
    • Display Name - Enter a name for your target filesystem.
    • Data Lake Storage Endpoint - The storage endpoint to connect to. You can override the default value (dfs.core.windows.net) by replacing it with a custom or private endpoint.
    • Authentication Type - Select Service Principal (OAuth2).
    • Account Name - The name of your ADLS Gen2 storage account.
    • Container Name - The name of the container in your storage account that you want to migrate data to.
    • Client ID - The client ID (also known as application ID) for your Azure service principal.
    • Secret - The client secret (also known as application secret) for the Azure service principal.
    • OAuth2 Endpoint - The client endpoint for the Azure service principal. Use the format https://login.microsoftonline.com/<tenant>/oauth2/v2.0/token where <tenant> is the directory ID for the Azure service principal.
    • Use Secure Protocol - When enabled, Data Migrator will use TLS to connect to the Azure Data Lake Storage. Enabled by default.
  5. Select Save. You can now use your ADLS Gen2 target in data migrations.

Next steps

If you haven't already, configure a source filesystem from which to migrate data. Then, you can create a migration to migrate data to your new ADLS Gen2 target.