Set Up a Databricks File Federation Connection (Beta)

Set up a Databricks File Federation connection between AWS or Azure hosted Databricks and Data Cloud to access data.

Before you begin:

• Make sure that Uniform V2 is enabled on every delta table. Unity Catalog is required instead of Hive metastore.
• Get familiar with Unity Catalog. The underlying cloud storage platforms supported are AWS and Azure.
• Make sure Unity Catalog credential vending for external system access is set.
• Enable Uniform (Iceberg Reads). For more information, see Read Delta Tables with Iceberg Clients.
• Verify your admin has enabled firewalls on the system you want Data Cloud to connect to by including these IP Addresses Used by Data Cloud. Make sure any network or VPC level allowlisting is considered.
• You can’t use private network mechanisms like AWS PrivateLinks or Azure Private Link with file federation.

1. In Data Cloud, go to Data Cloud Setup.

2. Under External Integrations, select Other Connectors.

3. Click New.

4. On the Source tab, select Databricks File Federation and click Next.

5. Enter a connection name and connection API name.

6. Enter the authentication and connection details.

   1. Enter the Azure storage type. Available options include S3, Catalogue Provided, and Azure.
   2. Enter Personal Access Token. It's used to authenticate access to resources and APIs. For more information, see Personal Access Tokens
   3. Enter other relevant authentication details based on the storage type.

7. For Connection Details, enter Catalog Endpoint. Catalog Endpoint is the Unity Catalog endpoint. The catalog endpoint must be an HTTPS endpoint.

8. To review configuration, click Test Connection.

9. Click Save.