Set Up a Databricks Data Federation Connection

Set up a connection between AWS or Azure hosted Databricks and Data 360 to access data.

User Permissions Needed 
To create a connection in Data Cloud:System Admin profile or Data Cloud Architect permission set

Before you begin:

  • Review Data 360 IP Allowlist, and update your Databricks allowlists.
  • Databricks clusters must use Databricks Runtime 13.1 or later and in shared or single-user access mode.
  • Make sure these compute and workspace requirements in Databricks are met: .
    • Workspace requirements - Query data on Databricks from Salesforce: None, Unity Catalog recommended.
    • Compute requirements:
      • Network connectivity from your Databricks Runtime cluster or SQL warehouse to Salesforce. For more information, see Networking recommendations for Lakehouse Federation.
      • Databricks clusters must use Databricks Runtime 13.1 or above and shared or single-user access mode.
      • SQL warehouses must be Pro or Serverless.
    • To set up connectivity for data federation, the account admin must have a Workspace admin role.
    • The Workspace admin must complete these setup tasks.

ClientID/ClientSecret authentication is only supported using Azure Databricks-managed Service Principals. Microsoft Entra ID-managed principals are not supported.

  1. In Data Cloud, go to Data Cloud Setup.

  2. Under Configuration, select Connectors.

  3. Click New.

  4. Under Source, select Databricks and click Next.

  5. Enter a connection name and connection API name.

  6. Select one of the three authentication methods.

    • Select Username & Password. Contact your Databricks administrator to obtain the Databricks username and password.

    • Select Client ID & Client Secret. Contact your Databricks administrator to obtain the client ID and secret.

    • Select Identity Provider Based and complete the setup in the next step.

  7. To use identity provider–based authentication, click Identity Provider Based and execute the steps shown in Set Up Identity Provider Based Authentication

  8. In the Connection URL field, enter the server hostname of the SQL warehouse along with the port number. For example, adb-8903155206260665.5.azuredatabricks.net:443.

  9. In the Http path field, enter the HTTP path value of the SQL warehouse in Databricks.

  10. To review your configuration, click Test Connection.

  11. Click Save.