Set Up a Databricks Connection

Set up the AWS or Azure hosted Databricks connection to start the batch ingestion of data into Data 360.

User Permissions Needed 
To create a connection:System Admin profile or Data Cloud Architect permission set

Before you begin:

  • Verify your admin has enabled firewalls on the system you want Data 360 to connect to by including these IP addresses to your allowlists.

ClientID/ClientSecret authentication is only supported using Azure Databricks-managed Service Principals. Microsoft Entra ID-managed principals are not supported.

  1. In Data Cloud, click Setup, and select Data Cloud Setup.

  2. Under External Integrations, select Other Connectors.

  3. Click New.

  4. On the Source tab, select Databricks and click Next.

  5. Enter a connection name, connection API name, and provide the authentication and connection details.

  6. Choose from the two Authentication Methods.

    • Select Username & Password. Contact your Databricks admin to obtain the Databricks username and password.

    • Select Client ID & Client Secret. Contact your Databricks admin to obtain the Databricks client ID and client secret.

  7. Enter the Connection URL.

    This is the server hostname of the SQL warehouse along with the port number. For example: adb-8903155206260665.5.azuredatabricks.net:443.

  8. Enter the HTTP Path.

    This is the HTTP path value of the SQL warehouse in Databricks.

  9. To review your configuration, click Test Connection.

  10. Click Save.

After the connector details are accepted, the connection is created and listed under Connectors. You can now create data streams.