Add a Self-Managed Private Network Route for Databricks on Azure

Connect your Databricks data source and Data 360 tenant with a dedicated network connection using Private Connect for Data 360. Create a single connection to a specific instance regardless of how many Data 360 instances you have. Before you set up a Databricks federation connection or data share, establish a network route.

Editions

Available in: All Editions supported by Data 360. See Data 360 edition availability.

Permissions

User Permissions Needed
To create a Databricks network route:	Data 360 Architect permission set and Manage External Connections or Customize Application permission.
To administer a Databricks environment	Databricks account admin

Prerequisites

You’ve created the Virtual Private Cloud (VPC) endpoint service name for the Java Database Connectivity (JDBC) and Azure Blob endpoints. For more information, see the Implementation Guide.

You’ve configured the Data 360 Principal ID for this PNR as an allowed principal in AWS. For more information, see the Implementation Guide.

Limitations

The number of Private Network Routes (PNRs) for a particular Databricks URL is limited to one per Data 360 functional domain (not per Data 360 org).

This topic covers only the Salesforce steps you need to complete. To complete the end-to-end process across all sources, see the Private Connect Implementation Reference Guide for Databricks on Azure.

Step 1: Create a Databricks Private Connection

Set up a Databricks private connection with the Amazon Web Services (AWS) VPC endpoint services in Data 360.

From the App Launcher, select Data 360.
From Setup, select Data Cloud Setup.
From Data Cloud Setup, under External Integrations, select Private Connect.
Select New. If the button doesn’t appear, the maximum number of licensed PNRs was exceeded.
On the Add Private Network Route page, select the Databricks tile.
In the Select a Network panel, select Azure, and then Add Private Network Route.
Go to the Configure Databricks Private Network Route page.
Go to Step 2: Account and complete these steps:
1. In the Databricks Account URL field, enter the URL for your Databricks account.
2. In the VPC Endpoint Service Name field, enter the AWS VPC Endpoint that you created earlier. For example: com.amazonaws.vpce.<region>.<vpc-endpoint-service-name>.
Go to Step 3: Internal Stage and complete these steps:
1. In the Azure Blob URL field, add your Azure account URL. For example: https://.blob.core.windows.net//<storage-account-name>.blob.core.windows.net/<contatiner-name>/<blob-name>.
2. In the VPC Endpoint Service Name field, enter the AWS VPC Endpoint. For example: com.amazonaws.vpce.<region>.<vpc-endpoint-service-name>
Save your changes and keep this browser tab open. Provisioning your network route takes several minutes. The next step is to confirm your connection request in Amazon Virtual Private Cloud.

Step 2: Confirm Connection Request and Verify Setup

Confirm your Private Connection is working.

Prerequisites: Complete Step 1.

Access the Amazon VPC console.
Go to the endpoint service for Databricks and the Databricks internal stage.
On the Endpoint Connections tab, go to your pending connection request for Databricks and Databricks internal stage. Then complete these steps:
1. Check the box next to the Databricks endpoint.
2. Select the Actions dropdown.
3. Select Accept Endpoint Connection Request. Repeat these steps for Databricks Internal Stage.
Go back to Data Cloud Setup.
Refresh the configuration by selecting the Data Cloud Setup Refresh button. Refreshing the browser page has no effect.
To verify the setup, go to Data Cloud Setup and select Private Connect. The dashboard shows the status of both PNRs, one for Databricks and one for Databricks internal stage.
Verify that both PNRs are listed as connected. You can select the individual PNR record to see its details.

Next Steps

This procedure created the Private Connect Integration. Your next task is to create the data stream for your data.

A Query Federated Data Stream - if you are connecting Data 360 to a Databricks SQL warehouse to query Databricks tables or views.

Instead of following the entire set of instructions, where appropriate, turn on Use Private Network Route and select the PNR you created using the Route Name you entered. When asked, you will need to supply the HTTP Path. See the resource table for details.

Edit the PNR

After you create the Private Network Route (PNR), you can edit its name and description.

From the App Launcher, find and select Data 360.
From Setup, in the Quick Find box, enter Data Cloud Setup, and then select Data Cloud Setup.
Expand Admin Tools, and then select Private Connect.
Find the row for the PNR that you created, and open its details page.
Click Edit.
Change the available settings and save your changes.
- PNR Name: Enter a different name.
- Description: Enter a different description.