Start the flow of data by creating a data stream for your Databricks data source. The data stream creates a data lake object (DLO) in Data 360 that you can map to the Customer 360 Data Model to use across Data 360.
| User Permissions Needed | |
|---|---|
| To create a connection: | Data Cloud Architect |
Before you begin:
In Data Cloud, on the Data Streams tab, click New.
You can also use App Launcher to find and select Data Streams.
Under Other Sources, select the Databricks connection source, and click Next.
Select from the available Databricks ingest connections.
Select the Databricks tile that has a description of Ingest data from a Databricks Lakehouse to Data 360.
Select the connection, then the database, then the schema, and the object that you want to import, and click Next.
You can select one new object on the Available Objects tab.
Under Object Details, for category, identify the type of data in the data stream.
For primary key, select a unique field to identify a source record.
If a composite key is required or a key doesn’t exist, you can create one using a formula field. See Create a Formula Field or Use Case 2: Create a Primary Key.
(Optional) Select a record modified field.
If data is received out of order, the record modified field provides a reference point to determine whether to update the record. The record with the most up-to-date timestamp is loaded.
(Optional) To identify the data lineage of a record’s business unit, add the organization unit identifier.
Click Next.
For Data Space, if the default data space isn’t selected, assign the data stream to the appropriate data space.
Click Deploy.
When the Last Run Status displays success, you can see how many records were processed and the total number of records that were loaded.
You can now map the created DLO to the data model to use the data in segments, calculated insights, and other use cases.
j
