Bring Your Google BigQuery Data Lake to Data Cloud: Part 2, Data Out

In Part 1 of our two-part series on connecting Google BigQuery with Salesforce Data Cloud, we explained how to share data from BigQuery to Data Cloud (Data In). In this post, we’ll show you how to share data from Data Cloud to Google BigQuery (Data Out). We’ll use Data Cloud’s Zero Copy Integration capabilities for seamless, bi-directional data sharing that does not rely on an exact, transform, and load (ETL) process.

A diagram showing the ingress/egress patterns available with the BYOL functionality with Data Cloud

What is Google BigQuery?

Google BigQuery is a fully managed, serverless data warehouse that enables scalable analysis across petabytes of data. It is a part of the Google Cloud Platform, providing a powerful SQL engine to run fast queries against large datasets. Developers use BigQuery to perform big data analytics, harnessing Google’s infrastructure to analyze data in real-time. They can also integrate BigQuery with other Google Cloud services or third-party tools to ingest, store, and visualize data.

Data Sharing Using Zero Copy data share

As part of the Zero Copy data sharing capability, we introduced the concept of “data sharing,” which allows users to assemble a set of data objects from Salesforce Data Cloud and link them with Google BigQuery as the data sharing target. After linking the data share objects with their respective targets, a connection is forged between Salesforce and BigQuery via Analytics Hub. This allows the objects to be natively presented as tables or views within the BigQuery console.

After establishing data sharing in Salesforce Data Cloud and connecting it to Google BigQuery, the integration enables almost instantaneous access to the objects, ensuring that BigQuery displays the most current data. Specifically, if the data is in Salesforce Data Cloud, it can reside there because now Salesforce Data Cloud can serve as the host of a supplemental and rich unified dataset for analysis to be done in BigQuery.

Using Data Cloud’s no-code and low-code tools, you can create a data share target, link the data share objects with Google BigQuery to generate BI visualizations, build ML models using the Google Vertex AI platform, and consume the data for downstream processing.

Diagram showing how Bring Your Own Lake works with Data Cloud

Create the Data Share Target

First, a Data Share Target will need to be created. A Data Share Target can be created by navigating to the Data Share Target tab in the Data Cloud application, then clicking “New.” Next, select “Google BigQuery.”

Image of New Data Share Target Creation screen with Google BigQuery selected

Next, enter a name for Data Share Target in the label field. The API name will be automatically populated. Next, enter the principal that you want to use to make the connection. The principal entered should be in the form of an email address. Make sure that the principal has BigQuery permissions.

Image of New Data Share Target screen with Label, API Name, Google Cloud Principal, and Type fields

The page will redirect to a sign-in page, like the one shown below. Choose the account you want to use.

Image of a screen asking to Choose an account

The page will redirect to another page. Click continue.

Image of a Sign-in screen with cancel and continue buttons

The connection between Salesforce Data Cloud and Google BigQuery is successful when the Data Share Target status is “Active”, and the Authentication status is “Successful.”

Image of Google BigQuery Integration Data Share Target with Data Share Target Name, Status, Created By, Created Date, Connection Type, Google Cloud Principal, Type, and Authentication Status fields

Create the Data Share

Navigate to the “Data Shares” tab and click “New”. Enter a name for the Data Share in the “Label” field. Select a Data Space to associate the Data Share to. Enter an optional description.

Image of New Data Share Creation page with fields for Label, Name, Data Space, and Description.

Select the objects that you want to share. There is the option to share either Data Lake Objects, Data Model Objects, or Calculated Insights. We’ll start by sharing three data model objects within the Profile category: Unified Individual, Unified Contact Point Phone, and Unified Contact Point Email. Click save.

Image showing a new data share table with Data Model Objects as the selected table and rows of data model object names

Click Link/Unlink Data Share Target in the top right corner to link the Data Share Target that was created earlier.

Image of Google BigQuery Share Data Share with a red box around Link/Unlink Data Share Target.

Choose the Data Share Target to link and click save. This is where the power of Salesforce Data Cloud will be evident. Once the data share object is linked with the Google BigQuery user or a Group, the objects within that category will appear with minimal latency in BigQuery.

Image of Link or Unlink Data Share Target table with a list of the available Data Share Targets

View objects in BigQuery

In BigQuery navigate to Analytics Hub.

Image of Google BigQuery home page

In Analytics Hub click Search Listings.

Image of Analytics Hub with buttons for Create Exchange, Create Clean Room, and Search Listings

Locate the data share from Salesforce Data Cloud and click it.

Image of Salesforce Data Cloud Data Share in Analytics Hub

Click + Add Dataset to Project to add the data model objects to BigQuery.

Image of Salesforce Data Cloud Data Share with a button for +Add Dataset to Project

In the Explorer tab of BigQuery you should see your Data Share from Salesforce Data Cloud and the data model objects.

Image of BigQuery Studio Explorer tab with the data share expanded and data model objects listed

To query a data model object in BigQuery click the three vertical dots next to the data model object and select Query. This will open a new window.

Image of Explorer console with a red box around three vertical dots beside the Unified Individual Data Model Object

You can add the fields that you wish to query and click Run. In the below example we are getting ID, first name, last name, and email from the Unified Individual Data Model Object. After you click run the results will show under Query Results. You can then use this data with BigQuery’s functionality like creating data sets, tables, and downloadable files with the data.

Image of a Query and Query Results in BigQuery

Conclusion

The seamless integration of Google BigQuery with Data Cloud using the Zero Copy strategy represents a monumental shift from ETL for developers. By effectively breaking down data silos, this approach not only streamlines the aggregation and analysis of diverse data types but also significantly enhances the ability to deliver personalized customer experiences.

Be sure to read Part 1 of our series on Google BigQuery and Data Cloud, where we focus on sharing data from BigQuery to Salesforce (data in).

Explore our new partnership page to discover more about the collaboration between Salesforce and Google.

Further resources

Salesforce and Google announcement
Spring 24 Release Notes on BYOL data share
Trailhead module: Get to Know Data Cloud: Quick Look
Trailhead module: BYOL Data Shares in Data Cloud: Quick Look

About the authors

Danielle Larregui is a Senior Developer Advocate at Salesforce focusing on the Data Cloud platform. She enjoys learning about cloud technologies, speaking at and attending tech conferences, and engaging with technical communities. You can follow her on X(Twitter).

Gina Nichols is a Director on the Salesforce Data Cloud product team. She has a background as an architect working with some of Salesforce’s enterprise customers in the MarTech space. She is also a co-author of the award-winning, Transform the Consumer Experience Customer 360 Guide, which won an award of Merit at the STC Chicago competition. You can follow her on LinkedIn.