Set Up a Google Cloud Storage Connection

After you create your Data Cloud instance, you can set up a Google Cloud Storage (GCS) connection. You can create up to five GCS connections. A connection is defined as the unique combination of the bucket name and parent directory.

Before you begin:

• Set Up Permissions for the Google Cloud Storage Connector
• Review the IP addresses to make sure that the GCS connection has the necessary access

1. In Data Cloud, go to Data Cloud Setup.

2. Under Configuration, select More Connectors.

3. Click New.

4. Under Source, select Google Cloud Storage and click Next.

5. Enter a connection name and connection API name.

6. Enter the connection details.

   Field Label	Description
   Access Key	Programmatic username for API access to Google Cloud Storage bucket.
   Secret Key	Programmatic password for API access to Google Cloud Storage bucket.

7. Enter the authentication details.

   All folders under the parent directory are migrated to a staging environment where you can select the objects to import into Data Cloud. Because data access charges can apply, it’s recommended that the parent directory and its subfolders contain only essential files.

   Field Label	Description
   Bucket Name	Specific public cloud storage resource in Google Cloud Service.
   Parent Directory	Parent folder that corresponds to the GCS bucket. The root directory can’t serve as the parent directory.

8. To review your configuration, click Test Connection.

9. Click Save.

After the connection details are accepted, Data Cloud begins syncing data between your GCS instance and the Data Cloud staging environment. The sync is designed to mirror the structure of your GCS bucket, so if objects are deleted, they’re also deleted from the staging environment. Your Data Aware Specialist can then create data streams for any objects under the parent directory.

Expect some latency for the initial sync. After the Day 0 load, incremental sync tasks run hourly.