Set Up a Google Cloud Storage Connection

After you create your Data Cloud instance, you can set up a Google Cloud Storage (GCS) connection. You can create up to five GCS connections. A connection is defined as the unique combination of the bucket name and parent directory.

User Permissions Needed 
To create a connection:System Administrator

Before you begin:

  1. In Data Cloud, go to Data Cloud Setup.

  2. Under Configuration, select More Connectors.

  3. Click New.

  4. Under Source, select Google Cloud Storage and click Next.

  5. Enter a connection name and connection API name.

  6. Enter the connection details.

    Field LabelDescription
    Access KeyProgrammatic username for API access to Google Cloud Storage bucket.
    Secret KeyProgrammatic password for API access to Google Cloud Storage bucket.
  7. Enter the authentication details.

    All folders under the parent directory are migrated to a staging environment where you can select the objects to import into Data Cloud. Because data access charges can apply, it’s recommended that the parent directory and its subfolders contain only essential files.

    Field LabelDescription
    Bucket NameSpecific public cloud storage resource in Google Cloud Service.
    Parent DirectoryParent folder that corresponds to the GCS bucket. The root directory can’t serve as the parent directory.
  8. To review your configuration, click Test Connection.

  9. Click Save.

After the connection details are accepted, Data Cloud begins syncing data between your GCS instance and the Data Cloud staging environment. The sync is designed to mirror the structure of your GCS bucket, so if objects are deleted, they’re also deleted from the staging environment. Your Data Aware Specialist can then create data streams for any objects under the parent directory.

Expect some latency for the initial sync. After the Day 0 load, incremental sync tasks run hourly.