Create an Amazon S3 Data Stream

Create a data stream to programmatically stream data from an Amazon S3 data source. With the data stream you can ingest data into your Data Cloud data lake.

Get familiar with Using Metadata API, and then complete these steps to create your data stream.

1. Define metadata in your local client application.

   This diagram describes how each object relates to other objects.

   

2. Create a zip file with package.xml. For more information, see Deploying and Retrieving Metadata with the Zip File.

3. Use the deploy() call to move your metadata package from your local file system to Data Cloud.

4. Note the data stream definition ID so that you can use it in your POST request.

   After you create the POST request or include the credentials in the Data Connector S3 XML metadata, update the Amazon S3 credentials of the data stream in Data Cloud.

5. Create a POST request for the User Interface API to create the data stream. When a new data stream is created, a data lake object gets created to store the data ingested from the data stream.

• Post Data from an Amazon S3 Data Source

  Use the POST method to stream data from an Amazon S3 data source.

See Also

• Salesforce Help: Create a Cloud Storage Data Stream
• DataStreamDefinition
• DataSource
• MktDataTranObject
• ExternalDataConnector
• ObjectSourceTargetMap
• DataConnectorS3
• Get Started with User Interface API