Salesforce CDP API Developer Guide

Quick Start
Connect Data
Ingest Data
Load Data into Customer Data Platform
Get Started
Walkthrough for Streaming Insert
Walkthrough for Bulk Insert
Streaming Ingestion
Bulk Ingestion
Create an Amazon S3 Data Stream
Extract Data
PDF

Walkthrough for Bulk Insert

This walkthrough guides you through the steps for loading records using the Bulk API.
Before you can start, make sure you’ve completed the prerequisites required to set up your Ingestion API:
  • Setup Ingestion API connector
  • Create and Deploy Data Stream
  • Configure connected app
  1. Create a CSV file containing data that matches the data stream you defined. For example, orders.csv. 
    1id,contact_name,created_date,tax_exempt,ship_address,total,tax_rate,modified_date
2f19cae7e-5cdb-41ce-92ba-539afea04f9d,Jadyn Kozey,2021-07-22T08:57:08.118Z,false,9818 Walter Course Apt. 811,77506,84,2021-07-05T09:31:44.457Z
348c8b6a6-8179-4a15-ba19-ec5c8206ce28,Kristina Jakubowski,2021-07-08T22:57:02.008Z,false,245 Schowalter Port Suite 245,16986,61,2021-07-03T06:20:52.886Z
  2. Request a Salesforce CDP access token. The access_token property in the token exchange response contains the bearer token to use for the authorization header. The instance_url is the Salesforce CDP instance where the API is hosted.
  3. To upload the data, create a job. In this example, we’re uploading Orders data from our ecomm Ingestion API connector. If you’re deleting records, you would use the delete operation instead. You should get a JSON response that includes the job id, with a job state of Open. You'll need the job id in the next steps.
    Request
    1curl --location --request POST 'https://{instance_url}/api/v1/ingest/jobs' \
2--header 'Authorization: Bearer {access_token}' \
3--header 'Content-Type: application/json' \
4--data-raw '{
5   "object":"Orders",
6   "sourceName":"ecomm",
7   "operation":"upsert"
8}'
    Response
    1{
2   "object": "Order",
3   "id": "913a8b00-7a9e-4331-aaf6-a019ae5021ab",
4   "operation": "upsert",
5   "sourceName": "ecomm_api",
6   "createdById": "005R0000000zqc8IAA",
7   "createdDate": "2021-08-12T17:06:42.921387Z",
8   "systemModstamp": "2021-08-12T17:06:42.921387Z",
9   "state": "Open",
10   "contentType": "CSV",
11   "apiVersion": "v1",
12   "contentUrl": "/api/v1/ingest/jobs/913a8b00-7a9e-4331-aaf6-a019ae5021ab/batches"
13}
  4. Upload your CSV data. After creating a job, you're ready to upload your data. You provide record data using the CSV file you created earlier. You can submit up to 100 CSV files as part of a job. Each file should not exceed 150 MB.
    Request
    1curl --location --request PUT 'https://{instance_url}/api/v1/ingest/jobs/913a8b00-7a9e-4331-aaf6-a019ae5021ab/batches
2' \
3--header 'Authorization: Bearer {access_token}' \
4--header 'Content-Type: text/csv' \
5--data-binary '@orders.csv'
    Response
    1201 Created
  5. Upload complete. After you're done submitting data, you can inform Salesforce CDP that the job is ready for processing by closing the job.
    Request
    1curl -v --location --request PATCH 'https://{instance_url}/api/v1/ingest/jobs/913a8b00-7a9e-4331-aaf6-a019ae5021ab' \
2--header 'Authorization: Bearer {access_token}' \
3--header 'Content-Type: application/json' \
4--data-raw '{
5"state" : "UploadComplete"
6}'
    Response
    1{
2   "object": "Order",
3   "id": "913a8b00-7a9e-4331-aaf6-a019ae5021ab",
4   "operation": "upsert",
5   "sourceName": "ecomm_api",
6   "createdById": "005R0000000zqc8IAA",
7   "createdDate": "2021-08-12T17:06:42.921387Z",
8   "systemModstamp": "2021-08-12T17:36:42.921387Z",
9   "state": "UploadComplete",
10   "contentType": "CSV",
11   "apiVersion": "v1",
12   "contentUrl": "/api/v1/ingest/jobs/913a8b00-7a9e-4331-aaf6-a019ae5021ab/batches"
13}
  6. Check the job status and results. To get basic status information on a job, such as the overall job state or the number of records processed, use a Get Job Info request. The job state changes from UploadComplete to either JobComplete or Failed after the files have been loaded into the data lake.
    Request
    1curl -v --location --request GET 'https://{instance_url}/api/v1/ingest/jobs/913a8b00-7a9e-4331-aaf6-a019ae5021ab' \
2--header 'Authorization: Bearer {access_token}' \
    Response
    1{
2   "object": "Order",
3   "id": "913a8b00-7a9e-4331-aaf6-a019ae5021ab",
4   "operation": "upsert",
5   "sourceName": "GreasyConnector",
6   "createdById": "005R0000000zqc8IAA",
7   "createdDate": "2021-08-12T17:06:42.921387Z",
8   "systemModstamp": "2021-08-12T18:06:42.921387Z",
9   "state": "JobComplete",
10   "contentType": "CSV",
11   "apiVersion": "v1",
12   "contentUrl": "/api/v1/ingest/jobs/913a8b00-7a9e-4331-aaf6-a019ae5021ab/batches",
13   "retries": 0,
14   "totalProcessingTime": 0
15}