Working with Bulk API and finding that your code is a little bulky itself? The new Bulk API v2 might be just the diet you’re looking for.
Now generally available in Winter ’18 (API version 41.0), Bulk API v2 brings the power of bulk transactions from Bulk API v1 into a simplified, easier-to-use API. Bulk API v2 lets you create, update, or delete millions of records asynchronously, just like Bulk API v1, but offers the following core improvements:
Bulk API v2 also has a number of new features that aren’t available in Bulk API v1, but more on that later.
First, to get a sense of how Bulk API v2 is going to make things easier for you, let’s compare the process of creating a set of records using Bulk API v1 versus Bulk API v2.
With Bulk API v1, you’d need to write code to do the following steps:
So how does the same process look in Bulk API v2?
Notice how much easier and more convenient the Bulk API v2 process is. Everything from initial authentication to getting failed records is easier. You no longer have to write a lot of extra error-prone code to handle tasks like assembling a list of failed records from a set of failed batches. Also notice how the basic overall process of creating and submitting asynchronous jobs is still the same — you don’t have to learn and code a whole new paradigm to upload your bulk data.
The whole process for Bulk API v2 is so simple that we can walk through the actual steps in detail in this blog post. Let’s walk through creating some Contact records.
Authenticate using OAuth
First, you need to authenticate. Using your preferred choice of OAuth flow, issue requests to https://login.salesforce.com/services/oauth2/authorize
to obtain an authentication token.
Create a job
Next, create a new Bulk API v2 job by issuing a POST request to /services/data/v41.0/jobs/ingest/
with the following request body:
{ "object" : "Contact", "contentType" : "CSV", "operation" : "insert" }
This creates a job that will insert new Contact records.
Upload the data
Issue a PUT request using the JOB ID returned from the previous request to the following URI:
/services/data/v41.0/jobs/ingest/JOB ID/batches/
The request body will be CSV data of all the records you want to upload (with the Content-Type
request header set to text/csv
).
Close the job
Issue a PATCH request again using the JOB ID, to the following URI:
/services/data/v41.0/jobs/ingest/JOB ID/
With the following request body:
{ "state" : "UploadComplete" }
This tells Salesforce we’re done uploading data for the job, and Salesforce will start inserting the records.
Check the status of the job
Issue a GET request to:
/services/data/v41.0/jobs/ingest/JOB ID/
Look for a job state of JobComplete to know Salesforce is done processing the job.
Get errors for any failed records
If the job status indicates that some records encountered errors during processing, issue a GET request to the following URI to get a full list of the failed records:
/services/data/v41.0/jobs/ingest/JOB ID/failedResults/
And that’s it, really.
Bulk API v2 goes beyond Bulk API v1 and offers some additional features to make your life easier. These include:
/services/data/vXX.X/jobs/ingest?concurrencyMode=parallel
will return a list of all jobs in your org using parallel concurrency mode for processing.Note that you can’t do Bulk queries in Bulk API v2 yet.
Bulk API v2 reduces the amount of code you have to write and gives you more options on how to process your data. Plus, it simplifies data limits, so you can spend less time worrying about how much data you can work with, and spend more time actually running your integration jobs. Consider taking the time to switch over to using Bulk API v2 if you’re using v1, and your code will be slim and trim in no time!
For more information on Bulk API v2, see:
Bulk API 2.0 Developer Guide
Bulk API unit of the API Basics Trailhead module