BulkTK – Calling the Force.com Bulk API from Visualforce Pages

Building on the Force.com JavaScript REST Toolkit (ForceTK), 'BulkTK', a Force.com Bulk API JavaScript toolkit, allows you to manipulate thousands of records at a time from JavaScript in the browser. Learn about its creation and try it out!

Nearly four years ago, I first wrote about ForceTK, the JavaScript toolkit for calling the Force.com REST API from Visualforce pages, external web sites and hybrid mobile apps. Since then, ForceTK has become part of every Force.com client-side web developer’s toolbox.

Although the Force.com REST API makes accessing data in Salesforce very straightforward, it is limited to inserting, updating and deleting a single record at a time. The SOAP API is able to manipulate larger sets of records, but is more complex to work with. The Bulk API, on the other hand is based on REST principles and is optimized for loading or deleting large sets of data. You can use it to query, insert, update, upsert, or delete thousands of records asynchronously by submitting batches which are processed in the background by Salesforce.

Although the Bulk API is indeed ‘based on REST principles’, it is very different from the REST API itself. Rather than operating on records in Salesforce, clients create asynchronous jobs, upload records in batches, and periodically check for job status. Although data can be uploaded in CSV or XML format, clients exchange XML messages in the ‘control plane’.

A recent issue filed against Force.com prompted me to investigate whether it would be possible to call the Bulk API from JavaScript, uploading files with thousands of records to insert, update or delete, or executing bulk queries.

With the help of a public domain JavaScript XML Object Notation (JXON) implementation in the Ratotask project, it took just a few hours to implement a first cut of ‘BulkTK’, a Force.com Bulk API JavaScript toolkit.

BulkTK

BulkTK extends ForceTK by adding methods to the ForceTK client object. After initializing a ForceTK client with a session ID, you can create a Bulk API job,

client.createJob({
    operation : 'insert',
    object : 'Contact',
    contentType : 'CSV'
}, function(response) {
    jobId = response.jobInfo.id;
    console.log('Job created with id '+jobId+'\n');
});

add one or more batches of records,

var csvData = "FirstName,LastName,Department,Birthdate,Description\n"+
              "Tom,Jones,Marketing,1940-06-07Z,"Self-described as ""the top"" branding guru on the West Coast\n"+
              "Ian,Dury,R&D,,"World-renowned expert in fuzzy logic design. Influential in technology purchases."\n";
client.addBatch(jobId, "text/csv; charset=UTF-8", csvData, function(response){
    console.log('Batch state: ',response.batchInfo.state+'\n');
});

close the job,

client.closeJob(jobId, function(response){
    console.log('Job closed. State: '+response.jobInfo.state+'\n');
});

check batch status,

client.getBatchDetails(jobId, batchId, function(response){
    console.log('Batch state: '+response.batchInfo.state+'\n');
});

and retrieve results,

client.getBatchResult(jobId, batchId, false, function(response){
    console.log('Batch result: '+response);
});

which, for a CSV job, are in the form

"Id","Success","Created","Error"
"003E000001DJanlIAD","true","true",""
"003E000001DJanmIAD","true","true",""

A sample Visualforce page demonstrates bulk query, insert, update, upsert and delete for both CSV and XML data.

As of this writing, BulkTK should be considered ‘early Alpha’ quality, but brave (‘foolhardy?’) developers are invited to take it for a spin (in a Sandbox or Developer Edition, please!) and report back their findings. Leave a comment here, file an issue or, best of all, fork ForceTK and submit a pull request!

Leave your comments...

BulkTK – Calling the Force.com Bulk API from Visualforce Pages