Data loads and integrations that perform poorly or fail altogether are among the most significant problems during salesforce.com customers’ “go-live” efforts. They can both put timelines at risk and become unimaginably expensive, especially for enterprise-scale organizations.

If you’re a Salesforce architect or developer, you can save your organization both time and money—and get up to 20 million records into your organization per hour—when you properly plan and execute your large loads and integrations using the Bulk API. On the surface, one of the most important loading and integration best practices is simple: When you run a Bulk API job, processing more of its batches in parallel means giving that job a higher degree of parallelism, which in turn gives your run better throughput.

Mastering Parallelism to Scale Your Enterprise

Fast parallel data loading with the Bulk APIBut what is a good degree of parallelism, and how can you architect a solution that maximizes the throughput that parallelism gives you? Technical Enablement’s newest article walks you through several optimal and suboptimal test runs, which feature code samples and real-life metrics, so you can master parallelism, and improve your load and integration performance in your organization.

After reading this article, you should be able to:

  • Estimate your loads’ and integrations’ degrees of parallelism
  • Design an integration or loading tool that gives you the best possible throughput
  • Manage and avoid lock exceptions, both of which can significantly reduce throughput
  • Take some concrete, actionable best practices back to your organization to scale your enterprise

Related Resources

About the Author

Sean Regan is an Architect Evangelist within the Technical Enablement team of the salesforce.com Customer-Centric Engineering group. The team’s mission is to help customers understand how to implement technically sound salesforce.com solutions. Check out all of the resources that this team maintains on the Architect Core Resources page of Developer Force.

 

tagged , , , Bookmark the permalink. Trackbacks are closed, but you can post a comment.
  • ram_sj

    The limits currently exist are fine since developers are doing some work around, but as long term vision, DML lmits, Viewstate, data storage, number of concurrent transactions and Formula/Rollup should go up to reach more broader audience/complex systems. The salesforce architecture needs to evolve to support all needs as the customer orgs grows.