Data loads and integrations that perform poorly or fail altogether are among the most significant problems during salesforce.com customers’ “go-live” efforts. They can both put timelines at risk and become unimaginably expensive, especially for enterprise-scale organizations.

If you’re a Salesforce architect or developer, you can save your organization both time and money—and get up to 20 million records into your organization per hour—when you properly plan and execute your large loads and integrations using the Bulk API. On the surface, one of the most important loading and integration best practices is simple: When you run a Bulk API job, processing more of its batches in parallel means giving that job a higher degree of parallelism, which in turn gives your run better throughput.

Mastering Parallelism to Scale Your Enterprise

But what is a good degree of parallelism, and how can you architect a solution that maximizes the throughput that parallelism gives you? Technical Enablement’s newest article walks you through several optimal and suboptimal test runs, which feature code samples and real-life metrics, so you can master parallelism, and improve your load and integration performance in your organization.

After reading this article, you should be able to:

• Estimate your loads’ and integrations’ degrees of parallelism
• Design an integration or loading tool that gives you the best possible throughput
• Manage and avoid lock exceptions, both of which can significantly reduce throughput
• Take some concrete, actionable best practices back to your organization to scale your enterprise

Related Resources

• The Salesforce Bulk API – Maximizing Parallelism and Throughput Performance When Integrating or Loading Large Data Volumes
• Salesforce API Series: Fast Parallel Data Loading with the Bulk API
• Architect Core Resources
• Working with Very Large SOQL Queries
• Bulk API Developer’s Guide
• General Guidelines for Data Loads
• Long- and Short-Term Approaches for Tuning Force.com Performance
• Inside the Force.com Query Optimizer
• Database Query & Search Optimization Cheat Sheet

About the Author

Sean Regan is an Architect Evangelist within the Technical Enablement team of the salesforce.com Customer-Centric Engineering group. The team’s mission is to help customers understand how to implement technically sound salesforce.com solutions. Check out all of the resources that this team maintains on the Architect Core Resources page of Developer Force.