Loading Data from the API

GoalBest Practice
Improving performanceAny data operation that includes more than 2,000 records is a good candidate for Bulk API 2.0 to successfully prepare, execute, and manage an asynchronous workflow that makes use of the Bulk framework. Jobs with fewer than 2,000 records should involve “bulkified” synchronous calls in REST (for example, Composite) or SOAP.
Using the most efficient operations
  • Use the fastest operation possible—insert() is fastest, update() is next, and upsert() is next after that. If possible, also break upsert() into two operations: create() and update().
  • Ensure that data is clean before loading when using the Bulk API 2.0. Errors in batches trigger single-row processing for that batch, and that processing heavily impacts performance.
Reducing data to transfer and processWhen updating, send only fields that have changed (delta-only loads).
Reducing transmission time and interruptionsFor custom integrations:
  • Authenticate once per load, not on each record.
  • Use GZIP compression and HTTP keep-alive to avoid drops during lengthy save operations.
Avoiding unnecessary overheadFor custom integrations, authenticate once per load, not on each record.
Avoiding computationsUse Public Read/Write security during initial load to avoid sharing calculation overhead
Reducing computations
  • If possible for initial loads, populate roles before populating sharing rules.
    1. Load users into roles.
    2. Load record data with owners, triggering calculations in the role hierarchy.
    3. Configure public groups and queues, and let those computations propagate.
    4. Add sharing rules one at a time, letting computations for each rule finish before adding the next one.
  • If possible, add people and data before creating and assigning groups and queues.
    1. Load the new users and new record data.
    2. Optionally, load new public groups and queues.
    3. Add sharing rules one at a time, letting computations for each rule finish before adding the next one.
Deferring computations and speeding up load throughputDisable Apex triggers, workflow rules, and validations during loads; investigate the use of batch Apex to process records after the load is complete.
Balancing efficient batch sizes against potential timeoutsWhen using the SOAP API, use as many batches as possible—up to 200—that still avoid network timeouts if:
  • Records are large.
  • Save operations entail a lot of processing that cannot be deferred.
Optimizing the Lightning Platform Web Service Connector (WSC) to work with SalesforceUse WSC instead of other Java API clients, like Axis.
Minimizing parent record-locking conflictsWhen changing child records, group them by parent—group records by the field ParentId in the same batch to minimize locking conflicts.
Deferring sharing calculationsUse the defer sharing calculation permission to defer sharing calculations until after all data has been loaded. (See Defer Sharing Calculation.)
Avoiding loading data into SalesforceUse mashups to create coupled integrations of applications. (See Using Mashups.)