Big Objects Implementation Guide

Big Objects
Async SOQL
View Big Object Data in Reports and Dashboards
Running Async SOQL Queries
Async SOQL Use Cases
Supported SOQL Commands
PDF

Newer Version Available

This content describes an older version of this product. View Latest

Running Async SOQL Queries

Learn how to run Async SOQL queries on your objects and check on the status of your query using the Chatter REST API.

Formulating Your Async SOQL Query

To use Async SOQL effectively, it’s helpful to understand its key component and other related concepts. Each query is formulated in the POST request as a JSON-encoded list of three or four key-value pairs.

Request body for POST

Name Type Description Required or Optional Available Version
query String Specifies the parameters for the SOQL query you want to execute. The FROM object must be a big object. Required 35.0
operation String Specify whether the query is an insert or upsert. If the record doesn’t exist, an upsert behaves like an insert.

Upsert is not supported for big objects

Note

 Optional 39.0
targetObject String A standard object, custom object, external object, or big object into which to insert the results of the query. Required 35.0
targetFieldMap Map<String, String> Defines how to map the fields in the query result to the fields in the target object.

When defining the targetFieldMap parameter, make sure that the field type mappings are consistent. If the source and target fields don’t match, these considerations apply.

  • Any source field can be mapped onto a target text field.
  • If the source and target fields are both numerical, the target field must have the same or greater number of decimal places than the source field. If not, the request fails. This behavior is to ensure that no data is lost in the conversion.
  • If a field in the query result is mapped more than once, even if mapped to different fields in the target object, only the last mapping is used.

Note

 Required 35.0
targetValueMap Map<String, String> Defines how to map static strings to fields in the target object. Any field or alias can be used as the TargetValueMap value in the SELECT clause of a query.

You can map the special value, $JOB_ID, to a field in the target object. The target field must be a lookup to the Background Operation standard object. In this case, the ID of the Background Operation object representing the Async SOQL query is inserted. If the target field is a text field, it must be at least 15–18 characters long.

You can also include any field or alias in the SELECT clause of the TargetValueMap. They can be combined together to concatenate a value to be used.

 Optional 37.0
targetExternalIdField String The ID of the target sObject. Required for upsert operations. Optional 39.0

This simple Async SOQL example queries SourceObject__b, a source big object, and directs the result to TargetObject__c, a custom object. You can easily map the fields in the source object to the fields of the target object in which you want to write the results.

Example URI
1https://yourInstance.salesforce.com/services/data/v38.0/async-queries/
Example POST request body
1{ 
2   "query": "SELECT firstField__c, secondField__c FROM SourceObject__b",  
3   
4   "operation": "insert",
5   
6   "targetObject": "TargetObject__c", 
7        
8   "targetFieldMap": {"firstField__c":"firstFieldTarget__c",
9                      "secondField__c":"secondFieldTarget__c"
10                      },
11   "targetValueMap": {"$JOB_ID":"BackgroundOperationLookup__c",
12                      "Copy fields from source to target":"BackgroundOperationDescription__c"
13                     }
14}

The response of an Async SOQL query includes the elements of the initial POST request.

Response body for POST

Property Name Type Description Filter Group and Version Available Version
jobId String The ID of the Async SOQL query. This ID corresponds to an entry in the Background Operation standard object. It matches the ID that is used in the targetValueMap when $JOB_ID is used. To get the status of an async query job, use this ID in an Async Query, Status request (/async-queries/jobId). Big, 35.0 35.0
message String A text message that provides information regarding the query, such as an error message if the query failed. Big, 37.0 37.0
operation String Specify whether the query is an insert or upsert. If the record doesn’t exist, an upsert behaves like an insert.

Upsert is not supported for big objects

Note

 Big, 39.0 .39.0
query String Specifies the parameters for the SOQL query you want to execute. The FROM object must be a big object. Big, 35.0 35.0
status String Status of an async query job.
  • Canceled—The job was canceled before it could be run.
  • Complete—The job was successfully completed.
  • Failed—The job failed after the system submitted it or because the request exceeded the Async SOQL limits. The message field provides details on the reason for failure.
  • Running—The job is running successfully, and the org hasn’t exceeded any limits.
  • Scheduled—The new job has been created and scheduled, but is not yet running.
  • New—The job has been created but is not yet scheduled.
 Big, 35.0 35.0
targetExternalIdField String The ID of the target sObject. Required for upsert operations. Big, 39.0 39.0
targetFieldMap Map<String, String> Defines how to map the fields in the query result to the fields in the target object.

When defining the targetFieldMap parameter, make sure that the field type mappings are consistent. If the source and target fields don’t match, these considerations apply.

  • Any source field can be mapped onto a target text field.
  • If the source and target fields are both numerical, the target field must have the same or greater number of decimal places than the source field. If not, the request fails. This behavior is to ensure that no data is lost in the conversion.
  • If a field in the query result is mapped more than once, even if mapped to different fields in the target object, only the last mapping is used.

Note

 Big, 35.0 35.0
targetValueMap Map<String, String> Defines how to map static strings to fields in the target object. Any field or alias can be used as the TargetValueMap value in the SELECT clause of a query.

You can map the special value, $JOB_ID, to a field in the target object. The target field must be a lookup to the Background Operation standard object. In this case, the ID of the Background Operation object representing the Async SOQL query is inserted. If the target field is a text field, it must be at least 15–18 characters long.

You can also include any field or alias in the SELECT clause of the TargetValueMap. They can be combined together to concatenate a value to be used.

 Big, 37.0 37.0
targetObject String A standard object, custom object, external object, or big object into which to insert the results of the query. Big, 35.0 35.0
Example POST response body
1{ 
2   "jobId": "08PD000000003kiT", 
3   
4   "message": "",
5         
6   "query": "SELECT firstField__c, secondField__c FROM SourceObject__b",  
7             
8   "status": "New",
9         
10   "targetObject": "TargetObject__c", 
11         
12   "targetFieldMap": {"firstField__c":"firstFieldTarget__c", 
13                      "secondField__c":"secondFieldTarget__c"
14                     },
15   "targetValueMap": {"$JOB_ID":"BackgroundOperationLookup__c",
16                      "Copy fields from source to target":"BackgroundOperationDescription__c"
17                     } 
18}

Tracking the Status of Your Query

To track the status of a query, specify its jobID with an HTTP GET request.
1https://yourInstance.salesforce.com/services/data/v38.0/async-queries/<jobID>
The response is similar to the initial POST response but with updated status and message fields to reflect the status.
Example GET response body
1{
2              "jobId": "08PD000000000001",
3              "message": "",
4              "query": "SELECT firstField__c, secondField__c FROM SourceObject__b",
5              "status": "Complete",
6              "targetObject": "TargetObject__c",
7              "targetFieldMap": {"firstField__c":"firstFieldTarget__c",
8              "secondField__c":"secondFieldTarget__c" } 
9              }
You can get status information for all queries with the following HTTP GET request. 
1https://yourInstance.salesforce.com/services/data/v38.0/async-queries/
Example GET response body
1{
2              "asyncQueries" : [ {
3              "jobId" : "08PD00000000002",
4              "message" : "",
5              "query" : "SELECT String__c FROM test__b",
6              "status" : "Running",
7              "targetFieldMap" : {
8              "String__c" : "String__c"
9              },
10              "targetObject" : "test__b",
11              "targetValueMap" : { }
12              }, {
13              "jobId": "08PD000000000001",
14              "message": "Complete",
15              "query": "SELECT firstField__c, secondField__c FROM SourceObject__b",
16              "status": "Complete",
17              "targetObject": "TargetObject__c",
18              "targetFieldMap": {"firstField__c":"firstFieldTarget__c",
19              "secondField__c":"secondFieldTarget__c" } 
20              }
21              }

Canceling a Query

You can cancel a query using an HTTP DELETE request by specifying its jobId.

1https://yourInstance.salesforce.com/services/data/v38.0/async-queries/jobId

Canceling a query that has already completed has no effect.

Note

Handling Errors in Async SOQL Queries

Two different types of errors can occur during the execution of an Async SOQL query.
  • An error in the query execution
  • One or more errors writing the results into the target object

Problems in executing the job cause some errors. For example, an invalid query was submitted, one of the Async SOQL limits was exceeded, or the query caused a problem with the underlying infrastructure. For these errors, the response body includes a status of Failed. The message parameter provides more information on the cause of the failure.

Other times, the query executes successfully but encounters an error while attempting to write the results to the target object. Because of the volume of data involved, capturing every error is inefficient. Instead, subsets of the errors generated are captured and made available. Those errors are captured in the BackgroundOperationResult object and retained for seven days. You can query this object with the Async SOQL query jobID to filter the errors for the specific Async SOQL query. Async SOQL job info is retained for a year.