Prompt Template Batch Processing in Apex

To asynchronously generate large quantities of responses for prompt templates you can use Prompt Template Batch Processing. You can batch and execute jobs in Flow and Apex. For Flow, see Flow Core Action: Prompt Template Actions.

See the object reference guide for AiJobRun and AiJobRunItem.

There are several immutability and status constraints for batch processing standard objects.

Once an AiJobRun begins processing and the status updates to InProgress, some AiJobRunItem fields become immutable.

  • You can't change the AiJobRunId and Input field values.
  • AiJobRunItem schema fields set to a non-null value (except status) can't be changed.
  • You can't delete the AiJobRun or any of its AiJobRunItems.

These statuses for both AiJobRun and AiJobRunItem can't be changed by the user:

  • InProgress
  • Completed
  • Failed

To initiate an AiJobRun for batch execution, update the status to ReadyToStart.

There's a limit of 1,000 AiJobRunItems for each AiJobRun.

When using models that support native batch processing, there is a limit of 10,000 AiJobRunItems for each AiJobRun. Additionally, in Apex, you can start up to 5 AiJobRun objects in 24 hours.

For a list of models that support native batch processing, see Large Language Model Support.

When an AiJobRun object is created, it's assigned a CreatedDate field with a dateTime timestamp. Multiple AiJobRun objects with a ReadyToStart status begin processing in chronological order of the CreatedDate value. However, if several AiJobRun objects are updated to ReadyToStart within a few seconds of one another, the start time may not strictly follow the CreatedDate order. This processing order behavior applies to all AiJobRun objects, regardless of how their status is updated.

Use this example when you need to generate LLM output for a large set of records without blocking a user-facing transaction. For instance, a nightly summarization of a Service Cloud case backlog.

The code below summarizes the Subject and Description of a set of Case records using a prompt template named Summarize_Case, and writes each summary back as an internal CaseComment (see Monitor Job Completion below). The same three-step pattern applies to any prompt template and any SObject input.

  • A prompt template with DeveloperName Summarize_Case exists in the org and declares a Case SObject input variable.
  • The user running the job is assigned the standard Prompt Template User permission set.

All three steps below run in the same Apex transaction. jobRun.Id is only available after the insert in Step 1, and every item must be inserted before the status flip in Step 3.

The prompt template reference lives on AiJobRun.Target, which is a string that accepts either the template's DeveloperName or its record Id. The DeveloperName is recommended since it's stable across orgs. Status is required on insert. Use New so Apex can flip it to ReadyToStart after the items are in place.

There’s one AiJobRunItem per input record. The Input field is a JSON string whose keys use the documented Input: prefix, and the value under Input:Case is a record pointer with an id. Insert Status = 'Ready' on every item.

For runs approaching the 10,000-item cap, wrap this step in a Database.Batchable so each execute scope gets fresh governor limits.

Once every item is inserted, updating the parent to ReadyToStart hands the batch off to the platform orchestrator. Nothing should change on the job or its items after this point. See Considerations: Immutability for more details.

The platform publishes an AiJobRunStatusEvent on each transition into InProgress, Completed, and Failed. Subscribe to AiJobRunStatusEvent with a platform-event trigger and filter for Completed. See Subscribe to Platform Event Notifications with Apex Triggers for more information.

When the job completes, you can query the related AiJobRunItem records to access both the original input data and the generated responses. The Input field contains the input data for each individual item within the job run, and the Response field contains the corresponding LLM response as JSON in the form {"promptResponse":"<LLM text>"}. Unwrap the JSON envelope before using the text.

Event schema: AiJobRunIdentifier (The parent AiJobRun Id as a string), Status, JobType, Target.

The handler queries the finished AiJobRunItem rows, parses each Input JSON ({"Input:Case":{"id":"<CaseId>"}}) back to the source Case Id, unwraps the Response envelope ({"promptResponse":"<LLM text>"}) to the LLM text, and writes that text as an internal CaseComment.

Platform events can be redelivered, so a production handler should remove duplicates (for example, by keying off AiJobRunId + ParentId). Individual items can also finish with Status = 'Failed'. Query and surface those alongside the successes rather than silently skipping them. For runs approaching the 10,000-item cap, offload the write-back to a Database.Batchable (or enqueue a Queueable per scope) so each execute gets fresh governor limits.

AiJobRun.JobType can differ based on how the batch is initiated.

  • Apex-created batch runs: PromptTemplate
  • Flow (Prompt Template Batch Generation action): GeneratePromptAsyncIA