# Use LLM Gateway Calls in Custom Chunking Functions

Make LLM gateway calls from custom chunking functions when your chunking behavior depends on semantic understanding. Use this approach to define chunking behavior in prompt instructions and map model responses to the required response schema.

| Edition Table                                                                                                                                                                                                 |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Available in: **Developer**, **Enterprise**, **Performance**, and **Unlimited** Editions. See [Data 360 edition availability](https://help.salesforce.com/s/articleView?id=sf.c360_a_dc_editions.htm&type=5). |

| Permission Sets Needed                             |                                                        |
| -------------------------------------------------- | ------------------------------------------------------ |
| To use LLM gateway calls in custom chunking logic: | Permission set: <ul><li>Data Cloud Architect</li></ul> |

## Prerequisites

- Set up Salesforce CLI, the Data Code Extension plugin, and the Python SDK. See [Set Up Salesforce CLI for Custom Functions](/docs/data/data-cloud-code-ext/guide/set-up-sdk-for-functions.md).
- Initialize a function package and make sure that you can run local function validation commands. See [Write a Custom Chunking Function](/docs/data/data-cloud-code-ext/guide/write-custom-chunking-function.md).
- Use a Salesforce org with at least one generative AI model configured.
- Configure an external client app to authenticate calls to LLM models. Complete only the [Create a Salesforce App](https://developer.salesforce.com/docs/ai/agentforce/guide/agent-api-get-started.md#create-a-salesforce-app) and [Obtain Credentials](https://developer.salesforce.com/docs/ai/agentforce/guide/agent-api-get-started.md#obtain-credentials) sections of [Get Started with the Agent API](https://developer.salesforce.com/docs/ai/agentforce/guide/agent-api-get-started.md).

## Use LLM Gateway for Semantic Chunking

Implement this flow inside the `function(request, runtime)` callable in `payload/entrypoint.py`: prompt construction, LLM Gateway invocation, response parsing, and chunk mapping. For more information on writing a custom function, see [Write and Validate a Custom Chunking Function](/docs/data/data-cloud-code-ext/guide/write-custom-chunking-function.md).

1. Read the document elements to chunk from `request.input`.

2. Build the prompt that you send to the model. Write your chunking instructions in a template file or directly in `entrypoint.py`, and insert the document text from `request.input` into those instructions at run time.

   For a reference implementation, see `example/chunking_with_llm/entrypoint.py` and `example/chunking_with_llm/files` in the initialized function package.

3. Build a text-generation request with `GenerateTextRequestBuilder` and set a supported model.

4. Call `runtime.llm_gateway.generate_text(...)`.

5. Parse the response text, for example, by using `---CHUNK---`.

6. Map each segment to a chunk (`text`, `seq_no`, `chunk_type`). Aggregate chunks across all elements with a continuous `seq_no`, and return `SearchIndexChunkingV1Response(output=chunks)`.

Handle failed Gateway responses explicitly. Log failures and raise controlled errors. Don't place sensitive data in prompts or logs.

## Validate LLM Gateway Calls Locally

1. From your function package root, confirm that your function entrypoint and test payload files are ready for local validation.

2. Authenticate with your org.

   ```bash
   sf org login web \
     --alias myorg \
     --instance-url https://{MY_DOMAIN_URL} \
     --client-id {CONSUMER_KEY} \
     --scopes "sfap_api api"
   ```

   Replace `{MY_DOMAIN_URL}` with your org's domain and `{CONSUMER_KEY}` with your external client app consumer key.

3. Run local validation against your function entrypoint and test the payload.
   - `sf data-code-extension function run -e payload/entrypoint.py -t tests/test.json`

## See Also

- [Write a Custom Chunking Function](/docs/data/data-cloud-code-ext/guide/write-custom-chunking-function.md)
- [Considerations When Writing Custom Chunking Functions](/docs/data/data-cloud-code-ext/guide/considerations-when-writing-custom-function.md)
- [Use Predictive Model Calls in Custom Chunking Functions](/docs/data/data-cloud-code-ext/guide/use-predictive-model-in-custom-function.md)
- [Use Custom Functions in Data 360](/docs/data/data-cloud-code-ext/guide/use-custom-function.md)