Use Custom Functions in Data 360

Create custom chunking functions for the search index builder by using the Salesforce CLI, the code extension plugin, and the Data Custom Code Python SDK. In code extension, function runs are specific to search index chunking. Write and test your function locally, then deploy it to Data 360. When you create a search index with Advanced Setup, select your deployed function as the chunking strategy to control how content is split for search and AI retrieval.

The search index Advanced Setup offers built-in chunking strategies, such as section-aware chunking and passage extraction, that work well for many scenarios. A custom chunking function adds another option when you need to tailor chunk boundaries to your content or retrieval needs. For example, keep a product name with its specifications in one chunk so that retrievers and prompts have full context and retrieval-augmented generation (RAG) answers stay accurate.

Your function is a targeted chunking extension, not a full data processing job. It does not read from data lake objects (DLOs) or data model objects (DMOs) directly during runtime. During one search index run, Data 360 can invoke your function multiple times depending on batching, so design your logic to process each request independently.

The search index pipeline processes content in this sequence:

  • Data 360 reads source content from selected DLOs, DMOs, or unstructured DMOs.
  • Data 360 parses the source content into SearchIndexChunkingV1DocElement objects.
  • Data 360 invokes your function and passes those objects in a SearchIndexChunkingV1Request payload.
  • Your function returns chunk output in a SearchIndexChunkingV1Response payload, and Data 360 vectorizes and indexes those chunks.
Edition Table
Available in: Developer, Enterprise, Performance, and Unlimited Editions. See Data 360 edition availability.
Permission Sets Needed
To use custom functions in Data 360:Permission set:
  • Data Cloud Architect

  1. Make sure that you have access to a Data 360 org with appropriate permission sets.

  2. Make sure that you have access to a Data 360 sandbox with appropriate permission sets. If you do not have a sandbox, create one. See Create a Data 360 Sandbox.

  1. Go to Data Cloud Setup.
  2. Under Feature Management, click Feature Manager.
  3. Click Enable next to Code Extension.
  4. Agree to the terms of service, and then confirm that you want to enable code extension.

After the feature is enabled, the Code Extension tab appears in the UI. See Enable Data 360 Features in Feature Manager.

Confirm that the data sources you plan to use in the search index contain the content you want to chunk, for example, fields on a data model object. Your function receives only content that was successfully ingested and is readable by the search index pipeline from those configured sources.

Salesforce CLI with the code extension plugin and the Data Custom Code Python SDK work together as your local toolchain. Set up Salesforce CLI to develop, test, package, and deploy custom chunking functions locally, and then deploy them to Data 360 for execution. See Set Up Salesforce CLI for Custom Functions.

Write your custom chunking logic in your preferred local development environment, and validate it with sample request payloads or in a Data 360 org. See Write a Custom Chunking Function and Considerations When Writing Custom Chunking Functions.

All code execution logs are written to a dedicated Logs DLO (DataCustomCodeLogs__dll) for observability and troubleshooting. Data governance policies do not apply to the Logs DLO, and any user with access to the Logs DLO can view its contents. To prevent inadvertent exposure of sensitive information, ensure that your custom functions output only information that is appropriate for users with access to the Logs DLO to see. Do not output personally identifiable information (PII), credentials, or other sensitive data to standard output (for example, Python print() statements or console logging).

The deployment process uploads the deployment package to Data 360. Deploy by using the Salesforce CLI or the Code Extension UI. See Deploy a Custom Function to Data 360 Sandbox by Using CLI and Deploy a Custom Function to Data 360 by Using UI.

Monitor your deployment to verify that it is successful and that your custom function is available for selection in the search index builder. See Monitor Custom Function Deployments.

Create a search index with Advanced Setup, select the fields to chunk, set Chunking Strategy to Code Extension, and select your deployed function. See Use a Custom Chunking Function in a Search Index.

After the search index run completes and the index reaches a Ready status, verify chunked output in Data Explorer or by using a retriever and Prompt Builder. See Verify Custom Chunking Results.

Review deployment status and logs in the sandbox to verify that chunking behaves as expected. If you find issues, return to Step 5 to modify and redeploy your function. See Monitor Custom Function Deployments, Query Code Extension Logs by Using Query Editor, and Query Code Extension Logs by Using the Query API.

Use a DevOps data kit to migrate from sandbox to production in either of these ways:

  • Move only the custom function deployment, and then create a new search index in production that uses that function.
  • Move the search index, custom function deployment, and required dependencies together in one migration.

See Migrate Custom Function to Production.

In production, confirm that your migrated function deployment is available, create or update the search index to use Code Extension chunking with that function, and monitor search index job status and logs until you see a stable Ready status and expected chunk content. See Use a Custom Chunking Function in a Search Index, Verify Custom Chunking Results, and Monitor Custom Function Deployments.