Use a Custom Chunking Function in a Search Index

After you deploy a custom chunking function, use it in a search index by creating a search index with Advanced Setup and selecting your function as the Chunking Strategy for the fields you want to chunk.

Edition Table
Available in: Developer, Enterprise, Performance, and Unlimited Editions. See Data 360 edition availability.
User Permissions Needed
To use a custom chunking function in a search index:Permission set:
  • Data Cloud Architect
  • Deploy your custom chunking function and make sure that its deployment status is Deployed. See Deploy a Custom Function to Data 360 by Using UI.
  • Confirm that the content you want to chunk, for example, an uploaded PDF, is ingested and is available as the data model object (DMO) or unstructured DMO (UDMO) that your search index uses. Chunking runs only on content that exists in the selected source; if a document didn’t upload or ingest successfully, no chunks are created for it.

  1. In Data 360, go to the Search Indexes and click New.

    If the tab is not visible, use More to find Search Index.

  2. Select Advanced Setup and click Next to open the full Search Index Builder.

  3. In the Search Index Builder, select the data model object that contains the data you want to chunk (for example, a DMO with unstructured or text fields). Click Next.

  4. In the Select Fields To Chunk section, click Manage Fields.

  5. In the Manage Fields modal, select the fields you want to chunk with your custom function (for example, a description or long-text field). Click Save.

  6. For each field you want to chunk with your custom code:

    • In the dropdown next to the field name, set Chunking Strategy to Code Extension.
    • Select your deployed custom function from the Function Name list.
    • Click Save.

  7. Click Next and complete the rest of the search index configuration (for example, embedding model for vectorization, fields for filtering, and index name and API name).

  8. Click Save to create the search index.

    Saving can take several minutes. Stay on the page until the builder finishes or shows an error.

  9. After the search index is created, reload the record page and wait until Search Index Last Run Status shows Ready before verifying chunked data. This can take several minutes.