Set Up Search Mode A/B Testing in B2C Commerce

Compare Agentic Commerce Search results against the native B2C Commerce search engine on live storefront traffic. Read the results in Google Analytics 4.

What You Need Before You Start

Agentic Commerce Search turned on for your storefront.
A Google Analytics 4 (GA4) account.
Familiarity with the Shopper Search API.
SLAS authentication configured for Shopper API calls.

How Search Mode A/B Testing Works

B2C Commerce lets you pin a search request to a specific engine with a searchMode value.

A semantic search request routes the query to the Agentic Commerce Search engine.
A lexical (case-insensitive) search request routes the query to the legacy B2C Commerce Search engine.

Every search response echoes back the mode that was actually served, so you always know which engine produced a given result set. The echo makes a clean A/B comparison possible.

Assign each shopper to a mode.
Tag your analytics with the mode they got.
Compare conversion between the two groups.

Platform and Storefront Responsibilities for Search Mode A/B Testing

Search Mode A/B testing splits work between the B2C Commerce platform and your storefront.

What B2C Commerce provides

The searchMode request parameter and response fields on SCAPI.
Safe fallback when the engine can’t serve the requested mode.
The mode actually served on every response.

What you implement

Deciding which shopper gets which mode (the bucketing algorithm).
Making the assignment sticky per shopper and getting it into the URL.
Emitting the assigned mode to your analytics platform.

The reference implementation is a starting point. You own that code and fork it to fit your storefront and analytics stack.

How Shopper Assignment Works

Three principles shape the design:

You own the bucketing. You pick a seed and a ratio. The assignment is SHA-256(seed:userId) % 100 < ratio ? semantic : lexical. Anyone with the seed and ratio can independently replay userId to mode mapping and verify it, with no opaque service-side bucketing.
Assignment is sticky per shopper. A shopper sees the same engine for every search in their session, and across sessions, for the life of their identity cookie. Otherwise, the conversion data is noise.
The mode rides in the URL. Edge-cached storefront responses are keyed on URL + querystring, so the assigned mode is appended to the search URL as ?searchMode=semantic / ?searchMode=lexical. This segregates the page cache per variant. Two shoppers on different modes never share a cache entry.

The reference implementation computes the mode at session start, redirects cacheable search routes to the URL carrying the mode, and reads that value back on the search request. On the SFRA path, it pins the mode via a Script API call on the product-search model before the query runs.

Call the Parameter on Product Search

Note: The effectiveSearchMode response fields are currently available only in the BETA API version. On lower versions, B2C Commerce silently strips them from the response. Request the BETA version to see them.

searchMode is a top-level parameter on the product-search resource, accepted by both SCAPI and OCAPI. Values are semantic or lexical, and any other value returns HTTP 400. Warning: OCAPI is deprecated as of April 2026. For new work, use SCAPI.

Every response echoes the effectiveSearchMode field, which is the mode actually served. The field may differ from what you requested. For more information, see When Semantic Search Falls Back to Lexical.

Field casing follows each API’s convention:

Surface	Request param	Response fields
SCAPI (`shopper-search`)	`?searchMode=semantic`	`effectiveSearchMode` (camelCase)
OCAPI (`product_search`)	`?searchMode=semantic`	`effective_search_mode` (snake_case)
Script API (`dw.catalog.ProductSearchModel`)	`setSearchMode("semantic")`	`getEffectiveSearchMode()`
SFRA storefront URL	`?searchMode=semantic`	- (cartridge-owned)
GA4 user properties	N/A	`effective_search_mode`

Note: These API calls require a Shopper Login and API Access Service (SLAS) access token. Also, replace {org} and the host with your B2C Commerce instance values.

SCAPI Example:

1GET /search/shopper-search/v1/organizations/{org}/product-search?q=shirt&searchMode=semantic

1{
2  "effectiveSearchMode": "semantic",
3  "hits": [ ... ]
4}

OCAPI Example:

1GET /s/{site}/dw/shop/v23_2/product_search?q=shirt&search_mode=semantic

1{
2  "effective_search_mode": "semantic",
3  "hits": [ ... ]
4}

Script API. On the SFRA / server-side path, the same contract is exposed on dw.catalog.ProductSearchModel. Call setSearchMode("semantic") before search() to pin the mode, and read getEffectiveSearchMode() afterward to see what was actually served. This is the surface the reference implementation uses internally.

When Semantic Search Falls Back to Lexical

The engine cannot serve semantic in a few cases.

The semantic index is missing or not ready.
A single-token product lookup.
A category-browse with the semantic toggle off.

In those cases the platform serves lexical. This is expected, not an error, but a high fallback rate during a test means the semantic engine is unstable and the comparison is contaminated.

Set Up Search Mode A/B Testing

The reference implementation ships as two cartridges:

The plugin_cimulate_ab_experience cartridge is the merchant-agnostic decider and hooks.
The int_merchant_ab_experience cartridge is your configuration and analytics loader.

After you deploy the cartridges, you own both and can fork and adapt them for your storefront.

Configure the abConfig.js File

The only file you must edit is the int_merchant_ab_experience/.../scripts/config/abConfig.js file.

1module.exports = {
2    SEED: 'search-mode-2026',           // pick once, never change mid-test
3    RATIO: 50,                          // 0..100 percent routed to semantic
4    GA4_MEASUREMENT_ID: 'G-XXXXXXXXXX'  // your GA4 measurement ID
5};

Set these values before you deploy.

SEED — Keep this value stable for the life of the test. If you change it, B2C Commerce re-buckets every shopper on their next session and mixes your data.
RATIO — Use any integer from 0 to 100. 50 is a clean 50/50 split to semantic search.
GA4_MEASUREMENT_ID — Match the GA4 property where you registered the custom dimension. If this value is empty, the analytics emission is a silent no-op. An empty measurement ID is a common reason that you see no data in GA4 even after you deploy the cartridges.

Add the Cartridges to the Site’s Cartridge Path

In Business Manager, click App Launcher and then select Administration > Sites > Manage Sites > site > Settings.
Prepend the two cartridges before app_storefront_base, with the merchant cartridge first:
1int_merchant_ab_experience:plugin_cimulate_ab_experience:app_storefront_base
The merchant cartridge overrides the plugin’s placeholder config, so prepend the merchant cartridge first.
Save your changes. SFRA picks up the path changes when you make your next request.

Upload and Deploy the Search Mode A/B Testing Cartridges

Deploy both cartridges to your instance with your normal cartridge upload process (dwupload / SFCC VS Code extension / your CI). Neither cartridge contains Java, so no platform build is required.

Register the Search Mode Dimension in GA4

Create or pick a GA4 property and a Web data stream, and copy the Measurement ID (G-XXXXXXXXXX) into the abConfig.js file.
In GA4, go to Admin > Data display > Custom Definitions > Create custom dimension.
Create a user-scoped custom dimension with these values:
- Name: Search Mode
- Scope: User
- User property: effective_search_mode
Without this dimension, GA4 silently drops the user properties. The values are sent, but never appear in any report or export column.
Grant DebugView access to whoever verifies that effective_search_mode appears in GA4.

Verify Search Mode A/B Testing Before You Run It

Run these checks before you trust GA4 reports for the test.

Run the Response Echo Test

Log in to the storefront as a real customer.
Open the browser’s DevTools and go to Network.
Perform a search and find the product-search response.
Confirm that effectiveSearchMode (or effective_search_mode on OCAPI) is semantic or lexical.

Run the Analytics Emission Test

Open the browser’s DevTools and go to Network.
Filter by collect. Confirm that the GA4 collect request body includes a user_properties block with effective_search_mode.
Alternatively, run dataLayer in the console and look for the gtag('set', 'user_properties', ...) entry.

Run the GA4 DebugView Test

In GA4, go to Admin > Data display > DebugView.
Append ?_dbg=1 to the storefront URL (or use the Google Analytics Debugger Chrome extension).
Click any event. The User Properties panel lists effective_search_mode.

Run the Search Mode A/B Test

After the assignment flows and the DebugView test shows effective_search_mode on every event, run the test. Track these questions as you run the test.

Question	Where to look	What to check
Did every shopper get assigned?	Realtime > Users by user property (dimension = `effective_search_mode`)	Populated on essentially every active user. A chunk of `(not set)` means the cartridge isn’t pinning and usually indicates a cartridge-path order issue.
What’s the assignment ratio?	Explore, dimension = `effective_search_mode`, metric = `users`	Roughly `RATIO%` semantic / `(100-RATIO)%` lexical.
Does semantic convert better?	Explore, dimension = `effective_search_mode`, metrics = `transactions`, `purchase_revenue`, `add_to_cart`	The actual A/B result. Wait for statistical significance before drawing conclusions.
Did tagging stick across the session?	DebugView, single user, multiple events	Every event for one user should carry the same `effective_search_mode`. If it flips mid-session, the assignment isn’t sticky.

Troubleshoot Search Mode A/B Testing

Symptom	Likely cause
`effective_search_mode` always `lexical` on `?searchMode=semantic` requests	The semantic engine is unhealthy for that site, and may indicate index missing, single-token product search, or category-browse with the semantic toggle off.
`effective_search_mode` populated on the response but not in GA4 reports	Custom dimensions weren’t registered. GA4 silently drops user properties that aren’t pre-registered.
`effective_search_mode` flips mid-session for the same user	The shopper’s session or identity isn’t surviving the request and typically indicates a cross-domain boundary or blocked cookies.
Assignment ratio is far off `RATIO`	`SEED` was changed mid-test. Reset and start fresh because the existing data is mixed.
Response omits the `effectiveSearchMode` field entirely	Request was made against a version below BETA. The fields are BETA-gated and stripped on lower versions. Retry against BETA.