agent test create
Create an agent test in your org using a local test spec YAML file.
To run this command, you must have an agent test spec file, which is a YAML file that lists the test cases for testing a specific agent. Use the “agent generate test-spec” CLI command to generate a test spec file. Then specify the file to this command with the –spec flag, or run this command with no flags to be prompted.
When this command completes, your org contains the new agent test, which you can view and edit using the Testing Center UI (legacy) or Agentforce Studio (NGT). This command also retrieves the metadata component associated with the new test to your local Salesforce DX project and displays its filename. By default, the legacy AiEvaluationDefinition is created; use –test-runner agentforce-studio to author an AiTestingDefinition (NGT) instead.
After you’ve created the test in the org, use the “agent test run” command to run it.
| Flag Name (Long) | Flag Name (Short) | Description |
|---|---|---|
‑‑api‑name | N/A | Type: Value API name of the new test; the API name must not exist in the org. |
‑‑api‑version | N/A | Type: Value Override the api version used for api requests made by this command |
‑‑flags‑dir | N/A | Type: Value Import flag values from a directory. |
‑‑force‑overwrite | N/A | Type: Boolean Don’t prompt for confirmation when overwriting an existing test (based on API name) in your org. |
‑‑json | N/A | Type: Boolean Format output as json. |
‑‑preview | N/A | Type: Boolean Preview the test metadata file without deploying to your org. |
‑‑spec | N/A | Type: Value Path to the test spec YAML file. |
‑‑target‑org | ‑o | Type: Value Required Username or alias of the target org. Not required if the target-org configuration variable is already set. |
‑‑test‑runner | N/A | Type: Value Valid Values: agentforce-studio, testing-centerExplicitly specify which test runner to use (agentforce-studio or testing-center). By default, the command automatically detects which test runner to use based on the test definition metadata type in your org. Use this flag to explicitly specify the runner type. ‘agentforce-studio’ uses AiTestingDefinition metadata. ‘testing-center’ uses AiEvaluationDefinition metadata. |
Create an agent test interactively and be prompted for the test spec and API name of the test in the org; use the default org:
Create an agent test and use flags to specify all required information; if a test with same API name already exists in the org, overwrite it without confirmation. Use the org with alias “my-org”:
Preview what the agent test metadata (AiEvaluationDefinition) looks like without deploying it to your default org:
Author an Agentforce Studio (NGT) test from an NGT-shaped YAML; writes an AiTestingDefinition metadata file: