Create Custom Job Step Types
To create a custom job step type, you must develop a task-oriented or chunk-oriented CommonJS script module or legacy pipeline to define what the step does. You must also create a JSON or XML steptypes file that describes the step, and then include the module or pipeline along with the steptypes file in a cartridge uploaded to the site.
To create a custom job step type, you must be familiar with creating and uploading a cartridge.
- Create a cartridge.
- Create a task-oriented or chunk-oriented CommonJS module to describe what the step does.
A best practice is to put the CommonJS module in the
cartridge/scriptsdirectory or subdirectory, for example,my_cartridge/cartridge/scripts/steps. Instead of using a CommonJS module, you can use the UX Studio plug-in to the Eclipse IDE to create a pipeline. However, we recommend using CommonJS modules for jobs in the new framework. - Create a
steptypes.jsonorsteptypes.xmlfile that describes the job step and put the file in the root directory of the cartridge. You can describe more than one custom job step in the same file. Each cartridge can have only onesteptypes.jsonorsteptypes.xmlfile. - Upload the cartridge, and include it in the cartridge path.
- For a production system, replicate the code to production, and activate the code version that includes your cartridge.
When an administrator creates a job in Business Manager, custom steps defined in the steptypes.json or steptypes.xml file are listed as steps available to add to the job.
You can define what a custom step type does using task- or chunk-oriented CommonJS script module, or a pipeline module, which executes a legacy pipeline.
Chunk-oriented steps cannot be implemented with a pipeline module.
A task-oriented CommonJS script module exposes a function to be called as the main function for the job step. When administrators create job steps using Business Manager, they set parameters that are available as scriptable objects for the module's function and for the dw.job.JobStepExecution object. The dw.job.JobStepExecution object allows read-only access to information about the current step execution and job execution.
The task-oriented script module is used as its best if there is a single task to execute where the progress cannot be calculated or the boundaries are hard to calculate.
To control the exit status, the script module's function can return a dw.system.Status object. If the script finishes with an unhandled exception, the exit status code is ERROR, and the error status flag is true by default. If no status object is returned and no exception occurs, the status code is OK by default.
The following example is a task-oriented CommonJS module:
A chunk-oriented CommonJS script module reads, processes, and writes items in chunks of a defined size. If the list contains more items than can be processed in one chunk, a new chunk is started. A chunk-oriented script can include any series of processing steps, not just database transactions. A job step that uses a chunk-oriented script allows fine-grained progress monitoring because the number of elements that are written is updated each time a chunk is finished.
The chunk-oriented script module is used as its best if the job step type processes countable data where the boundaries can be quickly calculated or are fixed.
When administrators create jobs using Business Manager, they set parameters that are available as scriptable objects for each exposed module function and for the dw.job.JobStepExecution object. The dw.job.JobStepExecution object allows read-only access to information about the current step execution and job execution. You can't define the exit status for a chunk-oriented script module. Chunk modules always finish with either OK or ERROR.
In the following chunk processing example, a list of 15 orders is exported to a file. Each chunk includes four orders:
| Chunk | Steps |
|---|---|
| Chunk 1 |
|
| Chunk 2 |
|
| Chunk 3 |
|
| Chunk 4 |
|
Chunk 4 read, processed, and wrote fewer items than the previous chunks, because only three items were left on the list.
To process the data in chunks, the developer has to implement the following methods:
| Function | Required or Optional | Purpose |
|---|---|---|
| beforeStep | Optional | Executed before a chunk step begins. Implements logic before all items of all chunks are read, processed, and written. |
| getTotalCount | Optional | Returns the total number of items that are available. Called by the framework exactly once before chunk processing begins. A known total count allows better monitoring, for example, to show that 50 of 100 items have already been processed. |
| beforeChunk | Optional | Executed before a chunk begins. Implements logic before a chunk of S items is read, processed, and written. |
| read | ❗Required❗ | Returns one item or nothing if there are no more items. |
| process | ❗Required❗ | Transforms items and applies business logic to them. It receives the item returned by the read function, performs a process, and returns one item.The item returned can be the same item that the read function returned if no processing logic is necessary, or it can be a new item of a different type. If the process function returns nothing, then the read function item is filtered and doesn't appear in the list of items to be written later. |
| write | ❗Required❗ | Receives a list of items. The list size matches the chunk size or, if the number of items in the last available chunk is smaller, it is smaller. The write function returns nothing. |
| afterChunk | Optional | Executed after a chunk finishes. Implements logic after a chunk of S items has been read, processed, and written successfully. |
| afterStep | Optional | Executed after a chunk step finished successfully. Implements logic after all items of all chunks are read, processed, and written successfully. |
The following example is a chunk-oriented CommonJS module:
A pipeline module executes the provided pipeline with the defined parameters. The pipeline is an XML file created and changed with UXStudio, a plugin for Eclipse.
The pipeline module should not be used anymore and available pipeline-based modules should be migrated to CommonJS script modules.
In a task-oriented script module or pipeline for a custom job step, you can create an object of type dw.system.Status to control the exit status and error message for the step. The Status object includes a boolean flag that indicates whether or not the status is error, and also a status code and message. Messages appear in Business Manager and are written to log files. You can't define the exit status for a chunk-oriented script module.
Chunk-oriented script modules always finish with either OK or ERROR.
The job framework uses the following rules to detect the step status and determine if job execution can continue:
| dw.system.Status.isError() | dw.system.Status.getCode() | Status Code | Error Status Flag | Job Execution Behavior (if not handled by a transition) | Notes |
|---|---|---|---|---|---|
| n/a | n/a | OK | FALSE | Continue | |
| FALSE | n/a | OK | FALSE | Continue | |
| FALSE | OK | OK | FALSE | Continue | |
| FALSE | <CUSTOM> | <CUSTOM> | FALSE | Continue | Custom status codes can't contain any comma (,) or wildcard (*) characters, include leading or trailing whitespace, or exceed 100 characters. |
| TRUE | n/a | ERROR | TRUE | Stop | |
| TRUE | ERROR | ERROR | TRUE | Stop | |
| TRUE | <CUSTOM> | ERROR | TRUE | Stop | Custom status codes for statuses that represent an error are not supported. If a custom status code is used the code is replaced with ERROR. |
If a pipeline or script module does not return a dw.system.Status object or if a pipeline or script module's function fails before a Status object is returned, the following rules apply.
| Script Module Function | Status Code | Error Status Flag | Job Execution Behavior (if not handled by a transition) |
|---|---|---|---|
| Finished with unhandled JavaScript exception | ERROR | TRUE | Stop |
| Runtime exceeds configured (or default) timeout | ERROR | TRUE | Stop |
Returns an object that isn't a dw.system.Status object | OK | FALSE | Continue |
| Returns no object | OK | FALSE | Continue |
| Pipeline | Status Code | Error Status Flag | Job Execution Behavior (if not handled by a transition) | Notes |
|---|---|---|---|---|
| Finished with stop node | ERROR | TRUE | Stop | |
| Finished with interaction node | ERROR | TRUE | Stop | |
| Finished with unhandled exception | ERROR | TRUE | Stop | |
| Finished with end node | OK | FALSE | Continue | The name of the end node isn't relevant |
Stores an object that isn't a dw.system.Status object under key ExitStatus in pipeline dictionary | OK | FALSE | Continue | |
Stores no object under key ExitStatus in pipeline dictionary | OK | FALSE | Continue |
Custom Step Types can be registered in the system with a steptypes.json file. Within this file, you describe the step types of the cartridge, their parameters, and status codes.
The file must be named steptypes.json and placed in a custom cartridge in the root folder. You can only have one steptypes.json file per cartridge.
The steptypes.json file requires a specific JSON syntax. If a steptypes.json file contains errors, the errors are logged and the custom step types are not registered. The system then loads step types from the steptypes.json files of other cartridges.
A typical error message for an invalid steptypes.json file is:
Invalid step [Step1]! Type with id [custom.MyCustomStep1] is unknown!.
You can have either a steptypes.json or a steptypes.xml file only! It is not supported to have both files in the cartridge.
Use the steptypes.schema.json file to check the syntax of your steptypes.json file.
The steptypes.schema.json file can be downloaded here.
| Attribute | Required or Optional | Description |
|---|---|---|
| step-types | ❗Required❗ | The root object of the steptypes.json file. |
| script-module-step | Basically optional, but required if no chunk-script-module-step or pipeline-step is defined. | Defines one or more task-oriented script step types. |
| chunk-script-module-step | Basically optional, but required if no script-module-step or pipeline-step is defined. | Defines one or more chunk-oriented script step types. |
| pipeline-step | Basically optional, but required if no script-module-step or chunk-script-module-step is defined. | Defines one or more pipeline-based step types. |
| Attribute | Required or Optional | Description |
|---|---|---|
| @type-id | ❗Required❗ | Identifies the step type. This is the name that users see in Business Manager, so make it descriptive. Must begin with custom.. Must not contain leading or trailing white space or more than 100 characters. Must be unique within the job definition. You can't register multiple steps with the same @type-id in different cartridges.The @type-id is validated as unique by parsing the steptypes files from all cartridges on the cartridge path. If there is a step with the same @type-id, the step isn't loaded. The @type-id value can't be the same as any system step, for example, ExecutePipeline or IncludeStepsFromJob. |
| @supports-parallel-execution | Optional | Determines if the step type can be used in parallel with itself or other step types. Must have value true or false. Default is true. If false, split flows that contain steps of this type are always executed sequentially and never in parallel. If true, split flows that contain steps of this type are executed in parallel, as long as: - The split flows don't contain steps with supports-parallel-execution = false. - The split flows are not configured to be executed sequentially. - There are enough resources available to do parallel execution. |
| @supports-site-context | Optional | Determines if the step type is intended to be used in site context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with one or more sites as scope. If false, steps of this type can't be used for flows with one or more sites as scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| @supports-organization-context | Optional | Determines if the step type is intended to be used in organization context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with organization scope. If false, steps of this type can't be used for flows with organization scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| description | Optional | The description of the step type. Not shown in Business Manager. Must not exceed 4000 characters. |
| module | ❗Required❗ | Path to the script module to be executed. Must not contain leading or trailing white space. |
| function | ❗Required❗ | The function of the script module to execute. Must not contain leading or trailing white space. If not defined, the script module must export a function named execute. |
| transactional | Optional | Indicates if the module requires a database transaction. Must have value true or false. Default is false. If this value is set to true, the job step executes as a single, potentially very large, transaction. To avoid a negative impact on system performance and allow more granular transaction control, keep the default setting of false. Implement transaction handling within the job step using the dw.system.Transaction API. |
| timeout-in-seconds | Optional | Sets the timeout in seconds for the script module's function. Must be an integer greater than zero. There is no default timeout, but setting a limit is recommended. |
| parameters | Optional | Parameters for the step, which the user configures in Business Manager. Contains one parameter array-element with one or more parameter objects. |
| status-codes | Optional | Defines the meta data for status codes returned for the step. Contains one status array-element with one or more status objects. |
| Attribute | Required or Optional | Description |
|---|---|---|
| @type-id | ❗Required❗ | Identifies the step type. This is the name that users see in Business Manager, so make it descriptive. Must begin with custom.. Must not contain leading or trailing white space or more than 100 characters. Must be unique within the job definition. You can't register multiple steps with the same @type-id in different cartridges.The @type-id is validated as unique by parsing the steptypes files from all cartridges on the cartridge path. If there is a step with the same @type-id, the step isn't loaded. The @type-id value can't be the same as any system step, for example, ExecutePipeline or IncludeStepsFromJob. |
| @supports-parallel-execution | Optional | Determines if the step type can be used in parallel with itself or other step types. Must have value true or false. Default is true. If false, split flows that contain steps of this type are always executed sequentially and never in parallel. If true, split flows that contain steps of this type are executed in parallel, as long as: - The split flows don't contain steps with supports-parallel-execution = false. - The split flows are not configured to be executed sequentially. - There are enough resources available to do parallel execution. |
| @supports-site-context | Optional | Determines if the step type is intended to be used in site context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with one or more sites as scope. If false, steps of this type can't be used for flows with one or more sites as scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| @supports-organization-context | Optional | Determines if the step type is intended to be used in organization context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with organization scope. If false, steps of this type can't be used for flows with organization scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| description | Optional | The description of the step type. Not shown in Business Manager. Must not exceed 4000 characters. |
| module | ❗Required❗ | Path to the script module to be executed. Must not contain leading or trailing white space. |
| before-step-function | Optional | References the function of the chunk-oriented script module to be executed before the step begins. If not defined, no before-step logic is used. |
| total-count-function | Optional | References the function to be executed to determine the total number of items. If not defined, no total count is used. |
| before-chunk-function | Optional | References the function of the chunk-oriented script module to be executed before a chunk begins. If not defined, no before-chunk logic is used. |
| read-function | Optional | Read function of the chunk script. If not defined, script must export a function named read. |
| process-function | Optional | Process function of the chunk script. If not defined, script must export a function named process. |
| write-function | Optional | References the write function of the chunk script. If not defined, script must export a function named write. |
| after-chunk-function | Optional | References the function of the chunk-oriented script module to be executed after a chunk has finished. If not defined, no after-chunk logic is used. |
| after-step-function | Optional | References the function of the chunk-oriented script module to be executed after the step has finished. If not defined, no after-step logic is used. |
| chunk-size | ❗Required❗ | Defines the size of one chunk. Must be a numeric value greater than zero. Must not contain leading or trailing white space. |
| transactional | Optional | Indicates if the module requires a database transaction. Must have value true or false. Default is false. If this value is set to true, the job step executes as a single, potentially very large, transaction. To avoid a negative impact on system performance and allow more granular transaction control, keep the default setting of false. Implement transaction handling within the job step using the dw.system.Transaction API. |
| parameters | Optional | Parameters for the step, which the user configures in Business Manager. Contains one parameter array-element with one or more parameter objects. |
| status-codes | Optional | Defines the meta data for status codes returned for the step. Contains one status array-element with one or more status objects. |
| Attribute | Required or Optional | Description |
|---|---|---|
| @type-id | ❗Required❗ | Identifies the step type. This is the name that users see in Business Manager, so make it descriptive. Must begin with custom.. Must not contain leading or trailing white space or more than 100 characters. Must be unique within the job definition. You can't register multiple steps with the same @type-id in different cartridges.The @type-id is validated as unique by parsing the steptypes files from all cartridges on the cartridge path. If there is a step with the same @type-id, the step isn't loaded. The @type-id value can't be the same as any system step, for example, ExecutePipeline or IncludeStepsFromJob. |
| @supports-parallel-execution | Optional | Determines if the step type can be used in parallel with itself or other step types. Must have value true or false. Default is true. If false, split flows that contain steps of this type are always executed sequentially and never in parallel. If true, split flows that contain steps of this type are executed in parallel, as long as: - The split flows don't contain steps with supports-parallel-execution = false. - The split flows are not configured to be executed sequentially. - There are enough resources available to do parallel execution. |
| @supports-site-context | Optional | Determines if the step type is intended to be used in site context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with one or more sites as scope. If false, steps of this type can't be used for flows with one or more sites as scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| @supports-organization-context | Optional | Determines if the step type is intended to be used in organization context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with organization scope. If false, steps of this type can't be used for flows with organization scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| description | Optional | The description of the step type. Must not contain leading or trailing whitespace or more than 256 characters. |
| pipeline | ❗Required❗ | Pipeline to execute for the step. Format must be <PipelineName>-<StartNodeName>. For example: Search-Show |
| parameters | Optional | Parameters for the step, which the user configures in Business Manager. Contains one parameter array-element with one or more parameter objects. |
| status-codes | Optional | Defines the meta data for status codes returned for the step. Contains one status array-element with one or more status objects. |
| Attribute | Required or Optional | Description |
|---|---|---|
| @name | ❗Required❗ | The name of the parameter. Must not contain leading or trailing whitespace. |
| @type | ❗Required❗ | Indicates the data type of the parameter value. Must not contain leading or trailing whitespace. Must have a value of boolean, string, long, double, datetime-string, date-string, or time-string. |
| @required | Optional | Describes if the parameter is required or optional. Must have a value true or false. Default is true. If true, the parameter is required. If false, the parameter is optional. |
| @trim | Optional | Specifies whether the leading and trailing whitespace for the parameter value entered in Business Manager is removed before the value is validated. Must not contain leading or trailing whitespace. Must have a value true or false. Default is true. |
| @target-type | Optional | The type to which the parameter value is converted, either long or date, after the user enters a value in Business Manager. Can be present only when @type is datetime-string, date-string, or time-string. If @target-type is not defined, date is the default value. |
| description | ❗Required❗ | The description of the parameter. Must not contain leading or trailing whitespace or more than 256 characters. |
| default-value | Optional | Default when no value is entered for the parameter. Must have a valid data type value for the parameter ( boolean, string, long, double, datetime-string, date-string, or time-string). If the parameter is assigned enum-values, the default-value must match one of the list of enum values. |
| pattern | Optional | Only for parameter types: string. Regular expression that defines the allowed values for the parameter. |
| min-length | Optional | Only for parameter types: string.Minimum length of the string value for the parameter. Must be at least 1 and less than max-length. If not defined, there are no restrictions other than the general restriction that parameter values cannot exceed 1000 characters. |
| max-length | Optional | Only for parameter types: string. Maximum length of a string value for this parameter. A string entered in Business Manager is validated against this restriction. Must be at least 1 and greater than or equal to min-length. If not defined, string length is not restricted other than by the general requirement that parameter values cannot exceed 1000 characters. |
| min-value | Optional | Only for parameter types: long, double, datetime-string, time-string.Minimum value for the parameter. Must not exceed the max-value. The minimum valid value for a long is -9223372036854775808 and for a double is 4.9E-324. If you do not explicitly define a min-value, those values are considered the min-values for long and double parameters. For datetime-string or time-string parameters, if you do not define a min-value, there are no restrictions other than the general restriction that parameter values cannot exceed 1000 characters. |
| max-value | Optional | Only for parameter types: long, double, datetime-string, time-string. Maximum numerical value for the parameter. Must be greater than or equal to min-value. The maximum valid value for a long is 9223372036854775807 and for a double is 1.7976931348623157E308. If you do not explicitly define a max-value, those values are considered the max-values for long and double parameters. For datetime-string or time-string parameters, if not defined, there are no restrictions other than the general restriction that parameter values cannot exceed 1000 characters. |
| enum-values | Optional | Allowed values for the parameter. The value entered by the user in Business Manager is validated against this list. Must not contain leading or trailing white spaces. The object contains a value array-object with the allowed values as strings. "enum-values": { "value": [ "One", "Two", "Three" ] } |
| Attribute | Required or Optional | Description |
|---|---|---|
| @code | ❗Required❗ | The status code, the step type can return. Must not contain leading or trailing white space. |
| description | Optional | The description of the status code. Not shown in Business Manager. Must not exceed 4000 characters. |
The following table shows the results of possible settings of the supports-organization-context and supports-site-context elements in the steptypes.json file.
| supports-organization-context | supports-site-context | Status |
|---|---|---|
| TRUE | TRUE | Invalid configuration. |
| TRUE | FALSE | Job is executable only if other steps in the same flow can be executed for the entire organization. |
| FALSE | TRUE | Job is executable only if other steps in the same flow can be executed for one or more sites. |
| FALSE | FALSE | Invalid configuration. |
This example shows a steptypes.json file that describes three custom steps: a step type defined using a task-oriented CommonJS module, a step type defined using a legacy pipeline, and a step type defined using a chunk-oriented CommonJS module.
Custom Step Types can be registered in the system with a steptypes.xml file. Within this file, you describe the step types of the cartridge, their parameters, and status codes.
The file must be named steptypes.xml and placed in a custom cartridge in the root folder. You can only have one steptypes.xml file per cartridge.
The steptypes.xml file requires a specific XML syntax. If a steptypes.xml file contains errors, the errors are logged and the custom step types are not registered. The system then loads step types from the steptypes.xml files of other cartridges.
A typical error message for an invalid steptypes.xml file is:
Invalid step [Step1]! Type with id [custom.MyCustomStep1] is unknown!.
You can have either a steptypes.xml or a steptypes.json file only! It is not supported to have both files in the cartridge.
The usage of steptypes.xml file should avoided. Please use the steptypes.json file instead and migrate the steptypes.xml to steptypes.json.
| Attribute | Required or Optional | Description |
|---|---|---|
| step-types | ❗Required❗ | The root object of the steptypes.json file. |
| script-module-step | Basically optional, but required if no chunk-script-module-step or pipeline-step is defined. | Defines one or more task-oriented script step types. |
| chunk-script-module-step | Basically optional, but required if no script-module-step or pipeline-step is defined. | Defines one or more chunk-oriented script step types. |
| pipeline-step | Basically optional, but required if no script-module-step or chunk-script-module-step is defined. | Defines one or more pipeline-based step types. |
| Attribute | Required or Optional | Description |
|---|---|---|
| @type-id | ❗Required❗ | Identifies the step type. This is the name that users see in Business Manager, so make it descriptive. Must begin with custom.. Must not contain leading or trailing white space or more than 100 characters. Must be unique within the job definition. You can't register multiple steps with the same @type-id in different cartridges.The @type-id is validated as unique by parsing the steptypes files from all cartridges on the cartridge path. If there is a step with the same @type-id, the step isn't loaded. The @type-id value can't be the same as any system step, for example, ExecutePipeline or IncludeStepsFromJob. |
| @supports-parallel-execution | Optional | Determines if the step type can be used in parallel with itself or other step types. Must have value true or false. Default is true. If false, split flows that contain steps of this type are always executed sequentially and never in parallel. If true, split flows that contain steps of this type are executed in parallel, as long as: - The split flows don't contain steps with supports-parallel-execution = false. - The split flows are not configured to be executed sequentially. - There are enough resources available to do parallel execution. |
| @supports-site-context | Optional | Determines if the step type is intended to be used in site context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with one or more sites as scope. If false, steps of this type can't be used for flows with one or more sites as scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| @supports-organization-context | Optional | Determines if the step type is intended to be used in organization context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with organization scope. If false, steps of this type can't be used for flows with organization scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| sub-tags: | ||
| description | Optional | The description of the step type. Not shown in Business Manager. Must not exceed 4000 characters. |
| module | ❗Required❗ | Path to the script module to be executed. Must not contain leading or trailing white space. |
| function | ❗Required❗ | The function of the script module to execute. Must not contain leading or trailing white space. If not defined, the script module must export a function named execute. |
| transactional | Optional | Indicates if the module requires a database transaction. Must have value true or false. Default is false. If this value is set to true, the job step executes as a single, potentially very large, transaction. To avoid a negative impact on system performance and allow more granular transaction control, keep the default setting of false. Implement transaction handling within the job step using the dw.system.Transaction API. |
| timeout-in-seconds | Optional | Sets the timeout in seconds for the script module's function. Must be an integer greater than zero. There is no default timeout, but setting a limit is recommended. |
| parameters | Optional | Parameters for the step, which the user configures in Business Manager. Contains one or more parameter elements. |
| status-codes | Optional | Defines the meta data for status codes returned for the step. Contains one or more status elements. |
| Attribute | Required or Optional | Description |
|---|---|---|
| @type-id | ❗Required❗ | Identifies the step type. This is the name that users see in Business Manager, so make it descriptive. Must begin with custom.. Must not contain leading or trailing white space or more than 100 characters. Must be unique within the job definition. You can't register multiple steps with the same @type-id in different cartridges.The @type-id is validated as unique by parsing the steptypes files from all cartridges on the cartridge path. If there is a step with the same @type-id, the step isn't loaded. The @type-id value can't be the same as any system step, for example, ExecutePipeline or IncludeStepsFromJob. |
| @supports-parallel-execution | Optional | Determines if the step type can be used in parallel with itself or other step types. Must have value true or false. Default is true. If false, split flows that contain steps of this type are always executed sequentially and never in parallel. If true, split flows that contain steps of this type are executed in parallel, as long as: - The split flows don't contain steps with supports-parallel-execution = false. - The split flows are not configured to be executed sequentially. - There are enough resources available to do parallel execution. |
| @supports-site-context | Optional | Determines if the step type is intended to be used in site context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with one or more sites as scope. If false, steps of this type can't be used for flows with one or more sites as scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| @supports-organization-context | Optional | Determines if the step type is intended to be used in organization context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with organization scope. If false, steps of this type can't be used for flows with organization scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| sub-tags: | ||
| description | Optional | The description of the step type. Not shown in Business Manager. Must not exceed 4000 characters. |
| module | ❗Required❗ | Path to the script module to be executed. Must not contain leading or trailing white space. |
| before-step-function | Optional | References the function of the chunk-oriented script module to be executed before the step begins. If not defined, no before-step logic is used. |
| total-count-function | Optional | References the function to be executed to determine the total number of items. If not defined, no total count is used. |
| before-chunk-function | Optional | References the function of the chunk-oriented script module to be executed before a chunk begins. If not defined, no before-chunk logic is used. |
| read-function | Optional | Read function of the chunk script. If not defined, script must export a function named read. |
| process-function | Optional | Process function of the chunk script. If not defined, script must export a function named process. |
| write-function | Optional | References the write function of the chunk script. If not defined, script must export a function named write. |
| after-chunk-function | Optional | References the function of the chunk-oriented script module to be executed after a chunk has finished. If not defined, no after-chunk logic is used. |
| after-step-function | Optional | References the function of the chunk-oriented script module to be executed after the step has finished. If not defined, no after-step logic is used. |
| chunk-size | ❗Required❗ | Defines the size of one chunk. Must be a numeric value greater than zero. Must not contain leading or trailing white space. |
| transactional | Optional | Indicates if the module requires a database transaction. Must have value true or false. Default is false. If this value is set to true, the job step executes as a single, potentially very large, transaction. To avoid a negative impact on system performance and allow more granular transaction control, keep the default setting of false. Implement transaction handling within the job step using the dw.system.Transaction API. |
| parameters | Optional | Parameters for the step, which the user configures in Business Manager. Contains one or more parameter elements. |
| status-codes | Optional | Defines the meta data for status codes returned for the step. Contains one or more status elements. |
| Attribute | Required or Optional | Description |
|---|---|---|
| @type-id | ❗Required❗ | Identifies the step type. This is the name that users see in Business Manager, so make it descriptive. Must begin with custom.. Must not contain leading or trailing white space or more than 100 characters. Must be unique within the job definition. You can't register multiple steps with the same @type-id in different cartridges.The @type-id is validated as unique by parsing the steptypes files from all cartridges on the cartridge path. If there is a step with the same @type-id, the step isn't loaded. The @type-id value can't be the same as any system step, for example, ExecutePipeline or IncludeStepsFromJob. |
| @supports-parallel-execution | Optional | Determines if the step type can be used in parallel with itself or other step types. Must have value true or false. Default is true. If false, split flows that contain steps of this type are always executed sequentially and never in parallel. If true, split flows that contain steps of this type are executed in parallel, as long as: - The split flows don't contain steps with supports-parallel-execution = false. - The split flows are not configured to be executed sequentially. - There are enough resources available to do parallel execution. |
| @supports-site-context | Optional | Determines if the step type is intended to be used in site context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with one or more sites as scope. If false, steps of this type can't be used for flows with one or more sites as scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| @supports-organization-context | Optional | Determines if the step type is intended to be used in organization context. Must have value true or false. Default is true. If true, steps of this type can be used for flows with organization scope. If false, steps of this type can't be used for flows with organization scope. @supports-organization-context and @supports-site-context cannot both have the same true or false setting. If @supports-site-context is false, @supports-organization-context must be true and vice versa. |
| sub-tags: | ||
| description | Optional | The description of the step type. Must not contain leading or trailing whitespace or more than 256 characters. |
| pipeline | ❗Required❗ | Pipeline to execute for the step. Format must be <PipelineName>-<StartNodeName>. For example: Search-Show. |
| parameters | Optional | Parameters for the step, which the user configures in Business Manager. Contains one or more parameter elements. |
| status-codes | Optional | Defines the meta data for status codes returned for the step. Contains one or more status elements. |
| Attribute | Required or Optional | Description |
|---|---|---|
| name | Required | The name of the parameter. Must not contain leading or trailing whitespace. |
| type | Required | Indicates the data type of the parameter value. Must not contain leading or trailing whitespace. Must have a value of boolean, string, long, double, datetime-string, date-string, or time-string. |
| required | Optional | Describes if the parameter is required or optional. Must have a value true or false. Default is true. If true, the parameter is required. If false, the parameter is optional. |
| trim | Optional | Specifies whether leading and trailing whitespace for the parameter value entered in Business Manager is removed before the value is validated. Must not contain leading or trailing whitespace. Must have value true or false. Default is true. |
| target-type | Optional | The type to which the parameter value is converted, either long or date, after the user enters a value in Business Manager. Can be present only when type is datetime-string, date-string, or time-string. If target-type is not defined, date is the default value. |
| description | Required | The description of the parameter. Must not contain leading or trailing whitespace or more than 256 characters. |
| default-value | Optional | Default when no value is entered for the parameter. Must have a valid data type value for the parameter (boolean, string, long, double, datetime-string, date-string, or time-string). If the parameter is assigned enum-values, the default-value must match one of the list of enum values. |
| pattern | Optional | Only for parameter types: string. Regular expression that defines the allowed values for the parameter. |
| min-length | Optional | Only for parameter types: string. Minimum length of the string value for the parameter. Must be at least 1 and less than max-length. If not defined, there are no restrictions other than the general restriction that parameter values cannot exceed 1000 characters. |
| max-length | Optional | Only for parameter types: string. Maximum length of a string value for this parameter. String entered in Business Manager is validated against this restriction. Must be at least 1 and greater than or equal to min-length. If not defined, string length is not restricted other than by the general requirement that parameter values cannot exceed 1000 characters. |
| min-value | Optional | Only for parameter types: long, double, datetime-string, time-string. Minimum value for the parameter. Must not exceed the max-value. The minimum valid value for a long is -9223372036854775808 and for a double is 4.9E-324. If you do not explicitly define a min-value, those values are considered the min-values for long and double parameters. For datetime-string or time-string parameters, if you do not define a min-value, there are no restrictions other than the general restriction that parameter values cannot exceed 1000 characters. |
| max-value | Optional | Only for parameter types: long, double, datetime-string, time-string. Maximum numerical value for the parameter. Must be greater than or equal to min-value. The maximum valid value for a long is 9223372036854775807 and for a double is 1.7976931348623157E308. If you do not explicitly define a max-value, those values are considered the max-values for long and double parameters. For datetime-string or time-string parameters, if not defined, there are no restrictions other than the general restriction that parameter values cannot exceed 1000 characters. |
| enum-values | Optional | Allowed values for the parameter. The value entered by the user in Business Manager is validated against this list. Must not contain leading or trailing white spaces. The object contains one or more value elements representing the allowed values. <enum-values> |
| Attribute | Required or Optional | Description |
|---|---|---|
| code | ❗Required❗ | The status code, the step type can return. Must not contain leading or trailing white space. |
| sub-tags: | ||
| description | Optional | The description of the status code. Not shown in Business Manager. Must not exceed 4000 characters. |
The following table shows the results of possible settings of the supports-organization-context and supports-site-context attributes in the steptypes.xml file.
| supports-organization-context | supports-site-context | Status |
|---|---|---|
| TRUE | TRUE | Invalid configuration. |
| TRUE | FALSE | Job is executable only if other steps in the same flow can be executed for the entire organization. |
| FALSE | TRUE | Job is executable only if other steps in the same flow can be executed for one or more sites. |
| FALSE | FALSE | Invalid configuration. |
This example shows a steptypes.xml file that describes three custom steps: a step type defined using a task-oriented CommonJS module, a step type defined using a legacy pipeline, and a step type defined using a chunk-oriented CommonJS module.