Write and Validate Custom Scripts

Write custom scripts for batch data transforms in a standard Python environment by using your preferred IDE and validate them locally against a sandbox.

Edition Table
Available in: Developer, Enterprise, Performance, and Unlimited Editions. See Data 360 edition availability.

Permission Sets Needed
To write custom scripts and validate them locally against a sandbox:	Permission set: Data Cloud Architect

Prerequisites

Set up your Data 360 environment.
1. Verify that you have access to a Data 360 sandbox with appropriate permission sets. If you don’t have a sandbox, create one. See Create a Data 360 Sandbox.
2. Enable Code Extension in Data Cloud Setup using the Feature Manager. See Enable Data 360 Features in Feature Manager.
3. Verify that you ingested data into your sandbox to create the DLOs or DMOs that your script references.
Set up Salesforce CLI for code extension and create a script package. For more information, see Set Up Salesforce CLI for Code Extension.

Best Practices

Work exclusively with either DLOs or DMOs in a single script.
Add any additional Python dependencies to requirements.txt.
Use only pip-installable dependencies. Libraries that require OS-level installations or system configurations don’t work in Data 360.
Use clear function names and comments, and handle errors gracefully.
Test your logic frequently in your selected environment.
Use logging for debugging and monitoring.

Field Access in Custom Scripts

Use exact API names for Data 360 object fields. Field names follow the FieldName__c pattern.

Data Writing Modes

Use one of these write modes when writing to DLOs or DMOs in your script.

Write Mode	Description	Example
APPEND	Adds new records to the target object without affecting existing data.	`client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.APPEND)`
OVERWRITE	Replaces all existing data in the target object with new data.	`client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.OVERWRITE)`
MERGE	Updates existing records and adds new records based on key fields.	`client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.MERGE)`

Authoring in Standard Python Virtual Environment

Go to your code extension package folder.
1cd </path/to/your/package>

Create and activate a Python virtual environment in the package folder.

macOS and Linux

1python3.11 -m venv venv
2source venv/bin/activate

Windows (PowerShell)

1py -3.11 -m venv venv
2.\venv\Scripts\Activate.ps1

Windows (Command Prompt)

1py -3.11 -m venv venv
2venv\Scripts\activate.bat

Add required dependencies to requirements.txt.
1# examples 2pandas>=1.5.0 3numpy>=1.21.0

Install the dependencies.

macOS and Linux

1pip install -r requirements.txt

Windows

1python -m pip install -r requirements.txt

To add new dependencies during development, always add them to requirements.txt and then install by using this command. This process ensures that all dependencies are included in your deployment package.

Note

Write your script in payload/entrypoint.py to reference the ingested data.
Test your script.
1sf data-code-extension script run --entrypoint <path_to_entrypoint> --target-org <org_alias>
- --entrypoint <path_to_entrypoint> (required): Path to your script entry point (typically ./payload/entrypoint.py in the default package layout).
- --target-org <org_alias> (required): The alias against which you’re testing your script.
Example:
1sf data-code-extension script run --entrypoint ./payload/entrypoint.py --target-org myorg
Optionally, control the number of rows returned during local testing.

By default, Data 360 queries return 1,000 rows when you test locally. To validate against a larger dataset, raise the row limit. To iterate faster on a smaller sample, lower it. Set the row limit in entrypoint.py before your script reads any data.
1. Import config in your entrypoint.py.
  1from datacustomcode.config import config
2. Set the row limit to the number of rows you need. For example, to return 5,000 rows:
  1config.reader_config.options["default_row_limit"] = 5000
  To remove the limit entirely and fetch all records, set the value to None:
  1config.reader_config.options["default_row_limit"] = None
If you package and upload your entrypoint with this setting, the package includes the row limit, which then applies when run in Data 360. Remove or reset the row limit before packaging for deployment.
Important
Update the Data 360 configuration file after changing your script.
1sf data-code-extension script scan --entrypoint ./payload/entrypoint.py
The sf data-code-extension script scan command doesn’t work as intended for DMO-to-DMO transforms. You must manually craft the config.json file. See Configure config.json for DMO-to-DMO Transforms.
Important

Next Step

Deploy a Custom Script to Data 360 Sandbox by Using UI.