Write and Validate Custom Scripts

Write custom scripts for batch data transforms in a standard Python environment by using your preferred IDE and validate them locally against a sandbox.

Edition Table
Available in: Developer, Enterprise, Performance, and Unlimited Editions. See Data 360 edition availability.
Permission Sets Needed
To write custom scripts and validate them locally against a sandbox:Permission set:
  • Data Cloud Architect

  1. Set up your Data 360 environment.

    1. Make sure that you have access to a Data 360 sandbox with appropriate permission sets. If you don’t have a sandbox, create one. See Create a Data 360 Sandbox.

    2. Enable Code Extension in Data Cloud Setup by using the Feature Manager. See Enable Data 360 Features in Feature Manager.

    3. Make sure that you ingested data into your sandbox to create the DLOs or DMOs that your script references.

  2. Set up Salesforce CLI for code extension and create a script package. See Set Up Salesforce CLI for Code Extension.

  • Work exclusively with either DLOs or DMOs in a single script.
  • Add any additional Python dependencies to requirements.txt.
  • Use only pip-installable dependencies. Libraries that require OS-level installations or system configurations don’t work in Data 360.
  • Use clear function names and comments, and handle errors gracefully.
  • Test your logic frequently in your selected environment.
  • Use logging for debugging and monitoring.

Use exact API names for Data 360 object fields. Field names follow the FieldName__c pattern.

Use one of these write modes when writing to DLOs or DMOs in your script.

Write ModeDescriptionExample
APPENDAdds new records to the target object without affecting existing data.client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.APPEND)
OVERWRITEReplaces all existing data in the target object with new data.client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.OVERWRITE)
MERGEUpdates existing records and adds new records based on key fields.client.write_to_dlo("Target_DLO__dll", df, write_mode=WriteMode.MERGE)

  1. Go to your code extension package folder.

  2. Create and activate a Python virtual environment in the package folder.

    macOS and Linux

    Windows (PowerShell)

    Windows (Command Prompt)

  3. Add required dependencies to requirements.txt.

  4. Install the dependencies.

    macOS and Linux

    Windows

To add new dependencies during development, always add them to requirements.txt and then install by using this command. This process ensures that all dependencies are included in your deployment package.

  1. Write your script in payload/entrypoint.py to reference the ingested data.

  2. Test your script.

    • --entrypoint <path_to_entrypoint> (required): Path to your script entry point (typically ./payload/entrypoint.py in the default package layout).
    • --target-org <org_alias> (required): The alias against which you’re testing your script.

    Example:

  3. Update the Data 360 configuration file after changing your script.

    The sf data-code-extension script scan command doesn’t work as intended for DMO-to-DMO transforms. You must manually craft the config.json file. See Configure config.json for DMO-to-DMO Transforms.

Deploy a Custom Script to Data 360 Sandbox by Using UI.