As a Salesforce developer or architect, moving towards source-based release management and Continuous Integration (CI) is key to improving the speed, reliability and quality of your Salesforce deployments.

Frequently deploying small changes is very valuable, but it can be challenging to automate when dealing with a large number of unpackaged sources in the source control system. This post will show you how to use a delta generation tool to significantly speed up and further automate the deployment process.

Salesforce DX, DevOps, and the need for incremental deployments for unpackaged sources

In the age of DevOps, we see a major trend towards Continuous Integration (CI) and Continuous Deployment (CD). More and more teams adopt development workflows based on git as their version control system and single source of truth. This move has also never been easier since the introduction of the Salesforce DX toolset.

Salesforce DX also introduced new solutions to move to modular development and a package-based approach, thanks to second-generation packaging and the new org-dependent unlocked packages (Beta).

Unlocked packages are a great way to organize your code and teams when working on custom apps, but they’re not necessarily the best fit for every project.

It’s not always easy to untangle the org and break it down into smaller packages when the initial implementations of Sales or Service Cloud used the production org, or a single metadata folder as the single source of truth (sometimes known as the “Happy Soup” approach, as opposed to the Package Development approach).

When used in combination with source control, the unpackaged development model usually implies that all sources lie in the same git repository. This approach is convenient in terms of dependency management: no need to manage dependencies, since everything is in one place.

However, grouping everything in the same repository comes at a cost: when moving to CI/CD you may end up re-deploying the entire repository every time. This approach guarantees that you didn’t skip any added or modified metadata since the last deployment, but it leaves room for optimizations:

• Full deployments are slow: Unless you select specific metadata types, sfdx force:source:deploy deploys all of the sources to the target org. As your repository grows, it becomes more time consuming to deploy all components every time, even when just a few of them changed. This impacts the speed to deploy small hot fixes, and causes bottlenecks in CI builds, consuming precious minutes.
• Full deployments won’t destroy metadata that has been removed from the repo: in order to delete metadata components from an org, you need to explicitly list those components (in a destructive changes package). Wouldn’t it be nice to simply remove the component and its references in the repository?

Well, if you are (or anticipate to be) in this situation, read along!

Introducing SFDX-git-delta

SFDX-git-delta (a.k.a. SGD) is a Salesforce CLI plugin that allows incremental deployments from Salesforce source format tracked in a git repository.

SGD is an unofficial and unsupported CLI plugin. Always perform your tests in a non-production environment, and make sure you have a proper backup and contingency plan if you encounter an unexpected behaviour with SGD.

Who needs SGD? What problems does it solve?

SGD is here to help you with typical medium and large Salesforce implementations that meet the following requirements:

• Your project uses a git repository as the source of truth.
• The repository holds metadata in the Source (DX) format.
• The metadata is not packaged (in other words, your repository contains all the unmanaged metadata of the project).

If your project meets these conditions, deploying the sources to an org probably looks like running the force:source:deploy command against your repo.

You may even have automated this step with a CI/CD pipeline running on your favorite CI tool (Jenkins, GitLab CI, Bitbucket Pipelines, GitHub Actions, Azure DevOps, Circle CI…). The great news is that SGD is a Salesforce CLI plugin, so it plays really nice with CI/CD, to even further automate your release management.

Using force:source:deploy to deploy of the whole repository works fine but it has some drawbacks, as we explained in the introduction: full deployments are slow, and they won’t destroy metadata that has been removed from the repo.
SGD is here to help you optimize your deployment process by allowing you to deploy just what you need to.

What does SGD actually do?

SGD compares the metadata in your repository between different states in time (commits). You can think of it as a sort of “git diff” that produces an output “Salesforce sources-ready” that can immediately be deployed to your target org.

SGD analyses how the repository changed between two commits, and generates:

• a package.xml file with what has changed between commits and needs to be deployed.
• a destructiveChanges.xml file with what has been removed.

So if you are deploying the sources on every commit, instead of deploying the full repo, you can use those files to perform a much faster “delta” deployment:

You could even use the --generate-delta parameter to generate all the files in source format in the output folder so that it can be deployed using the sfdx force:source:deploy --sourcepath command.

Now let’s see how it works in practice.

Walkthrough use case

Prerequisites

SGD is distributed as a plugin for the Salesforce CLI, meaning that you can install it by running sfdx plugins:install sfdx-git-delta

Note: You first need to have the the Salesforce CLI and Node v14.6.0 (or above) installed locally. Take a look at the README for the detailed installation instructions. Here’s an example of a Docker image that includes the plugin.

First, let’s commit some changes to our repo

Let’s consider the following scenario:
“The CI pipeline deploys the sources to Production anytime there is a new commit in the master branch.”

In this example, the latest commit to master is composed of:

In this situation, we expect the CI pipeline to:

1. Incrementally deploy three classes to Production (no matter how much metadata is present in the project): TriggerHandler, TriggerHandler_Test and TestDataFactory.
2. Delete one class from Production: AnotherTriggerFramework

So let’s do it!

Run the sfdx sgd:source:delta command

From the project repository folder, the CI pipeline runs the following command:

Which means: analyze the difference between HEAD (latest commit) and HEAD^ (previous commit), from the current source folder, and output the result in the same folder.

The sgd command produces two useful artifacts:

1) A package.xml file, inside a package folder. This file contains the metadata that has been added and changed, and that needs to be deployed in the target org.

Content of the package.xml file in our scenario:

Note for advanced use cases: If needed, we also could have generated a copy of the force-app folder with only the added and changed metadata, by using the --generate-delta flag. You can run sgd --help to review all the command parameters.

2) A destructiveChanges.xml file, inside a destructiveChanges folder. This file contains the metadata that has been removed or renamed, and that needs to be deleted from the target org.

Content of the destructiveChanges.xml file in our scenario:

You may note that there is also another package.xml file within our destructiveChanges folder. This empty file is required by the Salesforce CLI for the deployment of the destructive changes.

Deploy only the added/modified metadata

The CI pipeline uses the package/package.xml file to deploy only this subset of metadata:

Delete the removed metadata

The CI pipeline uses the destructiveChanges/destructiveChanges.xml file to deploy the corresponding destructive change:

Tip: You may want to add the --ignorewarnings flag to ensure success of the command, in case some of the components have already been deleted in the target org.

And voilà! 🥳

Wrap up

Welcome to the world of delta deployments for unpackaged sources!

SGD allows you to generate delta deliverables from Salesforce (unpackaged) metadata source format contained in a git repository to speed up deployment and streamline pipelines.

By comparing the metadata changes between two states of the repo, SGD allows you to:

• Deploy faster, by identifying and pushing only the metadata that has changed since a reference commit.
• Automate destructive deployments, by listing the deleted (or renamed) metadata in a destructiveChanges.xml file.

SGD is open source and is also available via npm. Contributions are welcome, feel free to raise issues and let us know how you use it!

Once installed, run sfdx sgd:source:delta -h to explore the different parameters, you can also follow the README instructions for a guided start.

About the authors

Sébastien Colladon is a Technical Architect at Salesforce Professional Services in Paris, France. He works closely with customers to help them deploy Salesforce in their IT Systems and boost their business. He enjoys learning about technology and working with others as a team. Check his GitHub projects @scolladon.

Mehdi Cherfaoui is a Technical Architect at Salesforce Professional Services in Lyon, France. Passionate mountaineer and rock climber on weekends, he loves helping Salesforce customers to solve their challenges on the platform (especially when it comes to Identity and DevOps!) during the week.

Philippe Ozil is a Principal Developer Advocate at Salesforce where he focuses on the Salesforce Platform. He helped the Tableau team create the Tableau Visualization Lightning web component. He writes technical content and speaks frequently at conferences. He is a full stack developer and enjoys working on robotics and VR projects.
Follow him on Twitter @PhilippeOzil or check his GitHub projects @pozil.