Working with Modular Development and Unlocked Packages: Part 4

This is the fourth installment in a series exploring how to begin working with your apps in modular pieces, incorporating packages into your app development lifecycle, and what packaging may mean for your team’s change management and release processes. Over the course of this series, we’ll talk about:

  • Part 1: What even is a package, anyway? How can you start to experiment with segmenting your org?
  • Part 2: How can you start to organize metadata from an app, let alone an entire org, into packages? How do you tackle organizing your metadata and projects in source control?
  • Part 3: What do these changes mean for app builder workflows? What will happen if I install an unlocked package into my production org today?
  • Part 4: How can you define a successful Git branching strategy that works best for most team sizes? How, when and where should packaging be added to your continuous deployment?

After you have untangled your application and set up your package directories and initial package version, it’s time to work on a strategy for managing your package versions and active development projects in your source control systems. Having a good and practical Git strategy will be crucial for your Continuous Integration and Continuous Deployment workflows. This blog post covers how to create such a strategy.

An intro to Git branching

Source control (and with that, Git) has been around for a long time in the Salesforce world. But for those who haven’t worked with Git and the Salesforce Platform yet, let’s look at briefly at a widely adapted Git branching strategy.

(Photo source)

In a nutshell:

  • All active development work happens in the development branch.
  • Creating new features or fixing bugs happens in dedicated feature or hotfix branches. Those branches inherit from development.
  • After the feature/bug has been successfully tested, the code gets merged back into the development branch.
  • Code that gets shipped into production gets merged from development into master. As versioning is important, the master branch gets tagged with a version number.

While that itself sounds simple, it can get a bit complicated, especially when you have multiple branches running at the same time (which is not uncommon in software development, especially when you work in a team).

Here’s a small real-world example from a code race that I did some years ago.

Blue is master, green is develop, the other are feature and hotfix branches.

Git branching strategy

Building upon that branching strategy and your own company’s requirements, you may have (or should) extend the model. Before you start designing your Git strategy you should have answers to the following questions:

  • At what stage in the process should automated tests run?
  • When should unlocked package versions be created, tested, and cleaned up?
  • How should promotion (aka installation) of package versions to environments like QA sandboxes or even production be executed?

You may answer some or all of those questions with “I want that on every commit.” But then you’ll experience the delay that it takes to create a new package version before it can be installed. Waiting eight minutes before you get feedback if everything is okay from your CI system can feel like a long time, even more if you commit multiple times a day. Last but not least, there are daily limits for scratch org creation and package version creation request, so you don’t want to have an overly aggressive package version strategy, as it will clutter your environment and eat up your available resources.

What if you want to test package version creation and installation “only” on every Pull Request? On what branches? Develop, master, all feature branches? There are many options. You want to find out if something is broken or may be broken as early as possible in your development workflow. However, you also don’t want to get slowed down by the process.

For our Easy Spaces app we defined the following naming conventions for our Git branches:

  • feature/package-name/* – a feature (and bug fix) branch for an individual package
  • packaging/package-name/* – a per package branch that tests package creation and installation for an individual package
  • packaging – this branch reflects the code change for partial sandbox or UAT sandbox
  • develop – this branch reflects pre-production, for example a full-copy sandbox
  • master – this branch is what reflects production

For our sample application, we settled with what are standard names for Git branches, like develop or master. You should use names that make most sense to you and your organization. For example, packaging could be partial-sandbox and develop could be full-sandbox. There’s no limit in how you can customize. Just keep in mind that it needs to be manageable.

We also defined pre-conditions that should be met before a branch gets merged or code gets committed as well as which specific tasks should run. Here’s an excerpt, based on the used CircleCI workflow (note that this graphic doesn’t include branch filtering yet):

feature/package-name/* branches – after every commit

  • Deploy all package folders using sfdx force:source:push.
  • Run all tests, like Apex and/or Lightning Testing Service.
  • Don’t create any package versions.
  • Delete scratch org.

packaging/package-name/* branches – on every pull request coming in from a feature/package-name branch

  • Create a new package version for package-name.
  • Deploy in package directory order (based on sfdx-project.json) to sandbox org. All package directories will be deployed as packages.
  • Run all tests, like Apex and/or Lightning Testing Service.
  • Delete scratch org.

packaging branch – on every pull request coming in from a packaging/package-name branch

  • Deploy in package directory order (based on sfdx-project.json) to sandbox org. All package directories will be deployed as packages.
  • Run all tests, like Apex and/or Lightning Testing Service.

develop branch – on every pull request coming in from the packaging branch

  • Deploy in package directory order (based on sfdx-project.json) to full-copy sandbox. All package directories will be deployed as packages.
  • Run all tests, like Apex and/or Lightning Testing Service.

master branch

  • No actions (we’ll explain later in this post)

Some of these decisions can and will be different for your organization. For example, you may want to run package creation also at a feature level if you want to make sure that your development team (or yourself) doesn’t introduce unpackageable features. For us, we decided that developers consult the documentation around supported Metadata types before they start working on things. Relevant metadata which is yet not packageable should be moved into separate folders, like we did with the branding images for Salesforce Branding & Theming in the Easy Spaces sample app.

Branches and tasks in detail

Let’s have a more detailed look at the why and how we made the aforementioned decisions for branches and their tasks.

feature/package-name/* branches
For these branches we decided to push all source using sfdx force:source:push and wouldn’t use any packages. This decision is based on the following reasons:

  • We don’t want to create package versions for every “simple” commit. At this branch level, working on feature code has higher priority than package testing.
  • We use dependencies for packages. Which means you cannot, for example, use sfdx force:source:push on the es-base-objects folder and then install es-base-code via package installation. This will fail because the es-base-code package has the dependency of an installed es-base-objects package.

If you think: “Hey, I can just remove the dependencies in the sfdx-project.json, and then do a push/install package mix…” – that won’t work. The dependencies are part of the created package version, so any manipulation in the sfdx-project.json won’t have an effect. Been there, done that.

Once we’re done with the feature we’re also modifying the package version creation command in the CI configuration with data like the version number, description and so on. This step is important because when we move the code via a PR into the ancestral packaging/package-name/ branch, the package version creation process kicks off.

packaging/package-name/* branches
After a feature is completely developed, we move it to its packaging branch. This is also where the focus goes from “a feature works” to “a feature works AND gets packaged”.

You can see this branch as the intermediary between active development and preparation for first sandbox testing. In your organization you may make the decision to skip this branch and instead let someone manually create the package version. That’s totally legit. Choose what works best for you.

We’ve chosen our approach because it allows us to see all involved steps and their output within the UI of the chosen CI solution. For this we’re executing a script that can be used via CI and can be also manually invoked from the developers workstation.

packaging branch
Once we know that the new feature works (thanks to testing) and that we have a new package, we want to install it in a sandbox. That can be a partial sandbox or a UAT sandbox for example. We’ve settled for our use case to name the branch “packaging,” but you can (and should!) use a name that reflects your stage in the process, i.e. “uat-sandbox”.

It’s noteworthy that we focus on installation of the packages. For that, we fetch all to be used versions for the packages via a bash script and deploy them. That way the sandbox gets hit with the latest code across all folders specified as package directories in the sfdx-project.json configuration.

develop branch
The develop branch reflects what could be for you a full-copy sandbox. The steps are the same as in the packaging branch with the focus on package installation. The steps don’t change. We only move to the next stage of testing environment before we move to production.

master branch
Now, if you remember, there’s still the master branch — with no actions. There’s also still the outstanding task of bringing the new package version(s) to production. This is the stage where we leave the path of continuous deployment and do some manual steps.

At this point some decisions need to be made. You have to define which package version is the one that you want to promote to production using the sfdx force:package:version:promote command. You may want to say “the latest,” but as development doesn’t stop in the world of continuous development, there maybe already the next version in testing in the packaging branch, depending on your strategy. So you have to manually pick a version and that’s what we recommend for now. Make a manual and educated decision because that will affect production and with that 5 or 5,000 users. After the promotion and installation of the package, submit the code to the master branch.

What’s always recommended is that you make use of tags on the master branch when you release a new version. That way you have your source of truth aligned with what is in production. Plus, you can go back anytime to see — and compare — changes between your custom application releases.

You remember the code race image? Here’s another one with multiple simultaneous branches going on. You want to make sure that always the correct one becomes the master.

Your next steps

Depending on your organizational requirements your Git branching strategy may be very similar or different to our approach. With the currently available capabilities of unlocked packages, not everything may work right away in that approach. It can even be different depending on the type of application or the level of modularization that you want or can introduce with unlocked packages.

What is most important is that you start thinking about a meaningful branching strategy — the sooner the better, as it will simplify your daily application development workflows. This not only applies to just unlocked packages but development in general.

If you need a refresh on Git, checkout the Git and Github Basics module on Trailhead. Take a look at the Easy Spaces sample app, which focuses on the modularization with unlocked packages. Finally, start experimenting with different Git branch strategies to find the perfect fit for your organization.

About the author

René Winkelmeyer works as Principal Developer Evangelist at Salesforce. He focuses on enterprise integrations, mobile, and security with the Lightning Platform. You can follow him on Twitter @muenzpraeger.

Leave your comments...

Working with Modular Development and Unlocked Packages: Part 4