Salesforce DX gives all of us building and delivering apps on the platform more options for how we go about our work. Over the next few weeks, we’re going to take a deeper look at the capabilities of unlocked packages and share some techniques your team could use when considering packaging or incorporating modular development in your app dev lifecycle.
We’re going to look at:
- Part 1: What even is a package anyway? How can you start to experiment with segmenting your org?
- Part 2: How can you start to organize metadata from an app, let alone an entire org, into packages? How do you tackle organizing your metadata and projects in source control?
- Part 3: What do these changes mean for app builder workflows? What will happen if I install an unlocked package into my production org today?
- Part 4: How can you define a successful Git branching strategy that works best for most team sizes? How, when and where should packaging be added to your continuous deployment?
Before we dive in, please note: we’re going to assume you’re already familiar with the basics of Salesforce DX. If you need to brush up, check out our Getting Started series.
In this kickoff post, I want to take a closer look at what packaging actually is, and how to start experimenting with breaking up an org into more modular units. We’ll be working with the Easy Spaces sample app in this series, which is based on the app showcased during the opening keynote at TrailheaDX ’18.
Org-based, source-based and package development
An important shift in the discussions about delivering apps with Salesforce DX is that Salesforce DX introduces a different way to think about your application delivery cycles. If you’ve ever installed an AppExchange app, those features were deployed to your org using packaging. With unlocked packaging (now in Beta), this way of deploying and organizing changes is available to customers and partners. And adopting packaging will impact how you’ll manage and think about the very structure of your Salesforce org.
To better illustrate this, let’s consider non-package-based development (i.e. the way that most of us have been building and delivering apps on the platform) as falling into the category of ‘org-based development’. Your production org is the ultimate structure, or source of truth, for all the work your teams carry out. Even if you’re using a source control tool today, your production environment(s) still hold the complete version of all your customizations.
With package development, you start to segment your org and your customizations into packages. The package will become the ultimate source of truth for whatever it contains, and you’ll bring different environments up to speed by installing your package(s) into those orgs. Those environments can be scratch orgs, sandboxes, and, of course, production. As you make updates to the metadata in your unlocked package, you create a new version of your package. Then you can install the new version in whatever environments you want to update. These could be different production environments, sandboxes, scratch orgs—the various environments your company uses.
Source-driven development, modularization and the new capabilities for package-based deployments are all tightly related topics, but they aren’t the same things. As we discussed in the Getting Started with DX series, you can adopt pieces of modular development and source control, and choose to wait to adopt packaging. You’ll be more successful with adopting packaging if you’ve first spent time and energy to understand how to modularize your org and get a strong system for source control in place.
As we move through ideas about dissecting your org into modules and creating representations of those modules that can live in source control, it’s important to keep the differences between source, modularization and packaging in mind.
Do I have to use packages for everything?
You don’t have to put every single piece of your org into a package. If you decide to explore packages, you could (and should) start small. If you choose to create a package to manage some pieces of your org, it doesn’t mean you can’t still manage other parts of your org with change sets or other deployment strategies. YOU get to decide what makes sense for your org.
As mentioned above, to adopt packaging, you’ll have to look at how to organize the pieces of metadata that make up your org into distinct modules and figure out a sensible plan to store those modules in source control. Even if you’re not looking to adopt packaging now, doing the work to better organize your metadata and figure out how to use source control can benefit your org and the teams working to support your org.
When you think about putting pieces of your org into packages, understanding the difference between metadata in managed packages, metadata in unlocked packages, and metadata that isn’t packaged at all will matter. When metadata is part of a managed package, you have limited ability to change it. In fact, depending on the kind of metadata and how the developer of the managed package built their package, you may not be able to make any changes.
With unlocked packages, as the name indicates, you can still make changes to metadata that’s part of that package, even once it’s installed into production. For example, if an object is in an unlocked package and a system administrator needs to update the help text or label of a field, they can do so. In a managed package, that update may not be allowed.
For those who are wondering: But what happens to the updates made in production to an unlocked package? Are they automatically pulled back into a new package version? The answer is NO. Just like today, you’ll need to figure out a system for capturing changes made directly in production and make sure you’re updating packages appropriately.
Whether or not you’re going to start experimenting with packages, knowing how different parts of your org are interconnected will help you better able to manage your org proactively. You’d be able to identify how changes to one part of your org will impact another part of your org before those changes hit production.
A lot of this understanding comes from knowing the shape of dependencies in your org.
What are dependencies anyway?
Dependencies can be a lot of different things. It’s a term that gets used to describe relationships between pieces of code, parts of a system, and even the teams that build systems and comprise companies. In this series, we’re going to be looking more deeply at metadata dependencies, the relationships between different parts of your org. For now, here’s a quick overview.
When different pieces of metadata refer to one another, you’ve got a dependency. Sometimes dependencies are obvious, but sometimes they’re not. For example, think about a page layout. It’s a piece of metadata by itself, but it’s made up of references to other things (other pieces of metadata) in an org.
Even simple page layouts reference an object’s fields, related lists, and object-specific actions. All of these references are dependencies. In the case of a simple layout, these dependencies could be fairly straightforward to identify.
However, more complex layouts might include references to Visualforce pages, etc. So now you’ve got a page layout that has dependencies on a Visualforce page—which has it’s own set of dependencies, like controller classes in Apex, and any metadata referenced by that code. So how do you start identifying, let alone breaking up and sorting out, all of these connections?
There is no one way to do this. And the reality right now is that identifying dependencies takes work. And perhaps more than a little trial-and-error. The nature of your org and your own skill sets will ultimately be huge factors in determining what a practical workflow may look like. There is a new API in pilot for Summer ’18, however, aimed at helping make this process easier, called the Dependency API. For more information on the pilot and the API, check out the link.
Let’s walk through how I approached turning a chunk of an org into a modular structure manually.
Let’s break up the party
One of the fastest ways to pull together dependencies is to let the platform do it for you. When you create an unmanaged package, if you select a piece of metadata with dependencies, Salesforce will pull in the additional metadata required. (For the most part — more on that below.) The workflow here is pretty simple: you select a piece of metadata in the Package Manager, and look at what gets pulled in to your package.
However, there are a couple key points to remember if you want to try this method. First, do not try to put everything in your org into a single package. Second, do not try to put everything in your org into a single package.
For this method to help you begin to ‘break up’ your org, you need to take the first step. So identify a piece to start with — such as a single app (maybe choose one without 100 tabs and 500 custom Visualforce pages). Or maybe a standalone Lightning App Page or one Visualforce page. I started with a simple app.
I started by pulling together an unmanaged package in one of our keynote demo orgs. The first piece of metadata I added was the Lightning app we created for the demos. I then reviewed the metadata Salesforce pulled into my package manifest automatically:
The reason why a particular piece of metadata got pulled in (i.e. your dependencies) will show up in the ‘Included By’ column. If you click the ‘View Dependencies’ button, you’re taken to a page that groups together some pieces of your package into categories and shows you why they’re included.
This page can be misleading. Not everything in your package will show up here. (So don’t be fooled by a blank dependencies screen!)
And because only some kinds of metadata show up here, what this page can actually show you is limited. For the metadata that does appear on this page, this view can help you identify some of the cruft—AKA metadata that’s not relevant—in your package. For example, I knew the stuff listed under ‘Object Operational Scope’ for Accounts, Contacts and Users didn’t actually have anything to do with my Lightning app, as the app didn’t actually have any functionality related to those objects. So the individual pieces of metadata that got pulled into my package because of those dependencies, which show up in this view pretty nicely, are actually cruft.
For the most part, you can’t remove metadata selected for you by Salesforce from an unmanaged package, but that’s OK. We’ll deal with cleaning up our metadata down the road. At this point in my process, it was more important for me to look out for what wasn’t yet in my package manifest.
Using a Lightning app as the first piece of the package pulled in quite a bit of metadata, but there were gaps. The Permission Set I’d created for the app hadn’t been added. As I looked closer, my custom objects hadn’t been added at all. My Apex test classes were also left out. So I needed to add those pieces of metadata manually. Because I knew enough about my app to notice which pieces of metadata had been left out, I could take care of adding those pieces. Knowledge of my app is also what let me identify the cruft that got pulled in more quickly, too.
However, even when I’ve had to use this process for apps and orgs that I’m not as familiar with, I can still identify what’s missing and what’s unneeded through this same process. I just have to spend more time drilling into Apex classes or objects and looking at what’s there. I also have to be unafraid to use a trial-and-error cycle to figure out what might be off with my package. I’ll pull together an unmanaged package, try to deploy it into another org, look at the failure messages, make changes, and try again.
This is why starting with a smaller piece of your org is a key part of being successful with this technique. If you’re going to be able to focus on tracking down what’s a ‘real’ dependency and what’s not, you’re setting yourself up for success by only having a limited amount of metadata to investigate.
Let’s talk source control
Once I’d assembled my unmanaged package, I fired up the Salesforce CLI. Using the CLI, I could pull the contents of my package out of the demo org and onto my local machine. While in the root directory of what would become my Salesforce DX project, I ran the following command:
I needed to enclose the name of my unmanaged package in single quotes because I gave it a name with a space. I wrote the .zip file created by the command to the ‘demoOrgExtract’ folder I’d created on my machine, using the -r parameter. The -u parameter specified which org the CLI used for the pull.
Once I’d pulled down the metadata and unzipped it, my project looked like this:
The last thing I needed to do was convert this extracted source into a format that was compatible with Salesforce DX and the CLI. I created a temporary project, using the sfdx force:project:create command. Then I ran sfdx force:mdapi:convert and set the -r attribute to my ‘unpackaged’ folder.
Now I had my metadata on my local machine, in a format that could (in theory) go into a source control system and work with Salesforce CLI. But I needed to do more work to get rid of the extra metadata in my package and get this mass better organized.
We’ll talk about that in our next post.
What’s next and what to do now
In our next post, we’ll look at breaking up a larger piece of metadata into smaller pieces within a Salesforce DX project and committing everything to source control. We’ll also touch on what development workflows look like when working on modular projects with Salesforce DX. In the meantime, you should head over to Trailhead and get hands-on with the modules and projects related to source control management and Salesforce DX:
You can also brush up on the tools and functionality of Salesforce DX and the power of the CLI with our Getting Started with Salesforce DX blog series.