An Introduction to Force.com Metadata

Abstract

Force.com allows you to use traditional development practices for creating on-demand applications. Force.com extends the cloud computing development process to encompass external tools such as integrated development environments, source control systems, and batch scripts to facilitate development and deployment.

The Force.com platform provides a rich and powerful metadata model and Metadata API. Platform metadata describes your application and the API encompasses a set of methods that provide programmatic access to the configuration and source of your application. The Force.com IDE and Force.com Migration Tool are examples of client applications that build upon the Metadata API.

After reading this article, you will understand the role of metadata and the Metadata API on the Force.com platform. You will also see how two popular tools, the Force.com IDE and Force.com Migration Tool, make use of the API. The final section of the article shows a few examples of programmatic access to the Metadata API, hopefully motivating you to create your own tool based on the Metadata API.

Metadata and the Metadata API

Metadata lies at the heart of the Force.com platform. The Force.com IDE, Force.com Migration Tool, and the browser-based user interface that you use to add records in your Force.com applications are all powered by metadata. The following sections define metadata, and show how Force.com uses it.

Defining Metadata

Most applications manipulate data, which is often stored in a database. Typically this data is information such as Account details or Contact information. However, you can consider the source code for the application itself as data that provides the information on how your application looks, feels, and functions. You can refer to this information as metadata.

Traditional development paradigms rely on software developers to take this code and configuration information, compile, package, and deploy it before users could begin using the application. The Force.com platform, however, delivers Software as a Service (SaaS) and immediately uses the metadata to provide an on-demand application directly in the cloud.

On the Force.com platform, metadata defines a rich set of components including all aspects of Force.com functionality, from the appearance of user interfaces through workflow. Examples of components of metadata include custom user interfaces created using Visualforce, generated page layouts, and even Apex classes for custom application functionality. In short, just about every aspect of a Force.com application, ranging from workflows to the configuration of database objects (that store data!), is metadata.

How Metadata Defines Your Applications

To make the concept of Force.com metadata concrete, consider the Force.com app builder interface in a Developer Edition (DE) environment (or other Edition), specifically the Setup menu. The pages that launch from the Setup menu are Web-based tools that you can use to create and manipulate metadata that corresponds to your applications, including the underlying database objects and overlying user interfaces.

For example, you can navigate to Setup -> Create -> Objects, click on New, and then create a new object. As you can see from the following figure, an object called TodoItem exists, and this object has, among other things, a description, lookup, and date field. A name and owner field is also present. Creating this object using the Web interface is a matter of pointing and clicking using the Setup menu.

Meta setup.jpg

A result of creating the object is that you get a lot of functionality out of the box, including metadata which describes all of the logic and visual presentation that make up the TodoItem.

Here's a look at the data entry screen for this object, which Force.com automatically generates:

Db todoitemcreation.jpg

Notice that the page has a date picker for the Due Date field. This automatically happens because the platform notices that the field's metadata has a data type of "Date", and thus renders an appropriate user interface element. Likewise, the TodoList field is actually a lookup, so the user interface element provides a button that allows you to look up a related record. Read An Introduction to the Force.com Database to learn more about this functionality.

Force.com also captures the business logic that you define as metadata. For example, you may define a workflow rule through the Setup menu. It won't correspond to a user interface of any kind, but the workflow rule instead adds functionality to your application, based on the metadata that defines it. For this reason, metadata is not exactly the same as source code for your application—it's more. Force.com uses metadata to configure and shape the default functionality of your specific applications.

Accessing Metadata Through the Metadata API

The Setup menu is not the only way of creating and manipulating metadata. The platform exposes a Metadata API—a SOAP-based Web service—that lets you access metadata in the same way you can access any of your Force.com applications, from any location on the Web. Because metadata controls the logic and presentation of your applications, you can develop or shape applications by creating and modifying the metadata directly. In other words, you can write programs that write programs.

The Metadata API, discussed in more detail below, exposes two sets of methods via the Web service interface. The first allows you to create, delete and update sets of metadata components. For example, you can use these methods to modify the schema of a database object to include an additional field. The second set of methods contains file-based metadata calls that allow you to retrieve and deploy file representations of metadata, which is especially important when migrating between organizations (described later). For example, using this metadata, you can retrieve the file representation of the TodoItem object shown above. Here is a snippet of the resulting XML file:

<CustomObject xmlns="http://soap.sforce.com/2006/04/metadata">
<deploymentStatus>Deployed</deploymentStatus>
<fields>
    <fullName>Description__c</fullName>
    <inlineHelpText>Some help text</inlineHelpText>
    <label>Description</label>
    <type>TextArea</type>
</fields>
<fields>
    <fullName>Due_Date__c</fullName>
    <defaultValue>TODAY() + 3</defaultValue>
    <label>Due Date</label>
    <type>Date</type>
</fields>
<!-- snip -->
<validationRules>
        <fullName>Due_date_in_future</fullName>
        <active>true</active>
        <errorConditionFormula>Due_Date__c - TODAY() < 0</errorConditionFormula>
        <errorMessage>Due date must be in the future</errorMessage>
</validationRules>
</CustomObject>

As you can see, this representation of the metadata describes the database object. It defines the name, default values, labels and type of each field, validation rules for data entry, and much more.

With an introduction to metadata and the Metadata API out of the way, the rest of this article examines organizations and moving metadata, and cloud development. It culminates with a concrete example of interacting with the Metadata API.

Organizations and Moving Metadata

The DE environment provides a home for your applications and data as you develop your functionality. This environment is referred to as an organization or org. You could begin creating an application by logging into this org using a Web browser, accessing the Setup menu, and creating your database objects.

You can have many such orgs. If you have deployed your application within your company, you probably did so in a production org, which may have license and data volume restrictions depending on your current salesforce.com subscription. Or you could be testing in a sandbox org, which mirrors some of the data found in your production org, but is separate from the production org and its operation.

One way to look at these orgs is as containers for your metadata. For you to copy an application from one org to another, all you need to do is download all of the metadata from the one, and upload it to the other. The Metadata API enables this.

Note: To access or move your data (as opposed to your metadata), you can use Force.com's SOAP API, REST API, or Bulk REST API, or other APIs, tools, and capabilities designed for this task, from data import tools built into the Setup environment to others such as the Data Loader.

As with other platforms, moving an application system from one org to another will probably involve moving both the source of the application, metadata on the Force.com platform, as well as the underlying data. Used in conjunction, the Metadata API and SOAP API provide this functionality.

The following figure depicts the Metadata API and SOAP API as facets of each org: the Metadata API for retrieving the metadata for components of an application, and the SOAP API for retrieving data stored in the database.

Meta cloudyorgs.jpg

Force.com Development Beyond the Browser

The preceding sections explain metadata, the Metadata API, and the SOAP API as tools for manipulating applications built upon the Force.com platform. But what happens if you need to expand your development process beyond the browser? Force.com has a set tools and APIs that expand the development process from a browser environment to include external tools, such as code editors, source control systems, and batch scripts to facilitate development and deployment. An important use of such features is when migrating metadata components from one organization to another. For example, when making changes in a sandbox organization and then promoting those changes to a production environment.

The Force.com Metadata API is at the heart of such capabilities. It makes development in the cloud considerably richer than a browser-only environment—and provides the opportunities for novel applications that interface with the cloud beyond the data layer. The following two sections look at such applications: The Force.com IDE, and the Force.com Migration Tool.

The Force.com IDE

The Force.com IDE is an integrated development environment built on the Eclipse platform. The Force.com IDE is a client application that brings the familiar Eclipse IDE environment to developers of applications on the Force.com platform.

Using the IDE, you will have access to editors with code completion and syntax highlighting along with features for testing classes, exploring the schema of your objects, metadata deployment, and advanced code sharing. As a result of the code sharing support, which is usually provided through the use of a source code repository such as Subversion or Git, you can have a team of developers working on the same application.

All of the features of the Force.com IDE work through the Metadata API. In fact, the IDE supports all current metadata component types. If you save an Apex class, the IDE will save the class locally, as well as giving you the option to send the newly modified class code to the Force.com platform using the Metadata API. Force.com will compile the class in your org and send back any error messages to the IDE. This process delivers a familiar programming environment that extends to the cloud.

Meta ide.jpg

As you can see from this figure, the IDE provides a directory structure for your metadata as well.

When creating projects using the Force.com IDE, you can determine which set of metadata objects to retrieve and keep in sync with your org.

Ide initialprojectcontents.jpg

The IDE produces it's functionality using metadata file-based declarative calls, which you'll examine shortly.

The Force.com Migration Tool

The Force.com Migration Tool is an Ant library that lets you move metadata to an org using configuration files and a command-line interface. After adding your credentials to a simple build configuration file, you can begin issuing commands to retrieve and deploy metadata. For example, here's a simple command:

ant retrieveUnpackaged

This will invoke the retrieveUnpackaged action in the build script, which in turn performs a retrieve from the Force.com platform based on the contents of a default configuration file (package.xml). By default, this configuration file specifies (with wild cards) all classes, triggers, and static resources—and as a result, the retrieve action brings back all of the metadata associated with these objects and places them into a local directory called retrieveUnpackaged.

The output of the retrieveUnpackaged command in a directory structure containing the metadata components. For example:

classes                      
triggers
package.xml

Here, the classes folder contains all Apex classes and the triggers folder all the triggers. The special package.xml file is an XML package deployment descriptor that the Metadata API Developer's Guide documents, a file that describes the set of metadata components that are now available in this directory structure.

After retrieving the metadata, you can make modifications (for example, adding a new trigger or modifying the source code for a class) and then redeploy the changes using a similar command:

ant deployUnpackaged

As you can imagine, by changing a few parameters, you can deploy this code to a different org or environment, making it easy to migrate metadata from one organization to another.

Because this is a command-line utility, you can also use it in scripted files—adding to your arsenal of cloud development tools. For example, you could script migration from staging to production organizations, script multistage release processes, and more.

It's worth pointing out that what the Metadata API calls do is retrieve a textual representation of the metadata utilizing the file-based Metadata API calls discussed previously.

Summary

Metadata and the Metadata API provide a powerful mechanism for controlling logic, presentation, and configuration information for Force.com applications. You can access metadata either via the visual tools accessible in the Setup menu of the browser-based development environment or programmatically via the Metadata API.

Force.com's Metadata API supports app development approaches outside the platform's browser-based environment to include external tools such as code editors, source control systems, and batch scripts to facilitate development and deployment. The Force.com IDE and Migration tool, based on the Metadata API, are two examples of tools that you can use to complete typical development activities including source-control integration, automation of tasks and migration betweens orgs.

With a solid understanding of metadata and the Metadata API, you can leverage the benefits of the Force.com platform with a better understanding of applications and powerful development tools that support typical development scenarios and provide the opportunity to extend the platform beyond that which Force.com provides out-of-the-box.

Pitfall Notes

When using the Metadata API, you may encounter a few quirks. If you come across one, document it here to help others.

Destructive Changes

  • If deleting a picklist, don't include the picklist values in destructiveChanges.xml.

References

  • The Metadata API Developer's Guide describes the Metadata API in detail, providing many useful examples and a full description of the package.xml deployment descriptor.
  • Developer Force provides access to free DE accounts, which you can use to start programming immediately. Developer Force also provides links to documentation, forums, and more.
  • The Force.com IDE page provides access to the Force.com IDE and its documentation, as well as the article An Introduction to the Force.com IDE.