SOQL tags for Golang

If you are a Golang developer intending to write a client that interacts with Salesforce, this post is for you. We’ll share how you can annotate your Golang structs to generate SOQL queries to be used in Salesforce APIs and how this will make it easy for you to write Golang clients for Salesforce, without having to hard code those queries.

Querying Salesforce Records via APIs

Salesforce offers REST APIs that allow any third party to integrate with the platform, using their language of choice. One of the API endpoints (/services/data/<version>/query) allows developers to query Salesforce records using SOQL queries. SOQL is a very powerful SQL-like query language that allows developers to query Salesforce. It has extensive support for a conditional expression to filter the objects being retrieved, as well as very powerful way of performing join operations using relationship queries.

Let’s take a look at an example using the Salesforce object ‘Account’. Below is an incomplete list of fields for the ‘Account’ object. For context, you can find the complete list of fields here.

If you wanted to query all Accounts where AccountSource is one of ‘Advertisement’ or ‘Data.com’ you can write the following SOQL query:

1SELECT Name,AccountNumber,AccountSource,BillingAddress,HasOptedOutOfEmail,LastActivityDate,NumberOfEmployees FROM Account WHERE AccountSource IN ('Advertisement', 'Data.com')

You would then pass the above string as a query parameter to the /services/data/<version>/query endpoint to fetch the relevant records from Salesforce as follows:

1/services/data/v48.0/query?q=%0ASELECT%20Name%2CAccountNumber%2CAccountSource%2CBillingAddress%2CHasOptedOutOfEmail%2CLastActivityDate%2CNumberOfEmployees%20FROM%20Account%20WHERE%20AccountSource%20IN%20%28%27Advertisement%27%2C%20%27Data.com%27%29%0A

Now let’s look at how we might fetch Salesforce records using Golang.

Using Golang to fetch Salesforce Records

Golang is a statically typed, compiled programming language designed at Google. In recent years it has gained enough popularity to consider it a widely-adopted enterprise language. Want to learn more? Get started with Golang here.

Let’s see how quickly we can call the Salesforce query API using Golang. You start by defining the Golang struct representing the Salesforce object(s) you want to query. Let’s continue using the Account object described earlier. The Golang struct representation looks as follows (note the JSON annotations to marshal and unmarshal struct to/from JSON):

1type Account struct {
2    Name               string    `json:"Name"`
3    AccountNumber      string    `json:"AccountNumber"`
4    AccountSource      string    `json:"AccountSource"`
5    BillingAddress     string    `json:"BillingAddress"`
6    HasOptedOutOfEmail bool      `json:"HasOptedOutOfEmail"`
7    LastActivityDate   time.Time `json:"LastActivityDate"`
8    NumberOfEmployees  int       `json:"NumberOfEmployees"`
9}

Then we would model the response from query API as follows:

1type QueryResponse struct {
2    Done            bool      `json:"done"`
3    NextRecordsURL  string    `json:"nextRecordsUrl"`
4    Accounts        []Account `json:"records"`
5    TotalSize       int       `json:"totalSize"`
6}

The following snippet shows one of the ways you can call the Salesforce query API and pass along the required SOQL string:

1// Select clause specifying which columns to select from Account object
2selectClause := "SELECT Name,AccountNumber,AccountSource,BillingAddress,HasOptedOutOfEmail,LastActivityDate,NumberOfEmployees FROM Account"
3
4// Filter criteria for AccountSource
5sources := []string{"Advertisement", "Data.com"}
6
7// Build the soql query using strings.Builder
8var soqlBuff strings.Builder
9soqlBuff.WriteString(selectClause)
10
11// Specify where clause with AccountSource IN as filter
12soqlBuff.WriteString(" WHERE AccountSource IN (")
13
14// Iterate over filter criteria and add those 
15for indx, source := range sources {
16    if indx > 0 {
17        soqlBuff.WriteString(",")
18    }
19    soqlBuff.WriteString("'")
20    soqlBuff.WriteString(source)
21    soqlBuff.WriteString("'")
22}
23soqlBuff.WriteString(")")
24
25// Add SOQL query as query parameter
26values := url.Values{}
27values.Set("q", soqlBuff.String())
28
29// Create the encoded URL path
30path := fmt.Sprintf("/services/data/v48.0/query?%s", values.Encode())
31serverURL := "https://<your domain>.salesforce.com"
32
33// Create http request
34req, err := http.NewRequest(http.MethodGet, serverURL+path, nil)
35if err != nil {
36    // Handle error case
37}
38req.Header.Add("Authorization", "Bearer <session ID>")
39req.Header.Add("Content-Type", "application/json")
40
41// Create http client
42httpClient := &http.Client{}
43
44// Perform the query
45resp, err := httpClient.Do(req)
46if err != nil {
47    // Handle error case
48}
49
50// Read the response
51payload, err := ioutil.ReadAll(resp.Body)
52if err != nil {
53    // Handle error case
54}
55
56// Unmarshal the response in QueryResponse
57var queryResponse QueryResponse
58err = json.Unmarshal(payload, &queryResponse)
59if err != nil {
60    // Handle error case
61}

Note that constructing a SOQL query is quite tedious—especially if you want to add or remove any field from the ‘select’ clause, or add/remove/update filter criteria from the ‘where’ clause. What is especially error prone is that you have to maintain the select clause separately from Account struct and there is always the danger of things falling out of sync. Additionally, adding a new filter criteria means more code changes with special consideration for different logical operators.

If you were to expose this as a method to be used by different parts of your program, then you will end up taking the SOQL string—or at least the ‘where’ clause—as parameter. That might look something like this:

1func getAccounts(queryCriteria string) ([]Account, error) {
2    // Select clause specifying which columns to select from Account object
3    selectClause := "SELECT Name,AccountNumber,AccountSource,BillingAddress,HasOptedOutOfEmail,LastActivityDate,NumberOfEmployees FROM Account"
4    
5    // Build the soql query using strings.Builder
6    var soqlBuff strings.Builder
7    soqlBuff.WriteString(selectClause)
8    soqlBuff.WriteString(queryCriteria)
9    
10    // Rest of the code similar to above snippet
11    ...
12}

Notice that you expose yourself to SOQL injection as you are constructing the query by concatenating the string passed to your method. You will need to add enough validations in your method to make sure that doesn’t happen.

You might be asking, isn’t there a simpler way? The answer is: Yes! That’s where new go-soql library comes in.

Introducing go-soql Library

As more and more developers embrace Golang it’s important to provide easier means and ways to interact with Salesforce APIs. One such step is to provide a SOQL annotation library that allows Golang developers to tag their structs similarly to how they tag their structs for JSON marshaling/unmarshaling. And that is the aim of new go-soql library.

This library was developed as an internal project within Salesforce, however we saw value in open sourcing this library for benefit of the Golang developer community. That being said, it’s important to note that this library is not officially supported by Salesforce. It is maintained by the team which heavily relies on this library for its own use cases.

Let’s take a look at how it works. We’ll use the same Account object and Golang struct defined above, so you can see exactly how this new SOQL tag library makes your life easier. We start by annotating our Account struct as follows:

1type Account struct {
2    Name               string    `soql:"selectColumn,fieldName=Name" json:"Name"`
3    AccountNumber      string    `soql:"selectColumn,fieldName=AccountNumber" json:"AccountNumber"`
4    AccountSource      string    `soql:"selectColumn,fieldName=AccountSource" json:"AccountSource"`
5    BillingAddress     string    `soql:"selectColumn,fieldName=BillingAddress" json:"BillingAddress"`
6    HasOptedOutOfEmail bool      `soql:"selectColumn,fieldName=HasOptedOutOfEmail" json:"HasOptedOutOfEmail"`
7    LastActivityDate   time.Time `soql:"selectColumn,fieldName=LastActivityDate" json:"LastActivityDate"`
8    NumberOfEmployees  int       `soql:"selectColumn,fieldName=NumberOfEmployees" json:"NumberOfEmployees"`
9}

If we just want to generate the select clause without conditional expressions then it can be done as follows:

1import "github.com/forcedotcom/go-soql"
2...
3selectClause, err := soql.MarshalSelectClause(Account{}, "")

This will result in following string:

1Name,AccountNumber,AccountSource,BillingAddress,HasOptedOutOfEmail,LastActivityDate,NumberOfEmployees

Now if you add/remove any field in the Account struct it will automatically be reflected in the ‘select’ clause. As you might have already noticed this annotation goes hand in hand with json annotations so the response from the Salesforce API can be directly unmarshalled into the Account struct (as can be seen from the client code snippet in previous section).

While this addresses one problem in our initial code (addition/removal of new field needs to be maintained in two places), it is, of course, of limited use. We want the full SOQL query to be automatically generated. In order to do so, we need to define a couple more structs:

1type AccountQueryCriteria struct {
2    AccountSource []string `soql:"inOperator,fieldName=AccountSource"`
3}
4
5type AccountSoqlQuery struct {
6    SelectClause  Account              `soql:"selectClause,tableName=Account"`
7    WhereClause   AccountQueryCriteria `soql:"whereClause"`
8}

Now, we can generate a complete SOQL query using the above two structs:

1soqlStruct := AccountSoqlQuery{
2    SelectClause: Account{},
3    WhereClause: AccountQueryCriteria{
4        AccountSource: []string{"Advertisement", "Data.com"},
5    },
6}
7soqlQuery, err := soql.Marshal(soqlStruct)

And, voila! This will generate the SOQL query that we expect. Now we can use that in our call to /services/data/<version>/query API. Note that you can change the AccountQueryCriteria struct to modify the conditional expression. If you add more than one field to AccountQueryCriteria then they are combined using the AND logical operator.

Now your client code will look something like this. No more SOQL query string maintenance, or worrying about how to construct where clauses with combinations of different logical operators. Just define your struct and marshal it into SOQL query.

1// Construct SOQL struct that includes which object you want to query and its query criteria
2soqlStruct := AccountSoqlQuery{
3    SelectClause: Account{},
4    WhereClause: AccountQueryCriteria{
5        AccountSource: []string{"Advertisement", "Data.com"},
6    },
7}
8
9// Generate SOQL query using the above struct
10soqlQuery, err := soql.Marshal(soqlStruct)
11
12// Add SOQL query as query parameter
13values := url.Values{}
14values.Set("q", soqlQuery)
15
16// Create the encoded URL path
17path := fmt.Sprintf("/services/data/v48.0/query?%s",values.Encode())
18serverURL := "https://<your domain>.salesforce.com"
19
20// Create http request
21req, err := http.NewRequest(http.MethodGet, serverURL+path, nil)
22if err != nil {
23   // Handle error case
24}
25req.Header.Add("Authorization", "Bearer <session ID>")
26req.Header.Add("Content-Type", "application/json")
27
28// Create http client
29httpClient := &http.Client{}
30
31// Perform the query
32resp, err := httpClient.Do(req)
33if err != nil {
34   // Handle error case
35}
36
37// Read the response
38payload, err := ioutil.ReadAll(resp.Body)
39if err != nil {
40   // Handle error case    
41}
42
43// Unmarshal the response in QueryResponse
44var queryResponse QueryResponse
45err = json.Unmarshal(payload, &queryResponse)
46if err != nil {
47   // Handle error case
48}

You can also create a function that takes AccountQueryCriteria as a parameter and exposes it to be used by different parts of your program to query Account objects using different query criteria.

1func getAccounts(queryCriteria AccountQueryCriteria) ([]Account, error) {
2   soqlStruct := AccountSoqlQuery{
3        SelectClause: Account{},
4        WhereClause: queryCriteria,
5    }
6    soqlQuery, err := soql.Marshal(soqlStruct)
7    // Same code as above snippet to actually construct and perform http query
8}

Since SOQL is generated by using this struct, the SOQL injection vulnerability is addressed. go-soql will take care of constructing a valid SOQL by properly escaping the user provided parameters.

As you can see from above, using the go-soql library for writing Golang clients for Salesforce is much easier since all you need to do is define the struct and annotate it.

Summary

Hopefully, you now have an idea of the power of the go-soql library. This blog post just scratches the surface of what the go-soql library can do. The library has extensive support for logical operators as well as child-to-parent and parent-to-child relationships. You can find all those details and more in the README of the repo.

Additional Resources
Git repo: https://github.com/forcedotcom/go-soql
Related Trailhead modules:

• https://trailhead.salesforce.com/en/content/learn/modules/api_basics
• https://trailhead.salesforce.com/en/content/learn/modules/apex_database/apex_database_soql

Author Bio:
Atul Kshirsagar is a Software Architect at Salesforce. He is passionate about technology, open source projects and loves to build software. He has presented at multiple conferences and contributed to many open source projects. In his current role as Software Architect at Salesforce he is focussed on leading the efforts to automate the entire patching lifecycle for the DB layer within the Salesforce Platform.
LinkedIn: https://www.linkedin.com/in/kshirsagar
Twitter: @atulckshirsagar