Einstein Vision is a service that helps you build smarter applications by using deep learning to automatically recognize images. It provides an API that lets you use image recognition to build AI-enabled apps.

Golang is becoming a popular programming language as it is extremely fast, has a low-runtime footprint, and has a statically linked binary with minimal dependencies. If you’re a developer who wants to get familiar with Golang and Einstein Vision, this blog is for you. Today we learn how to quickly integrate your Go applications with Einstein Vision.

Because visual data is expanding rapidly at a 9% CAGR (Compound Annual Growth Rate) annually, it’s vital for organizations to tap into non-textual data to discover conversations about their brand.

Some use-cases to consider include:

• Tracking your brand perception on social channels like Twitter and Facebook
• Giving field service agents the ability to identify and classify objects, such as factory equipment, to handle service updates faster

Before we dive into more detail, let’s explore the algorithms that power Einstein Vision.

The algorithms behind Einstein Vision

Einstein Vision uses advancements made in deep learning and an underlying architecture of neural networks. It’s able to classify images with high accuracy as it uses multiple hidden layers to extract hidden features and assign appropriate weights and biases to each feature output.

It’s outside the scope of this blog to discuss neural networks in great detail. In general, think of them as multiple neurons tied together in a graph. There’s an input layer, where images are fed, and a hidden layer, which extracts each feature of the image’s contours, corners, and shapes to eventually predict new images.

The last layer in a neural network is the output layer, which predicts outputs. So now that we know a little bit more about neural networks, let’s look at the specifics of how Einstein Vision models are built using a training dataset.

How does Einstein Vision work?

Einstein Vision is part of a supervised learning technique in which a model is trained on prelabeled training data. The training dataset consists of labeled images and is loaded into Einstein via an API call. Next, a REST API call is made to train the dataset, and the output is a trained model, which has a unique model ID. Then an API call is made to predict the label of a new image that the model has never seen before.

Golang Einstein library
In this section you will learn about the class structure and various functions exposed by the Golang library.

The class structure

Golang wrapper

Wrappers are contained in a couple of classes under the vision package as listed here:

Functions in vision.go
Functions in this file focus on the actual classification of a new image against a generic prebuilt classifier or a custom classifier.

Functions in datasets.go file

Functions in this file focus on the lifecycle of the DataSet.

We also have some Go structs to parse the response coming from Einstein Vision service. These are located in types.go

Code examples

Create a DataSet

Code walkthrough

1. First we get the access token from the environment variable.
2. Call vision.createDataSet with the web URL where the training data set is stored.
   1. In this call, we make an HTTP POST call.
   2. Create an http.Client object.
   3. Set the HTTP header with the access token.
   4. Set the dataset PATH in the body.
   5. Make HTTP POST call.
3. Process the return, and print the response.

After the dataset has been created, the next step is to train a model based on this dataset.

Train a model

Code walkthrough

1. First we get the access token from the environment variable ACCESSKEY
2. Call vision.TrainDataSet (listing below) with the web URL where the training data set is stored.
   1. In this call we make an HTTP POST call.
   2. Create an http.Client object.
   3. Set HTTP header with the access token.
   4. Set the datasetId in the body.
   5. Make HTTP POST call.
3. Process the return, and print the response.
4. In the response, you will see the output describing the model.

Output of the code listing

Once we train a model, we can use it to predict the label of an image that isn’t in the training set.

Predict using a model with local image file

In the following code listing, we use a local image along with a modelId to get the classification.

Code walkthrough

1. Instantiate the modelId with predefined model FoodImageClassifier
2. Instantiate samplePath variable with local path to the Image we want to classify.
3. Get the access token from the environment variable ACCESSKEY
4. Call vision.ClassifyLocal(modelId, samplePath, accessToken) with appropriate parameters

Predict using a model with remote image file

In the following code listing, we use a Remote image along with a modelId to get the classification.

Code walkthrough

1. Instantiate the modelId with predefined model FoodImageClassifier
2. Instantiate samplePath variable with remote Http path to the Image we want to classify.
3. Get the access token from the environment variable ACCESSKEY
4. Call vision.Classify(modelId, samplePath, accessToken) with appropriate parameters

Output of the code listing above

Summary

This blog post provides you with the basics on how to use Einstein Vision from a Golang stand-alone application. Hopefully these samples provide a good starting point for developers interested in extending their Golang applications to work with Einstein Vision APIs. You can find the code repo for the library and samples at https://github.com/rajdeepd/einstein-go-apis.

You can also try Einstein Vision on Trailhead to learn how to integrate Einstein Vision into your Force.com workflows using Apex and Visualforce. If you have any questions, feel free to reach out on Salesforce Developer Forums.

About the author
Rajdeep Dua is a director of developer relations at Salesforce. He is passionate about helping developers learn about cloud computing, machine learning, and Salesforce. He has over 17 years of experience in software product development and developer relations.

Related Links

https://metamind.io/
https://trailhead.salesforce.com/en/projects/predictive_vision_apex
https://github.com/rajdeepd/einstein-go-apis