Access Models API with REST
The Models REST API is a REST API that connects your application to large language models (LLMs). Start by creating a Salesforce app and generating a JSON Web Token (JWT) that you can use to access this API.
All Models API requests are subject to Salesforce’s usage and billing rates. See Einstein Usage.
The quickest way to get started with the Models REST API is with our Postman collection.
This video shows you how to set up your org to use the REST API.
The video contains a simplified version of the steps below. You can watch the video or follow along step by step.
To securely communicate with Salesforce using the Models REST API, you must create an External Client App or a Connected App. Both app types allow external services to integrate with Salesforce APIs using well-known authorization protocols, such as OAuth. External Client Apps (ECAs) are the new generation of Salesforce apps, and we suggest that you use an ECA for this purpose. However, these basic guidelines apply to both app types.
To create an ECA, opt in to external client apps from Setup and then create an external client app with OAuth and JWT enabled. Use these instructions to get set up.
When creating the app, include these settings.
-
Use these OAuth scopes.
- Manage user data via APIs (api): Gives you access to user data.
- Perform requests at any time (refresh_token, offline_access): Permits you to get an OAuth access token.
- Access the Salesforce API Platform (sfap_api): Enables access to the Salesforce REST API platform.
-
Select these additional OAuth settings.
- Enable Client Credentials Flow: Allows your app to exchange its client credentials for an access token.
- Issue JWT Web Token (JWT)-based access tokens for named users: Allows app to issue tokens for named users.
-
After creating the app, ensure that the API caller has the correct client credentials and that the client can issue JWT-based access tokens.
- Click the Policies tab for the app, and then click Edit.
- Select the Enable Client Credentials Flow checkbox.
- Specify the client user in the Run As field.
- Select the Issue JSON Web Token (JWT)-based access tokens checkbox. By default, this token expires in 30 minutes. You can change this value to less than 30 minutes.
A JSON Web Token (JWT) is required for authorization.
-
From Setup, in the Quick Find box, enter
External Client Apps
, and then select External Client Apps Manager. -
Select your app, and click the Settings tab.
-
Expand the OAuth Settings section.
-
Click the Consumer Key and Secret button, and copy your key and secret. You need these values to mint this token.
Store your consumer secret in a secure location.
-
From Setup, in the Quick Find box, enter
My Domain
, and then select My Domain. -
Copy the value shown in the Current My Domain URL field.
-
Request a JWT from Salesforce using a POST request, specifying the values for your domain, consumer key, and consumer secret. For example:
If you receive an
invalid_grant
error with the description,request not supported on this domain
, verify that the domain is the domain specified when you view My Domain in Setup. -
Use the
access_token
value in this response for theAuthorization
header in your API requests. For example:
When you have a valid token, you can call any of the Models REST API endpoints.
The Models REST API provides four different endpoints for four different capabilities.
Capability | Endpoint Path | Description |
---|---|---|
Generate Chat | /models/{modelName}/chat-generations | Generate a response based on a list of messages representing a chat conversation. |
Generate Embeddings | /models/{modelName}/embeddings | Create an embedding vector representing the input text. |
Generate Text | /models/{modelName}/generations | Generate a response based on the prompt provided. |
Submit Feedback | /feedback | Submit feedback for generated text. |
Choose a model to use when you call the generations or embeddings endpoints. Each endpoint, other than the feedback endpoint, requires the API name of the model as part of the path. For instance, if you want to use the GPT-3.5 Turbo model for a generation request, specify the API name sfdc_ai__DefaultGPT35Turbo
by making a request to this endpoint.
The Models API supports models from various Salesforce-enabled providers. See Supported Models for Models API.
These headers are relevant for API requests.
Header | Description | Example |
---|---|---|
Authorization | Required. Authorization method for this request. This header must contain your JWT. | Bearer {token} |
Content-Type | Required. Specify the application/json content type. | application/json |
x-sfdc-app-context | Required. This value is reserved for future use. For now, specify EinsteinGPT . | EinsteinGPT |
x-client-feature-id | Required. This value is reserved for future use. For now, specify ai-platform-models-connected-app . | ai-platform-models-connected-app |
Make a POST request to one of the Models REST API endpoints. This example demonstrates a call to the generations
endpoint.
If you receive a 401
or 404
response code, verify that your Salesforce app includes all the required scopes and settings. See Create a Salesforce App. Also, verify that you're using the complete access_token
value that you generated earlier. See Generate a JWT.
And here’s a sample response.
After you successfully call the API, refer to these topics to optimize your implementation.
- Supported Models for Models API
- Rate Limits for Models API
- Specify Languages and Locales with Models API
- Toxicity Scoring with Models API
- Data Masking with Models API
- Get Generation Feedback Using Models API
- Access Models API with Apex
- Build Lightning Web Components and Flows with Models API
- Models REST API Reference (Beta)
- Salesforce Help: About Einstein Generative AI
- Salesforce Help: External Client Apps
- Salesforce Help: Connected Apps