Access Models API with REST

The Models REST API is a REST API that connects your application to large language models (LLMs). Start by creating a Salesforce app and generating a JSON Web Token (JWT) that you can use to access this API.

API Flow

All Models API requests are subject to Salesforce’s usage and billing rates. See Einstein Usage.

The quickest way to get started with the Models REST API is with our Postman collection.

This video shows you how to set up your org to use the REST API.

Access the Models REST API
undefined

The video contains a simplified version of the steps below. You can watch the video or follow along step by step.

To securely communicate with Salesforce using the Models REST API, you must create an External Client App or a Connected App. Both app types allow external services to integrate with Salesforce APIs using well-known authorization protocols, such as OAuth. External Client Apps (ECAs) are the new generation of Salesforce apps, and we suggest that you use an ECA for this purpose. However, these basic guidelines apply to both app types.

To create an ECA, opt in to external client apps from Setup and then create an external client app with OAuth and JWT enabled. Use these instructions to get set up.

When creating the app, include these settings.

  1. Use these OAuth scopes.

    • Manage user data via APIs (api): Gives you access to user data.
    • Perform requests at any time (refresh_token, offline_access): Permits you to get an OAuth access token.
    • Access the Salesforce API Platform (sfap_api): Enables access to the Salesforce REST API platform.

    OAuth settings scope

  2. Select these additional OAuth settings.

    • Enable Client Credentials Flow: Allows your app to exchange its client credentials for an access token.
    • Issue JWT Web Token (JWT)-based access tokens for named users: Allows app to issue tokens for named users.

    OAuth settings checkboxes

  3. After creating the app, ensure that the API caller has the correct client credentials and that the client can issue JWT-based access tokens.

    • Click the Policies tab for the app, and then click Edit.
    • Select the Enable Client Credentials Flow checkbox.
    • Specify the client user in the Run As field.
    • Select the Issue JSON Web Token (JWT)-based access tokens checkbox. By default, this token expires in 30 minutes. You can change this value to less than 30 minutes.

    OAuth user settings

A JSON Web Token (JWT) is required for authorization.

  1. From Setup, in the Quick Find box, enter External Client Apps, and then select External Client Apps Manager.

  2. Select your app, and click the Settings tab.

  3. Expand the OAuth Settings section.

  4. Click the Consumer Key and Secret button, and copy your key and secret. You need these values to mint this token.

    Store your consumer secret in a secure location.

  5. From Setup, in the Quick Find box, enter My Domain, and then select My Domain.

  6. Copy the value shown in the Current My Domain URL field.

  7. Request a JWT from Salesforce using a POST request, specifying the values for your domain, consumer key, and consumer secret. For example:

    If you receive an invalid_grant error with the description, request not supported on this domain, verify that the domain is the domain specified when you view My Domain in Setup.

  8. Use the access_token value in this response for the Authorization header in your API requests. For example:

When you have a valid token, you can call any of the Models REST API endpoints.

The Models REST API provides four different endpoints for four different capabilities.

CapabilityEndpoint PathDescription
Generate Chat/models/{modelName}/chat-generationsGenerate a response based on a list of messages representing a chat conversation.
Generate Embeddings/models/{modelName}/embeddingsCreate an embedding vector representing the input text.
Generate Text/models/{modelName}/generationsGenerate a response based on the prompt provided.
Submit Feedback/feedbackSubmit feedback for generated text.

Choose a model to use when you call the generations or embeddings endpoints. Each endpoint, other than the feedback endpoint, requires the API name of the model as part of the path. For instance, if you want to use the GPT-3.5 Turbo model for a generation request, specify the API name sfdc_ai__DefaultGPT35Turbo by making a request to this endpoint.

The Models API supports models from various Salesforce-enabled providers. See Supported Models for Models API.

These headers are relevant for API requests.

HeaderDescriptionExample
AuthorizationRequired. Authorization method for this request. This header must contain your JWT.Bearer {token}
Content-TypeRequired. Specify the application/json content type.application/json
x-sfdc-app-contextRequired. This value is reserved for future use. For now, specify EinsteinGPT.EinsteinGPT
x-client-feature-idRequired. This value is reserved for future use. For now, specify ai-platform-models-connected-app.ai-platform-models-connected-app

Make a POST request to one of the Models REST API endpoints. This example demonstrates a call to the generations endpoint.

If you receive a 401 or 404 response code, verify that your Salesforce app includes all the required scopes and settings. See Create a Salesforce App. Also, verify that you're using the complete access_token value that you generated earlier. See Generate a JWT.

And here’s a sample response.

After you successfully call the API, refer to these topics to optimize your implementation.