Access Models API with REST

This feature is a Beta Service. Customer may opt to try such Beta Service in its sole discretion. Any use of the Beta Service is subject to the applicable Beta Services Terms provided at Agreements and Terms.

The Models REST API is a REST API that connects your application to large language models (LLMs). Start by creating a Salesforce app and generating a JSON Web Token (JWT) that you can use to access this API.

API Flow

All Models API requests are subject to Salesforce’s usage and billing rates. See Einstein Usage.

To securely communicate with Salesforce using the Models REST API, you must create an External Client App or a Connected App. Both app types allow external services to integrate with Salesforce APIs using well-known authorization protocols, such as OAuth. External Client Apps (ECAs) are the new generation of Salesforce apps, and we suggest that you use an ECA for this purpose. However, these basic guidelines apply to both app types.

To create an ECA, opt in to external client apps from Setup and then create an external client app with OAuth and JWT enabled. Use these instructions to get set up.

When creating the app, include these settings.

  1. Use these OAuth scopes.

    • Manage user data via APIs (api): Gives you access to user data.
    • Perform requests at any time (refresh_token, offline_access): Permits you to get an OAuth access token.
    • Access the Salesforce API Platform (sfap_api): Enables access to the Salesforce REST API platform.

    OAuth settings scope

  2. Select these additional OAuth settings.

    • Enable Client Credentials Flow: Allows your app to exchange its client credentials for an access token.
    • Issue JWT Web Token (JWT)-based access tokens for named users: Allows app to issue tokens for named users.

    OAuth settings checkboxes

  3. After creating the app, ensure that the API caller has the correct client credentials and that the client can issue JWT-based access tokens.

    • Click the Policies tab for the app, and then click Edit.
    • Select the Enable Client Credentials Flow checkbox.
    • Specify the client user in the Run As field.
    • Select the Issue JSON Web Token (JWT)-based access tokens checkbox. By default, this token expires in 30 minutes. You can change this value to less than 30 minutes.

    OAuth user settings

A JSON Web Token (JWT) is required for authorization.

  1. From Setup, in the Quick Find box, enter External Client Apps, and then select External Client Apps Manager.

  2. Select your app, and click the Settings tab.

  3. Expand the OAuth Settings section.

  4. Click the Consumer Key and Secret button, and copy your key and secret. You need these values to mint this token.

    Store your consumer secret in a secure location.

  5. Request a JWT from Salesforce using a POST request. For example:

    If you receive an invalid_grant error with the description, request not supported on this domain, verify that the domain is the domain specified when you view My Domain in Setup.

  6. Use the access_token value in this response for the Authorization header in your API requests. For example:

When you have a valid token, you can call any of the Models REST API endpoints.

The Models REST API provides four different endpoints for four different capabilities.

CapabilityEndpoint PathDescription
Generate Chat/models/{modelName}/chat-generationsGenerate a response based on a list of messages representing a chat conversation.
Generate Embeddings/models/{modelName}/embeddingsCreate an embedding vector representing the input text.
Generate Text/models/{modelName}/generationsGenerate a response based on the prompt provided.
Submit Feedback/feedbackSubmit feedback for generated text.

Choose a model to use when you call the generations or embeddings endpoints. Each endpoint, other than the feedback endpoint, requires the API name of the model as part of the path. For instance, if you want to use OpenAI’s GPT-3.5 Turbo model for a generation request, you must specify the API name sfdc_ai__DefaultOpenAIGPT35Turbo by making a request to this endpoint.

The Models API supports models from various Salesforce-enabled providers. See Supported Models for Models API.

These headers are relevant for API requests.

HeaderDescriptionExample
AuthorizationRequired. Authorization method for this request. This header must contain your JWT.Bearer {token}
Content-TypeRequired. Specify the application/json content type.application/json
x-sfdc-app-contextRequired. This value is reserved for future use. For now, specify EinsteinGPT.EinsteinGPT
x-client-feature-idRequired. This value is reserved for future use. For now, specify ai-platform-models-connected-app.ai-platform-models-connected-app

Make a POST request to one of the Models REST API endpoints. This example demonstrates a call to the generations endpoint.

And here’s a sample response.

After you successfully call the API, refer to these topics to optimize your implementation.