Supported Models for Models API

The Models API supports large language models (LLMs) from multiple providers, such as Amazon Bedrock, Azure OpenAI, OpenAI, and Vertex AI from Google.

In addition to standard and custom configured models from Einstein Studio, the API also supports these special types of models that are covered later in this guide.

Announcements for new models and model deprecations are part of the Einstein Platform release notes on a monthly basis.

Model deprecation is the process where a model provider gradually phases out a model (usually in favor of a new and improved model). The process starts with an announcement outlining when the model will no longer be accessible or supported. The deprecation announcement usually contains a specific shutdown date. Deprecated models are still available to use until the shutdown date.

After the shutdown date, you will not be able to use that model in your application and requests to that model will be rerouted to a replacement model. We recommend that you start migrating your application away from a model as soon as its deprecation is announced. During migration, update and test each part of your application with the replacement model that we recommend.

These models are deprecated and will be shut down in the future.

Deprecated ModelRecommended ReplacementDeprecated DateShutdown Date
Azure OpenAI GPT 3.5 Turbo 16kOpenAI GPT 3.5 Turbo2023-11-062024-10-01
OpenAI GPT 3.5 Turbo 16kOpenAI GPT 3.5 Turbo2023-11-062024-09-13
OpenAI GPT 4 32kTo be determined2024-06-062025-06-06

To access an LLM with the Models API, you must know its API name.

Most endpoints for the Models API require the model’s API name in the URL path. For example, to use the GPT 3.5 Turbo model with the /generations endpoint, the URL looks like this:

The API name is also required in the modelName property when making a Models API request using Apex. For example:

The API name is a string made up of substrings:

  • Namespace: sfdc_ai
  • Separator: __
  • Configuration name: Default
  • Provider name: OpenAI (not used with most geo-aware models)
  • Model name: GPT35Turbo (not used with BYOLLM and custom-configured models)

To look up the API name of any custom or standard model configuration in Einstein Studio:

  1. Go to the Models page.
  2. Click the Generative tab.
  3. Click the name of a configured model.
  4. The API name is shown in the configured model details.

Einstein Studio API Name

Salesforce-managed models are operated on Amazon Bedrock infrastructure entirely within the Salesforce Trust Boundary. In contrast, other models are operated by Salesforce partners, either inside a shared trust zone or through the LLM provider directly using Einstein Studio’s bring your own LLM (BYOLLM) feature.

The first available Salesforce-managed model is Claude 3 Haiku from Anthropic. To learn more about Anthropic and the Claude 3 family of models, jump to Anthropic in the About the Providers section.

This table lists the API names for all the standard configuration models in Einstein Studio, plus one API-only model. These models don’t support geo-aware routing. In addition to these models, you can use the API name from any custom model configuration in Einstein Studio.

ModelAPI NameNotes
Anthropic Claude 3 Haiku on Amazonsfdc_ai__DefaultBedrockAnthropicClaude3HaikuSalesforce-managed
Azure OpenAI Ada 002sfdc_ai__DefaultAzureOpenAITextEmbeddingAda_002Embeddings only
Azure OpenAI GPT 3.5 Turbosfdc_ai__DefaultAzureOpenAIGPT35Turbo
Azure OpenAI GPT 3.5 Turbo 16ksfdc_ai__DefaultAzureOpenAIGPT35Turbo_16kDeprecated
Azure OpenAI GPT 4 Turbosfdc_ai__DefaultAzureOpenAIGPT4TurboNot supported by Models API. Use BYOLLM instead.
OpenAI Ada 002sfdc_ai__DefaultOpenAITextEmbeddingAda_002Embeddings only
OpenAI GPT 3.5 Turbosfdc_ai__DefaultOpenAIGPT35Turbo
OpenAI GPT 3.5 Turbo 16ksfdc_ai__DefaultOpenAIGPT35Turbo_16kDeprecated
OpenAI GPT 4sfdc_ai__DefaultOpenAIGPT4Older model
OpenAI GPT 4 32ksfdc_ai__DefaultOpenAIGPT4_32kDeprecated
OpenAI GPT 4o (Omni)sfdc_ai__DefaultOpenAIGPT4OmniLatest frontier model
OpenAI GPT 4 Turbosfdc_ai__DefaultOpenAIGPT4TurboOlder model

The Models API doesn’t support OpenAI’s snapshot model names, such as gpt-3.5-turbo-0613. Always test your prompts to make sure that they perform as expected with new model versions.

Proximity to the nearest LLM server is determined by the region in which your Einstein generative AI platform instance is located. If you enabled the Einstein generative AI platform on or after June 13, 2024, then your Einstein generative AI platform region is the same as your Data Cloud region (Data Cloud: Data Center Locations). Otherwise, contact your Salesforce account executive to learn where it’s provisioned.

A geo-aware model automatically routes your LLM request to a nearby data center based on where Data Cloud is provisioned for your org. Geo-aware routing offers greater control over data residency, and using nearby data centers minimizes latency.

A geo-aware model can be routed to an older version of the model than you expect. To determine exactly which model versions are available in each region, review this page from Azure: Model summary table and region availability.

To learn more about geo-aware routing, see Geo-Aware LLM Request Routing in Salesforce Help.

This table lists the API names for all geo-aware models.

ModelAPI NameNotes
GPT 3.5 Turbosfdc_ai__DefaultGPT35Turbo
GPT 3.5 Turbo 16Ksfdc_ai__DefaultGPT35Turbo_16kDeprecated
GPT 4 Omnisfdc_ai__DefaultGPT4OmniLatest frontier model

For OpenAI models, requests are routed to a nearby data center provided by Azure OpenAI and hosted in one of its Azure availability zones.

If there is a problem with the nearby data center, requests are routed to a data center provided by OpenAI in the United States. This fallback routing to the United States can’t be disabled.

For Brazil, Canada, the United States, and all other countries where geo-aware routing is not yet supported, the request is routed directly to OpenAI in the United States.

The Trust Layer also has separate data residency regions for:

  • Data masking and toxicity detection models
  • Audit Trail data stored in Data Cloud

This table describes the countries and data center regions where data resides or passes through for geo-aware models from OpenAI, such as GPT 3.5 Turbo.

Data Cloud CountryTrust Layer CountryAzure Data Center RegionOpenAI Fallback Data Center
AustraliaAustraliaAustralia EastOpenAI East US 2
BrazilUnited States and Brazil*Not supportedNot supported
CanadaUnited StatesNot supportedNot supported
FranceGermanyFrance CentralOpenAI East US 2
IndiaIndiaSouth IndiaOpenAI East US 2
ItalyGermanyFrance CentralOpenAI East US 2
JapanJapanJapan EastOpenAI East US 2
GermanyGermanyFrance CentralOpenAI East US 2
SpainGermanyFrance CentralOpenAI East US 2
SwedenGermanyFrance CentralOpenAI East US 2
SwitzerlandGermanyFrance CentralOpenAI East US 2
United KingdomGermanyFrance CentralOpenAI East US 2
United StatesUnited StatesNot supportedNot supported
All othersUnited StatesNot supportedNot supported

*For Brazil, data masking models and toxicity detection models are hosted in the United States and Audit Trail data is hosted in Brazil.

This table describes the countries and Amazon Bedrock data center regions where data resides or passes through for geo-aware models from Anthropic, such as Claude 3 Haiku.

Data Cloud CountryTrust Layer CountryAmazon Bedrock Data CenterData Center ID
AustraliaAustraliaAsia Pacific (Sydney)ap-southeast-2
BrazilUnited States and Brazil*South America (São Paulo)sa-east-1
GermanyGermanyEU (Frankfurt)eu-central-1
IndiaIndiaAsia Pacific (Mumbai)ap-south-1
JapanJapanUS West (Oregon)us-west-2
United States (East)United StatesUS West (Oregon)us-west-2
United States (West)United StatesUS West (Oregon)us-west-2
All othersUnited StatesUS East (N. Virginia)us-east-1

When you bring your own LLM, you consume 30% fewer Einstein Requests compared to other models. For details, see Einstein Usage.

The Models API supports Einstein Studio’s bring your own LLM (BYOLLM) feature, which currently supports Amazon Bedrock, Azure OpenAI, OpenAI, and Vertex AI from Google as foundation model providers. With BYOLLM, you can add a foundation model from a supported provider, configure your own instance of the model, and connect to the model using your own credentials. Although inference is handled by the customer’s model, the request is still routed through the Models API and Trust Layer features are fully supported.

Using a BYOLLM model with the Models API is the same with any other model. Look up the API Name of the configured model in Einstein Studio and use it as the {modelName} in the REST endpoint path or as the modelName property of the Apex request object.

This table lists all the foundation models that you can add in Einstein Studio with BYOLLM.

ProviderModelNotes
Azure OpenAIOpenAI GPT 3.5 Turbo
Azure OpenAIOpenAI GPT 3.5 Turbo 16kDeprecated
Azure OpenAIOpenAI GPT 4 TurboOlder model
Amazon BedrockClaude 3 Haiku
Amazon BedrockClaude 3 SonnetVersion 3, not 3.5
Amazon BedrockClaude 3 Opus
OpenAIGPT 3.5 Turbo
OpenAIGPT 3.5 Turbo 16kDeprecated
OpenAIGPT 4Older model
OpenAIGPT 4 32kDeprecated
OpenAIGPT 4 TurboOlder model
Vertex AI (Google)Gemini Pro 1.5

To learn more about BYOLLM, see Bring Your Own Large Language Model in Einstein 1 Studio on the Salesforce Developers Blog.

For most tasks, choose a model that offers a balance of many criteria like Claude 3 Haiku or GPT 3.5 Turbo.

To choose the right model for your application, consider these criteria.

Capabilities: What can the model do? Advanced models can perform a wider variety of tasks (usually at the expense of higher costs and slower speeds—or both). The ability to follow complex instructions is a key indicator of model capabilities.

Cost: How much does the model cost to use? For details on usage and billing, see Einstein Usage.

Quality: How well does the model respond? The quality of model responses can be hard to measure quantitatively, but a good place to start is the LMSYS Chatbot Arena.

Speed: How long does it take the model to complete a task? Includes measures of latency and throughput.

For benchmarks and evaluations of LLMs and embedding models, see these resources.

The following models are available to use with the Models API.

The context window determines how many input and output tokens the model can process in a single request. The context window includes system messages, prompts, and responses.

The latest versions of GPT 3.5 Turbo and GPT 4 Turbo have a hard limit of 4,096 tokens on output, despite their extended context window for input. All models currently have a maximum context window of 32,768 tokens to ensure compatibility with Trust Layer features.

ProvidersModelGood ForContext Window
Amazon BedrockAnthropic Claude 3 HaikuMost tasks (balanced)32,768 tokens
Azure OpenAI, OpenAIAda 002Retrieval-augmented generation8,191 tokens
Azure OpenAI, OpenAIGPT 3.5 TurboMost tasks (balanced)16,385 tokens
Azure OpenAI, OpenAIGPT 3.5 Turbo 16kDeprecated16,385 tokens
Azure OpenAI, OpenAIGPT 4Deprecated8,192 tokens
Azure OpenAI, OpenAIGPT 4 32kDeprecated32,768 tokens
Azure OpenAI, OpenAIGPT 4o (Omni)Advanced tasks (latest frontier model)32,768 tokens
Azure OpenAI, OpenAIGPT 4 TurboAdvanced tasks (older model)32,768 tokens
Vertex AI (Google) via BYOLLMGemini 1.5 ProAdvanced tasks (latest model)32,768 tokens

Salesforce has partnered with several LLM providers to offer you a wide range of models to choose from. Learn more about each provider and what they have to offer.

Amazon Bedrock is a managed service by Amazon Web Services (AWS) for hosting LLMs from leading AI companies. Salesforce has partnered with Amazon to provide a Salesforce-managed version of Anthropic’s Claude 3 Haiku model. You can also use Claude 3 Haiku, Claude 3 Sonnet, and Claude 3 Opus via Amazon Bedrock with Einstein Studio’s BYOLLM feature.

Anthropic’s LLMs are designed to be interpretable, steerable, and aligned with human values. They use a novel approach to model alignment that they call “constitutional AI,” and their models are known for their long context windows. The version of Anthropic’s Claude 3 Haiku model that is supported by the Models API is managed by Salesforce entirely within the Trust Boundary.

To learn more about the Claude family of models, see Anthropic’s Claude page.

The Azure OpenAI service, offered by Microsoft, enables Salesforce to provide models developed by OpenAI with additional enterprise features that aren’t yet offered by OpenAI themselves. Features include:

  • Regional availability outside of the United States
  • Certified compliance with HIPAA, ISO27001, SOC 1, SOC 2 (type 1 and 2), and SOC 3
  • More formal processes and procedures for access control, data management, and security testing

To learn more about a particular model, see Azure OpenAI’s models overview.

OpenAI is one of the best-known AI labs due to the popularity of their ChatGPT product. Their GPT 4 series of models is focused on advanced capabilities, while the GPT 3.5 series is optimized for speed.

To learn more about a particular model, see OpenAI’s models overview.

Google’s Gemini Pro 1.5 model is available through their Vertex AI service and can be added as foundation model using Einstein Studio’s BYOLLM feature. Google’s models are the product of longstanding investment in AI research and vast amounts of data and compute. Their Gemini models are known for their advanced capabilities, long context windows, and excellent performance on recall tasks.

To learn more about the Gemini family of models, see Google’s Gemini models overview.