WatsonX

IBM watsonx.ai exposes foundation models through text generation and chat APIs. Model availability varies by region and account.

Supported Models

IBM watsonx.ai provides foundation models through its inference API. The promptfoo WatsonX provider supports text generation and chat models that can be called directly via API.

Finding Available Models

To see the models available in your region, use IBM's API or review IBM's supported foundation models:

curl "https://us-south.ml.cloud.ibm.com/ml/v1/foundation_model_specs?version=2024-05-01" \
  -H "Authorization: Bearer YOUR_TOKEN"

Currently Available Models

The examples use ibm/granite-4-h-small. Confirm its availability with the model-specs endpoint above, or select another text/chat model from your region's catalog.

Other Model Types

IBM watsonx.ai also offers:

Deploy on Demand Models - Curated models that require creating a dedicated deployment first
Embedding Models - For generating text embeddings (e.g., ibm/granite-embedding-278m-multilingual)
Reranker Models - For improving search results (e.g., cross-encoder/ms-marco-minilm-l-12-v2)
Vision and Guardrail Models - Models with APIs or payloads that differ from the provider's current text/chat workflow

Additional Model Types Not Currently Supported

The promptfoo WatsonX provider focuses on text generation and chat models only. Deploy on Demand, embedding, and reranker models use different API endpoints and workflows. For these model types, use IBM's API directly or create a custom provider.

Model Availability

Region-specific: Model availability varies by IBM Cloud region
Version changes: IBM regularly updates available models
Deprecation: Models marked "deprecated" will be removed in future releases

Always verify current availability using IBM's API or check your watsonx.ai project's model catalog.

Prerequisites

Before integrating the WatsonX provider, ensure you have the following:

IBM Cloud Account: You will need an IBM Cloud account to obtain API access to WatsonX models.
API Key or Bearer Token, and Project ID:
- API Key: You can retrieve your API key by logging in to your IBM Cloud Account and navigating to the "API Keys" section.
- Bearer Token: To obtain a bearer token, follow this guide.
- Project ID: To find your Project ID, log in to IBM WatsonX Prompt Lab, select your project, and locate the project ID in the provided curl command.

Make sure you have either the API key or bearer token, along with the project ID, before proceeding.

Installation

The WatsonX SDKs are optional peers and are not installed with promptfoo by default. To use this provider:

Install promptfoo and the supported SDK versions together in your project:
```
npm install promptfoo @ibm-cloud/watsonx-ai@^1.7.16 ibm-cloud-sdk-core@5.6.2
npm install --save-exact ibm-cloud-sdk-core@5.6.2
```
For a global installation, add -g to both commands so both SDKs are installed alongside promptfoo. Missing or incompatible SDKs are reported when the WatsonX provider is used.

The core SDK is temporarily restricted to 5.6.2 to avoid the debug-log redaction regression introduced in 5.6.3. Version 5.6.2 also has known redaction gaps, so avoid IBM SDK HTTP debug logging with credentials or sensitive data (DEBUG=ibm-cloud-sdk-core:* or NODE_DEBUG=axios). Promptfoo checks the core version resolved by the WatsonX SDK too, including nested installations.

For Yarn, also add this resolution to your project's root package.json before installing the SDKs. This pins WatsonX's transitive core dependency too; resolutions in an installed package do not apply to your project.
```
{
  "resolutions": {
    "ibm-cloud-sdk-core": "5.6.2"
  }
}
```
Set up the necessary environment variables:

You can choose between two authentication methods:

Option 1: IAM Authentication (Recommended)
```
export WATSONX_AI_APIKEY=your-ibm-cloud-api-key
export WATSONX_AI_PROJECT_ID=your-project-id
```
Option 2: Bearer Token Authentication
```
export WATSONX_AI_BEARER_TOKEN=your-bearer-token
export WATSONX_AI_PROJECT_ID=your-project-id
```
Force Specific Auth Method (Optional)
```
export WATSONX_AI_AUTH_TYPE=iam  # or 'bearertoken'
```
Authentication Priority
If WATSONX_AI_AUTH_TYPE is not set, the provider will automatically use:
1. IAM authentication if WATSONX_AI_APIKEY is available
2. Bearer token authentication if WATSONX_AI_BEARER_TOKEN is available

Alternatively, you can configure the authentication and project ID directly in the configuration file:

providers:
  - id: watsonx:ibm/granite-4-h-small
    config:
      # Option 1: IAM Authentication
      apiKey: your-ibm-cloud-api-key

      # Option 2: Bearer Token Authentication
      # apiBearerToken: your-ibm-cloud-bearer-token

      projectId: your-ibm-project-id
      serviceUrl: https://us-south.ml.cloud.ibm.com

Usage Examples

Once configured, you can use the WatsonX provider to generate text responses based on prompts. Here's an example using the Granite 4 H Small model:

promptfooconfig.yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
providers:
  - watsonx:ibm/granite-4-h-small

prompts:
  - "Answer the following question: '{{question}}'"

tests:
  - vars:
      question: 'What is the capital of France?'
    assert:
      - type: contains
        value: 'Paris'

You can also use other models by changing the model ID:

providers:
  # Replace this ID with a text/chat model returned by the model-specs endpoint.
  - watsonx:ibm/granite-4-h-small

Configuration Options

Text Generation Parameters

The WatsonX provider supports the full range of text generation parameters from the IBM SDK:

Parameter	Type	Description
`maxNewTokens`	number	Maximum tokens to generate (default: 100)
`minNewTokens`	number	Minimum tokens before stop sequences apply
`temperature`	number	Sampling temperature (0-2)
`topP`	number	Nucleus sampling parameter (0-1)
`topK`	number	Top-k sampling parameter
`decodingMethod`	string	`'greedy'` or `'sample'`
`stopSequences`	string[]	Sequences that cause generation to stop
`repetitionPenalty`	number	Penalty for repeated tokens
`randomSeed`	number	Seed for reproducible outputs
`timeLimit`	number	Time limit in milliseconds
`truncateInputTokens`	number	Max input tokens before truncation
`includeStopSequence`	boolean	Include stop sequence in output
`lengthPenalty`	object	Length penalty configuration

Example with Parameters

providers:
  - id: watsonx:ibm/granite-4-h-small
    config:
      temperature: 0.7
      topP: 0.9
      topK: 50
      maxNewTokens: 1024
      stopSequences: ['END', 'STOP']
      repetitionPenalty: 1.1
      decodingMethod: sample

Length Penalty

For more control over output length:

providers:
  - id: watsonx:ibm/granite-4-h-small
    config:
      lengthPenalty:
        decayFactor: 1.5
        startIndex: 10

Cost estimates

Cost estimates use pricing metadata for the configured service and account. If required metadata is unavailable or a required pricing tier is unrecognized, the response omits cost and still returns the generated output. A token category with a reported count of zero does not require a price.

For account-specific pricing, set inputCost and outputCost in USD per token, or set cost for a shared input/output rate. These values override the built-in tier prices; an explicit zero is supported. Estimates do not determine whether your account can use a model.

Chat Mode

WatsonX also supports chat-style interactions using the textChat API. Use the watsonx:chat: prefix:

providers:
  - id: watsonx:chat:ibm/granite-4-h-small
    config:
      temperature: 0.7
      maxNewTokens: 1024

Chat mode automatically parses messages in JSON format:

prompts:
  - |
    [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "{{question}}"}
    ]

providers:
  - watsonx:chat:ibm/granite-4-h-small

For plain text prompts, the chat provider automatically wraps them as a user message.

Chat vs Text Generation

Feature	Text Generation (`watsonx:`)	Chat (`watsonx:chat:`)
API Method	`generateText`	`textChat`
Input Format	Plain text	Messages array or plain text
Best For	Completion tasks	Conversational applications
System Messages	Not supported	Supported

Environment Variables

Variable	Description
`WATSONX_AI_APIKEY`	IBM Cloud API key for IAM authentication
`WATSONX_AI_BEARER_TOKEN`	Bearer token for token-based authentication
`WATSONX_AI_PROJECT_ID`	WatsonX project ID
`WATSONX_AI_AUTH_TYPE`	Force auth type: `iam` or `bearertoken`

Migrating from IBM BAM

The IBM BAM provider has been deprecated (sunset March 2025). To migrate:

Change provider prefix from bam: to watsonx:
Update authentication to use WatsonX credentials
Update model IDs to WatsonX equivalents (e.g., ibm/granite-4-h-small)

Supported Models​

Currently Available Models​

Other Model Types​

Prerequisites​

Installation​

Usage Examples​

Configuration Options​

Text Generation Parameters​

Example with Parameters​

Length Penalty​

Cost estimates​

Chat Mode​

Chat vs Text Generation​

Environment Variables​

Migrating from IBM BAM​