Built for voice agents that can't wait 600ms
Sign up freeSign in

OpenAI: GPT Realtime 2.1

openai/gpt-realtime-2-1
active speech-to-speech 1 provider
Playground

OpenAI realtime reasoning model for speech-to-speech conversations with configurable reasoning effort, instruction following and tool use.

AuthorOpenAI
Languagesnot specified
Response formatsStreaming: PCM · µ-law · A-law
Statusactive

Providers

Different providers can host the same model. Choose one provider when you need a fixed backend, or let routing select among them.

ProviderSlugPer 1M tokensLatencyiFor TTS, latency is measured as time to first byte of audio. For STT, latency is measured as time to first token.ModesUptime
OpenAI 3 params
openai
$64
—
Streaming

Supported languages

No reviewed language list for this model yet.

Features

Speech to speech

Bidirectional realtime speech-to-speech over WebSocket or WebRTC.

speech_to_speech
Text input and output

Accepts text input and returns text output alongside audio.

text_input_output
Image input

Accepts image input in the conversation.

image_input
Turn detection

Server-side voice activity detection (server_vad) or semantic turn detection (semantic_vad).

turn_taking
Tool use

Function calling during the conversation.

tool_use
Input transcription

Optional transcription of input audio by a separate transcription model.

input_transcription
Configurable reasoning effort

Reasoning effort is configurable per session.

reasoning_effort

Code

Language

Realtime speech-to-speech with the OpenAI SDK

Sends question.pcm (24 kHz 16-bit mono PCM) as one turn over the OpenAI Realtime protocol and saves the spoken reply to reply.pcm.

import { once } from "node:events";
import { readFile, writeFile } from "node:fs/promises";
import OpenAI from "openai";
import { OpenAIRealtimeWS } from "openai/realtime/ws";

const client = new OpenAI({
  apiKey: process.env.ALLMODELS_API_KEY,
  baseURL: "https://api.allmodels.io/oai"
});
const realtime = new OpenAIRealtimeWS({ model: "gpt-realtime-2.1" }, client);
const reply = [];

realtime.on("response.output_audio.delta", ({ delta }) => reply.push(Buffer.from(delta, "base64")));
realtime.on("response.done", async () => {
  await writeFile("reply.pcm", Buffer.concat(reply));
  realtime.close();
});
realtime.on("error", console.error);

if (realtime.socket.readyState !== 1) await once(realtime.socket, "open");
const format = { type: "audio/pcm", rate: 24000 };
realtime.send({
  type: "session.update",
  session: {
    type: "realtime",
    output_modalities: ["audio"],
    audio: { input: { format, turn_detection: null }, output: { format, voice: "marin" } }
  }
});
realtime.send({ type: "input_audio_buffer.append", audio: (await readFile("question.pcm")).toString("base64") });
realtime.send({ type: "input_audio_buffer.commit" });
realtime.send({ type: "response.create" });