OpenAI Live

AllModels serves the GPT-Live protocol for two models: openai/gpt-live-1 and gemini/gemini-3.8-live. Name the model in the first frame, session.start, not in the URL. Use an openai SDK version that includes LiveWS (JavaScript) or client.live.connect() (Python).

Start a session

Send session.start, stream microphone audio with session.input_audio.append, and play session.output_audio.delta. To end, send session.close and stop at session.closed.

import OpenAI from "openai";
import { LiveWS } from "openai/resources/live/ws";

const client = new OpenAI({
  apiKey: process.env.ALLMODELS_API_KEY,
  baseURL: "https://api.allmodels.io/oai"
});
const live = new LiveWS(client);

live.on("session.output_audio.delta", ({ delta }) => {
  // Play Buffer.from(delta, "base64"): 24 kHz, 16-bit, mono PCM.
});
live.on("session.closed", ({ reason, usage }) => {
  console.log(`closed: ${reason}, ${usage.seconds} s`);
  live.close();
});
live.on("error", ({ error }) => console.error(error));

live.send({
  type: "session.start",
  session: {
    model: "gpt-live-1", // or "gemini-3.8-live"
    instructions: "You are a helpful voice assistant. Keep replies short.",
    audio: {
      format: { type: "audio/pcm", rate: 24000 },
      output: { voice: "marin" } // for gemini-3.8-live, a Gemini voice such as "Zephyr"
    },
    delegation: { type: "client" }
  }
});
// Stream microphone audio as session.input_audio.append, about every 100 ms.

For a complete example that streams a recorded question and saves the spoken reply, see Speech to speech.

Answer delegations

When the model hands a task to your app, it sends session.delegation.created. Send progress with session.thinking.append, which the model hears but does not speak, and the answer with session.commentary.append, which it speaks. Use delegation.id as delegation_id.

live.on("session.delegation.created", async ({ delegation }) => {
  live.send({ type: "session.thinking.append", delegation_id: delegation.id, content: "Checking the calendar." });
  const answer = await checkCalendar(); // your code
  live.send({ type: "session.commentary.append", delegation_id: delegation.id, content: answer });
});

The event carries no task text: read the request from the conversation (session.input_transcript.delta).

Differences from OpenAI

• Client delegation only. session.delegation must be absent or {"type": "client"}. Responses and Decisions delegation, Decision triggers and store: true are refused with an error and close 1008.
• Unsupported events. response.* and decision.* client events are answered with an error; the session stays open.
• Correlation. Send event_id on a client event to get it back as client_event_id on its acknowledgement or error.
• gemini-3.8-live:
  • session.update accepts only delegation; other fields return unknown_parameter.
  • session.output_audio.fixture.play is not supported.
  • session.instructions.append and session.thinking.append need a delegation id; null is accepted only on session.commentary.append.
  • The user's transcript arrives after they finish speaking, and the model waits for them to finish before it answers.
  • session.usage.updated has no context_window.

Events

Fields and examples for each event: client events and server events. Voices, audio formats and billing: Speech to speech.