OpenAI compatibility
Use the OpenAI SDK with AllModels for speech generation, transcription, and voice conversations.
AllModels implements the audio portion of the OpenAI SDK surface. Point the official SDK at https://api.allmodels.io/oai and use your AllModels API key.
Supported SDK surface
| Capability | OpenAI SDK method | AllModels route | Caveats |
|---|---|---|---|
| Generate speech | audio.speech.create() | POST /oai/audio/speech | stream_format: "sse" emits only speech.audio.delta and speech.audio.done events, and is rejected for tts-1 and tts-1-hd. |
| Transcribe a file | audio.transcriptions.create() | POST /oai/audio/transcriptions | stream: true is honored only when the selected provider streams file transcription natively; other models return unsupported_streaming. Adds a non-standard diarized_json response format. |
| Transcribe live audio | realtime.connect() (Python) / OpenAIRealtimeWebSocket (JavaScript) | WS /oai/realtime | Transcription intent only. intent must be absent or transcription; anything else returns unsupported_intent. |
| Voice conversation (Realtime) | realtime.connect() (Python) / OpenAIRealtimeWS (JavaScript) | WS /oai/realtime | A speech-to-speech model in model selects a conversation. See OpenAI realtime. |
| Voice conversation (Live) | live.connect() (Python) / LiveWS (JavaScript) | WS /oai/live/sessions | gpt-live-1 and gemini-3.8-live; the model is named in the first session.start frame. See OpenAI Live. |
/oai/realtime serves transcription or a voice conversation: intent=transcription or a transcription model selects transcription, and a speech-to-speech model in model selects a conversation.
Configure the client
import OpenAI from "openai";
const client = new OpenAI({
apiKey: process.env.ALLMODELS_API_KEY,
baseURL: "https://api.allmodels.io/oai"
});The SDK does not append /v1 when the base URL is set explicitly, so requests land on /oai/audio/speech, /oai/audio/transcriptions, /oai/realtime, and /oai/live/sessions. The key is sent as Authorization: Bearer. Keep it on a server; do not expose a long-lived tenant key in browser code.
Models and providers
Use the model names returned by GET /v1/providers. Canonical names such as deepgram/nova-3 select both the provider and the model, so a plain model string is usually all you need.
Provider preferences travel in the request body as provider and provider_options. The JavaScript SDK passes unknown body fields through; Python requires extra_body:
await client.audio.speech.create({
model: "elevenlabs/eleven-turbo-v2-5",
voice: "pNInz6obpgDQGcFmaJgB",
input: "Route this through Fal.",
response_format: "mp3",
provider: { only: ["fal"], allow_fallbacks: false },
provider_options: { speed: 1.1 }
});Every provider feature is available through provider_options; see Provider options. For realtime transcription, set provider and provider_options in the session.update event instead. See Providers for every provider option.
Not implemented
AllModels is an audio API. The rest of the OpenAI SDK is not implemented:
| Area | OpenAI SDK methods |
|---|---|
| Text and reasoning | chat.completions, responses, embeddings, moderations |
| Images | images.generate, images.edit, images.createVariation |
| Storage and jobs | files, uploads, batches, fineTuning, vectorStores, containers |
| Assistants | assistants, threads |
| Audio gaps | audio.translations |
| Discovery | models.list, models.retrieve |
| Realtime extras | beta.realtime.sessions / realtime.clientSecrets (ephemeral keys), WebRTC transport |
Any other path under /oai returns 501 with the code not_implemented in the OpenAI error envelope, plus an x-should-retry: false header so the SDK does not retry it.
AllModels does expose its own model discovery at GET /v1/models, but the response is an AllModels-shaped { "tts": [], "stt": [], "speech_to_speech": [] } payload rather than the OpenAI { "data": [] } list, and it is not reachable from an SDK pointed at /oai. Call it directly, or use GET /v1/providers. See Models.
Handle errors
Failed requests return the OpenAI envelope { "error": { "message", "type", "param", "code" } }, which the SDK raises as an API status error. Authentication and balance errors (401, 402, 403) are evaluated before the not-implemented check, so a keyless call to an unimplemented path fails with 401 rather than 501. See Errors for status codes and retry guidance.