Realtime TTS

Realtime TTS v2 is the canonical WebSocket protocol for new Kitta AI and Kitta Audio integrations. It streams MessagePack events over one WebSocket connection and keeps each generation tied to a caller-provided idempotency key.

Endpoints and protocol

Every connection must request the WebSocket subprotocol realtime.tts.msgpack.v2. Client and server application messages are MessagePack binary frames, not JSON text frames.

Existing /v1/tts/live integrations remain available for compatibility, but v1 is frozen. New integrations should use v2 and the contract on this page.

Site capabilities

The public voice_id must support the selected model. FishAudio supports speed, volume, language, emotion, chunk_length, and latency on CN. MiniMax supports speed, volume, language, and emotion. Qwen returns 24 kHz mono 16-bit PCM and supports language, emotion, instructions, and optimize_instructions. The server rejects combinations outside the runtime capability matrix.

Authentication

Backend clients should authenticate during the WebSocket handshake:

Authorization: Bearer API_KEY
Sec-WebSocket-Protocol: realtime.tts.msgpack.v2

The browser WebSocket API cannot set an Authorization header. A trusted browser tool may connect with the v2 subprotocol and send this as its first MessagePack frame:

{ event: 'auth', token: 'API_KEY' }

Send the frame before the authentication timeout. Never place an API key in the URL: token and api_key query parameters are rejected. Production browser applications should keep long-lived API keys on a backend and use a short-lived credential exchange.

Session flow

connect → authenticated → start → ready
        → text → flush → audio (one or more) → usage
        → [text → flush → audio → usage] → stop → finish

One WebSocket connection carries one generation request. stop flushes any remaining buffered text and completes the session. Reconnecting creates a new session and does not resume an interrupted provider stream.

Client events

All objects are strict and fields are case-sensitive.

request_id is required on start, must contain 1–128 characters, and is returned unchanged on request-level server events. It is also the idempotency key: after a v2 session has been created for an account, reusing the same request_id is rejected with invalid_state.

start.request parameters

Unsupported fields or provider combinations return unsupported_parameter or model_not_found; they are never silently ignored.

Example:

{
  event: 'start',
  request_id: crypto.randomUUID(),
  request: {
    voice_id: '00000000-0000-0000-0000-000000000000',
    model: 'fishaudio-s21pro-flash',
    format: 'mp3',
    speed: 1,
    volume: 0,
    language: 'en',
    chunk_length: 200,
    latency: 'normal'
  }
}

Server events

audio.audio is a byte array inside the MessagePack frame. Do not decode it as Base64. Concatenate audio frames in sequence order for the selected output format. When format is pcm, both ready and audio include the required sample_rate, channels, and bit_depth fields so the client can construct a WAV header or initialize a PCM player.

Error codes

Every error event has a stable code, a safe message, and retryable. The current codes are:

Minimal Node.js client

Install ws and @msgpack/msgpack, then wait for each protocol phase:

import { randomUUID } from 'node:crypto';
import { writeFileSync } from 'node:fs';
import { decode, encode } from '@msgpack/msgpack';
import WebSocket from 'ws';

const requestId = randomUUID();
const audio = [];
const ws = new WebSocket('wss://realtime.fishaudio.org/v2/tts/live', 'realtime.tts.msgpack.v2', {
  headers: { Authorization: `Bearer ${process.env.API_KEY}` },
});

ws.on('message', (frame) => {
  const event = decode(frame);

  if (event.event === 'authenticated') {
    ws.send(
      encode({
        event: 'start',
        request_id: requestId,
        request: {
          voice_id: process.env.VOICE_ID,
          model: 'fishaudio-s21pro-flash',
          format: 'mp3',
        },
      }),
    );
  } else if (event.event === 'ready') {
    ws.send(encode({ event: 'text', text: 'Hello from Realtime TTS v2.' }));
    ws.send(encode({ event: 'flush' }));
    ws.send(encode({ event: 'stop' }));
  } else if (event.event === 'audio') {
    audio.push(event.audio);
  } else if (event.event === 'error') {
    console.error(event.code, event.message, event.retryable);
  } else if (event.event === 'finish') {
    writeFileSync('realtime-output.mp3', Buffer.concat(audio.map((chunk) => Buffer.from(chunk))));
    console.log({ requestId: event.request_id, units: event.units, chunks: audio.length });
    ws.close(1000);
  }
});

Limits, billing, and retries

Each non-empty flush creates an independently billed segment. Quota is reserved before provider generation and committed when the first valid audio frame is delivered. A segment that fails before audio delivery is refunded. Once audio has been delivered, do not automatically replay that segment because doing so can duplicate both audio and usage.

The protocol defaults are 2,000 characters per text event, 10,000 characters per flushed segment, and 100,000 billable units per session. Deployments can enforce a lower session or concurrency limit. Treat server errors as authoritative because limits may evolve.

Retry only when retryable is true or the connection closes with 1012 or 1013. Use exponential backoff with jitter. Generate a new request_id only for a new user-intended generation; do not bypass duplicate protection to replay an uncertain request.

Handshake failures are HTTP responses before 101 Switching Protocols. Session failures are MessagePack error events followed by a WebSocket close.