Translation management API

Control the Palabra real-time translation pipeline over WebSockets or a WebRTC data channel.

Prerequisites

• WebSockets — you only need an API Key. Optionally, you can create a streaming session yourself and use its ws_url and publisher token instead.
• WebRTC — create a streaming session first. The response contains the webrtc_url (the URL of the Palabra WebRTC Speech-to-Speech Translation API) and the publisher JWT access token used to authorize your connection.

Choose a transport

Option 1. WebRTC

0. Create a streaming session.
1. Connect to webrtc_url with any LiveKit client using your publisher token.
2. Wait for the Palabra translator to join the room — it connects a moment after you do, and your session room contains only you and the translator, so the first remote participant to join is the translator. It may also already be in the room by the time you connect, so check the current participant list before subscribing to the participant-connected event.
3. Once the translator is in the room, send commands through the default (empty-topic) WebRTC data channel.

A command published before the translator is in the room is not delivered to anyone — the room accepts the data message, but there is no one subscribed to receive it, so the first set_task is silently lost and the translation never starts. See Step 6 of the WebRTC Quick Start for a ready-made wait helper.

This applies to WebRTC only. Over WebSockets you talk to the server directly, so set_task can be sent as soon as the connection is open.

Option 2. WebSockets

Connect to the WebSocket endpoint, passing your API Key as the token query parameter (or in the Authorization header). The server validates the key and creates a streaming session for the lifetime of the connection automatically:

const randomHash = crypto.randomUUID(); // any random URL-safe string
const endpoint = `wss://streaming.palabra.ai/streaming-api/${randomHash}/v1/speech-to-speech/stream?token=${apiKey}`;
const socket = new WebSocket(endpoint);

Replace the {random_hash} path segment with any random URL-safe string generated by your client for the connection — it is used to distribute connections across translation servers.

Alternatively, create a streaming session yourself and connect to the returned ws_url with the session's publisher token:

const endpoint = `${ws_url}?token=${publisher}`;
const socket = new WebSocket(endpoint);

Once the connection is open, you can start sending commands.

Message format

Every API message — request and response — has the same envelope:

{
  "message_type": "<string>",
  "data": { /* payload */ }
}

If message_type is "error", the data field contains diagnostic information.

Typical workflow

1. Create a task — send set_task and wait for task_status with event: "started" and task_status: "running" before sending audio.
2. Update the task — send another set_task to change settings during an ongoing translation.
3. Pause processing — send pause_task to pause the translation (billing stops). Resume with another set_task.
4. Interrupt processing — send interrupt_task to instantly stop transcription, translation, and speech of the current phrase (e.g. the speaker got interrupted) without pausing the translation of subsequent phrases.
5. Finish the task — send end_task. No new audio is accepted after it; the server finishes processing everything it already received, sends an end_of_stream message and closes your connection automatically; the session is invalidated within 1 minute.

Settings reference

• What each field means — see the Translation settings breakdown.
• Recommended values — see the Recommended settings.

Rate limits

Exceeding a command limit returns an error message; exceeding the connection limit closes the connection with code 1008.

Streaming audio configuration

Option 1. WebRTC audio I/O

Use the following input/output configuration in your set_task:

{
  "data": {
    "input_stream": {
      "content_type": "audio",
      "source": {
        "type": "webrtc"
      }
    },
    "output_stream": {
      "content_type": "audio",
      "target": {
        "type": "webrtc"
      }
    }
    // ...
  }
}

1. Publish your microphone track to the LiveKit room.
2. Subscribe to the translated tracks that Palabra publishes to the same room after you send set_task.

Option 2. WebSocket audio I/O

Use this configuration instead:

{
  "data": {
    "input_stream": {
      "content_type": "audio",
      "source": {
        "type": "ws",
        "format": "pcm_s16le", // or opus, wav
        "sample_rate": 24000,  // 16000 – 48000
        "channels": 1          // 1 or 2
      }
    },
    "output_stream": {
      "content_type": "audio",
      "target": {
        "type": "ws",
        "format": "pcm_s16le" // or zlib_pcm_s16le
      }
    }
  }
}

The output is always 24000 Hz, mono (sample_rate and channels cannot be changed).

1. Send base64-encoded audio chunks that exactly match the declared format.
2. Receive base64-encoded TTS chunks in output_audio_data responses (see below).

API messages

Requests (client → server)

set_task

Create a new task or modify the current one:

• Sent for the first time after creating a session — starts the translation.
• Sent after pause_task — resumes the translation.
• Sent during an ongoing translation — updates the settings in real time, no need to stop the translation.

{
  "message_type": "set_task",
  "data": {
    "input_stream":  { /* Depends on transport — see "Streaming audio configuration" above */ },
    "output_stream": { /* Depends on transport — see "Streaming audio configuration" above */ },
    "pipeline": {
      "transcription": {
        "source_language": "string",
        "detectable_languages": ["string"],
        "segment_confirmation_silence_threshold": "float",
        "only_confirm_by_silence": "bool",
        "sentence_splitter": {
          "enabled": "bool"
         },
        "verification": {
          "auto_transcription_correction": "bool",
          "transcription_correction_style": "string"
        }
      },
      // Translation and speech generation settings for one or more target languages
      "translations": [
        {
          "target_language": "string",
          "translate_partial_transcriptions": "bool",
          "speech_generation": {
            "voice_cloning": "bool",
            "voice_id": "string",
            "voice_timbre_detection": {
              "enabled": "bool",
              "high_timbre_voices": [],
              "low_timbre_voices": []
            }
          }
        }
        // You can add more targets
      ],
      "translation_queue_configs": {
        "global": {
          "desired_queue_level_ms": "int",
          "max_queue_level_ms": "int",
          "auto_tempo": "bool"
        },
        "es": {
          "desired_queue_level_ms": "int",
          "max_queue_level_ms": "int"
        }
      },
      // Select response types to receive
      "allowed_message_types": [
        "translated_transcription",
        "partial_transcription",
        "partial_translated_transcription",
        "validated_transcription"
      ]
    }
  }
}

See the Translation settings breakdown for details on each field and the Recommended settings for best-practice values.

get_task

Return the current task (as a current_task response). Set exclude_hidden to false to include advanced (hidden) fields in the response.

{ "message_type": "get_task", "data": { "exclude_hidden": true } }

pause_task

Pause the current task. No audio is processed and no billing applies while the task is paused. Use set_task to resume.

{ "message_type": "pause_task", "data": {} }

interrupt_task

Instantly stop the speech (TTS) of the phrase currently being processed, without pausing upcoming ones. Useful when the current phrase no longer needs finishing — for example, if your conversation partner interrupts you. Unlike end_task, nothing is finalized: the current phrase is dropped.

{
  "message_type": "interrupt_task",
  "data": {
    "languages": ["global"], // 'global' (all languages, default) or target languages from `translations`
    "pause_task": false      // pause the task after interrupting
  }
}

end_task

Finish the current task. After end_task no new audio is accepted; the server finishes processing everything it already received (the tail of the translation is fully delivered), sends an end_of_stream message (see below) and then closes the connection automatically.

{
  "message_type": "end_task",
  "data": {
    "force": false // drop the pipeline and disconnect immediately, without processing the tail
  }
}

tts_task

Generates TTS from text within the active task. The behavior depends on translate_text (default false):

• "translate_text": false — TTS only. language is the language to speak in and must be one of the target_language values configured in the task's translations section (otherwise a VALIDATION_ERROR is returned).
• "translate_text": true — the text is first translated into every target_language of the task, then synthesized. language is the language of the input text and must be a valid source language.

text is limited to 2048 characters. See the Text-to-Translated-Speech guide for details.

{
  "message_type": "tts_task",
  "data": {
    "text": "Hello, how are you?",
    "language": "en",       // text language
    "translate_text": false // default
  }
}

input_audio_data

Send a base64-encoded input audio chunk when WebSockets is selected as the audio transport. The audio chunks you push must match the format / sample_rate / channels declared in your set_task command. The optimal chunk length is 320 ms. The base64 payload must be between 1 KB and 512 KB (≈384 KB of raw audio per message).

{
  "message_type": "input_audio_data",
  "data": {
    "data": "base64 encoded data"
  }
}

Responses (server → client)

partial_transcription

Transcription of an uncompleted segment:

{
  "message_type": "partial_transcription",
  "data": {
    "transcription": {
      "transcription_id": "19f84f5e10a20085",
      "language": "en",
      "text": "The weather is nice",
      "segments": [
        {
          "text": "The weather is nice",
          "start": 7.36,
          "end": 8.32,
          "start_timestamp": 1784642133.39,
          "end_timestamp": 1784642134.35,
          "words": [
            {
              "word": "The",
              "start": 7.36,
              "end": 7.42
            },
            {
              "word": "weather",
              "start": 7.54,
              "end": 7.90
            },
            {
              "word": "is",
              "start": 7.96,
              "end": 8.04
            },
            {
              "word": "nice",
              "start": 8.12,
              "end": 8.32
            }
          ]
        }
      ]
    }
  }
}

partial_translated_transcription

Translation of an uncompleted segment (optional):

{
  "message_type": "partial_translated_transcription",
  "data": {
    "transcription": {
      "transcription_id": "19f84f5e10a20085",
      "translation_part_id": 0,
      "language": "es",
      "text": "Hoy hace buen",
      "segments": [
        {
          "text": "Hoy hace buen",
          "start": 7.36,
          "end": 8.32,
          "start_timestamp": 1784642133.39,
          "end_timestamp": 1784642134.35,
          "words": [...] // the same as in `partial_transcription`
        }
      ]
    }
  }
}

validated_transcription

Transcription of a completed segment:

{
  "message_type": "validated_transcription",
  "data": {
    "transcription": {
      "transcription_id": "19f84f5e10a20085",
      "language": "en",
      "text": "The weather is nice today.",
      "segments": [
        {
          "text": "The weather is nice today.",
          "start": 7.36,
          "end": 8.74,
          "start_timestamp": 1784642133.39,
          "end_timestamp": 1784642134.77,
          "words": [
            {
              "word": "The",
              "start": 7.36,
              "end": 7.42
            },
            {
              "word": "weather",
              "start": 7.54,
              "end": 7.90
            },
            {
              "word": "is",
              "start": 7.96,
              "end": 8.04
            },
            {
              "word": "nice",
              "start": 8.12,
              "end": 8.32
            },
            {
              "word": "today.",
              "start": 8.40,
              "end": 8.74
            }
          ]
        }
      ]
    }
  }
}

translated_transcription

Translation of a completed segment:

{
  "message_type": "translated_transcription",
  "data": {
    "transcription": {
      "transcription_id": "19f84f5e10a20085",
      "translation_part_id": 0,
      "language": "es",
      "text": "Hoy hace buen tiempo.",
      "segments": [
        {
          "text": "Hoy hace buen tiempo.",
          "start": 7.36,
          "end": 8.74,
          "start_timestamp": 1784642133.39,
          "end_timestamp": 1784642134.77,
          "words": [...] // the same as in `validated_transcription`
        }
      ]
    }
  }
}

output_audio_data

TTS audio chunk (when WebSockets is the audio transport):

{
  "message_type": "output_audio_data",
  "data": {
    "transcription_id": "190983855fe3404e",
    "translation_part_id": "0",  // part index when a phrase is split into several TTS pieces
    "language": "es",            // TTS language
    "last_chunk": false,         // true on the last generated chunk for this `transcription_id`
    "data": "base64 string",     // ≤512 KB base64 (≈384 KB of raw audio)
    "chunk_generation_delta": 120 // optional, ms between generation start and this chunk
  }
}

The audio is always 24 kHz mono; the encoding (pcm_s16le or zlib_pcm_s16le) matches the output_stream.target.format of your task.

task_status

The server acknowledges each successful set_task, pause_task, interrupt_task, or tts_task command with a task_status message sent over the same connection. pipeline.allowed_message_types does not filter these responses. If a command is rejected, the server sends an error response instead. Your client must handle acknowledgements alongside incoming transcriptions and audio.

{
  "message_type": "task_status",
  "data": { "command": "set_task", "event": "started", "task_status": "running" }
}

Wait for the set_task acknowledgement before sending audio. A resumed response also confirms that the settings supplied in that command have been applied.

interrupt_task leaves a paused task paused. To also pause a running task, set data.pause_task to true in the interrupt_task command. Audio already buffered by the client may still play unless the client clears its playback buffer.

For tts_task, the acknowledgement confirms that the text is queued for processing. Translation and speech generation may still be in progress.

end_task does not return a task_status acknowledgement. Continue receiving messages until end_of_stream, which confirms that the server has finished processing pending input and output. If this processing times out, the server returns error and closes the connection without end_of_stream. With data.force: true, it discards pending work and closes the connection without end_of_stream.

get_task returns a current_task response. The server does not acknowledge individual input audio chunks.

To determine whether settings have changed, the server compares the complete validated configuration, including default values and advanced settings, with the running task. JSON object key order does not matter; array order does. If the configurations match, the server returns unchanged and keeps the existing pipeline and buffers. Sending the same configuration to a paused task returns resumed.

current_task

The response to the get_task command. Contains the current task configuration plus its status:

{
  "message_type": "current_task",
  "data": {
    "input_stream":  { /* ... */ },
    "output_stream": { /* ... */ },
    "pipeline":      { /* ... */ },
    "task_status": "running" // running | paused | unknown
  }
}

end_of_stream

End-of-stream confirmation. Sent after an end_task, once the remaining audio has been processed and delivered; the connection is closed right after.

{ "message_type": "end_of_stream", "data": {} }

warning

Audio stream health warnings. The server monitors the pace of your input_audio_data stream:

{
  "message_type": "warning",
  "data": {
    "code": "AUDIO_STREAM_TOO_FAST", // AUDIO_STREAM_TOO_FAST | AUDIO_STREAM_TOO_SLOW | AUDIO_STREAM_STALLED
    "message": "Human-readable description"
  }
}

Send audio at real-time rate (e.g., one 320 ms chunk every 320 ms): pushing faster, slower, or stopping mid-stream triggers these warnings and degrades translation quality.

error

Validation, authorization, or other kinds of errors:

{
  "message_type": "error",
  "data": {
    "code": "VALIDATION_ERROR",
    "desc": "ValidationError(model='SetTaskMessage', errors=[{'loc': ('input_stream', 'content_type')",
    "msg": "value is not a valid enumeration member; permitted: 'audio'\", 'type': 'type_error.enum'",
    "param": null
  }
}