Audio Models

StepFun audio models cover speech understanding, speech synthesis, and speech recognition, providing the building blocks for voice assistants, smart customer service, audiobooks, A/V production, and other audio-driven experiences. This page summarizes what each model does; see the individual model pages for specifications and quick-start code.

​

Choosing a model

​

StepAudio 2.5 Chat

An end-to-end speech-understanding model served through an OpenAI-compatible Chat Completion API. It accepts audio or text input and returns a text response, interpreting not just the words but also vocal cues such as intonation, hesitation, and laughter.

• Paralinguistic understanding: captures a speaker’s emotional state and intent from vocal cues that a text transcript does not represent.
• Unified text and audio input: typed and spoken turns share the same endpoint, so one conversation can mix both.
• For spoken replies, pass the text output to a text-to-speech model such as StepAudio 2.5 TTS.

Common use cases: voice assistants and conversational agents, call and meeting analysis, and voice message triage.

​

StepAudio 2.5 Realtime

An end-to-end speech-to-speech model served over a WebSocket Realtime API. It takes streaming audio or text and responds with streaming audio plus a matching text transcript, with no separate transcription or synthesis stage in between, keeping latency low enough for natural back-and-forth conversation.

• Paralinguistic understanding: interprets vocal cues such as intonation, pacing, and hesitation in addition to the spoken words.
• Natural turn-taking: server-side voice activity detection (VAD) lets the model reply at the right moment and be interrupted mid-response.
• System voices: a set of English system voices selectable per session; see the voices list.

Common use cases: voice agents for customer support, live hands-free assistants, and interactive phone and IVR systems.

​

StepAudio 2.5 TTS

A contextual text-to-speech model that integrates contextual understanding into the full speech generation pipeline. Instead of matching preset tags, you describe the delivery you want in natural language, at two levels:

• Global context sets the overall mood, scene, and character relationships for an entire passage.
• Inline context fine-tunes how individual words and phrases are delivered.

Layered, mixed delivery intents are supported, such as “restrained sadness, no sobbing, with a slight tremble”. The model renders pauses, stress, rhythm, and tonal transitions accordingly, producing speech with natural breathing, dynamic emphasis, and emotional arcs. Zero-shot voice cloning: about 3 seconds of reference audio is enough to clone a voice precisely, and cloned voices keep full global and inline context control, with no fixed voice library or preset characters required. Common use cases: audiobooks, dubbing, ad narration, emotional storytelling, and other content that demands high vocal expressiveness.

​

StepAudio 2.5 ASR

A new-generation speech recognition model built on a 4B-parameter Multi-Token Prediction (MTP) architecture, maintaining SOTA transcription accuracy while sharply reducing latency. Supports Chinese and English recognition with ITN text normalization. It offers two access methods:

• One-shot recognition (stepaudio-2.5-asr): submit audio over HTTP + SSE and receive the transcription streamed back incrementally.
• Real-time bidirectional streaming (stepaudio-2.5-asr-stream): stream audio over WebSocket and receive incremental and final results with per-word timing; supports server-side VAD.
• Fits live captions, voice input, meeting transcription, and back-end batch processing.

​

Usage limits

• Text-to-speech input: up to 1,000 characters per request.
• Text-to-speech output formats: wav, mp3, flac, opus, pcm; the default is mp3.

​

Next steps