Tencent Gander Splits the Brain to Stop Voice AI Stuttering

What happened

Most voice assistants freeze the moment you ask them to think. You give an order, the speaker goes dead, and you stare at a spinning circle while a massive transformer grinds through tokens on a remote server rack. Tencent thinks it solved that awkward dead air by cutting its new system into two distinct pieces.

Developed by the Tencent Hunyuan Speech division alongside university researchers, Gander is a multimodal speech architecture designed to talk while it executes background labor. The system ingests audio, streaming video, and raw text in a continuous loop. Instead of forcing one model to choke down conversation timing and deep code planning at once, Gander divides the human nervous system into two components: a cerebellum and an interchangeable brain.

The cerebellum handles the conversational cadence. It operates on discrete one-second windows, evaluating whether to keep speaking, yield the floor, or jump into a conversational gap without needing a dedicated voice activity detection module. It relies on roughly two minutes of short-term conversational context to decide when the user actually finished a sentence.

Behind that twitchy front end sits the swappable cognitive engine. In benchmark tests, the Tencent team dropped an unspecified model from OpenAI's GPT-5.6 family into the background slot to act as the thinking core. That heavy model can write code, triage bugs, or parse documents without dragging down the speech delivery latency.

The split design treats conversational speech as an autonomic reflex while leaving the actual thinking to whatever heavy engine sits on the backend.

The benchmark numbers show an immediate behavioral shift. Running across all 100 evaluation scenarios in Full-Duplex-Bench v3, Gander hit conversational entry marks on time every single time. It stepped on the user's words in just 8 percent of conversational turns, undercutting the 13.5 percent interruption rate of OpenAI's GPT-Realtime and dismantling the nearly 48 percent collision rate logged by the weakest system in the benchmark pool.

Yet the structural division carries a price tag. Because Full-Duplex-Bench v3 tallies speech transcription failures and synthesis hitches against the overall agent score, Gander slipped behind the lowest-ranked commercial competitor on final task accuracy. When researchers stripped out the voice interface and fed raw text prompts straight into the backend engine, accuracy surged right back up.

Perception tasks took an even harder hit. When evaluated on basic spatial checks like spotting item coordinates in a visual frame or counting items on a desk, Gander performed worse than its own foundational base model. The researchers conceded that tuning a model to nail conversational rhythm actively degraded its visual and audio comprehension precision.

Tencent trained the setup on approximately 2.7 million multimodal examples. A substantial portion of that dataset focused exclusively on teaching the system to shut up when background babble fills a room or when multiple humans converse nearby without directing queries to the hardware.

The background

Voice agents have historically relied on ping-pong turn-taking. You wait for an endpoint chime, recite a command, wait out the round-trip latency, and endure an unbroken block of synthesized monologue. Real human conversations do not operate on clean half-duplex radio protocols; they feature overlapping speech, vocal acknowledgments, and sudden interruptions.

Tencent has been systematically constructing an enterprise agent footprint to break that rigid pattern. In July, the company rolled out Hy3, an open-weights foundational language model engineered to close the functional performance gap with foreign frontier labs on tool-use benchmarks. Tencent immediately integrated Hy3 across its domestic software empire, shipping it to production users in its WorkBuddy productivity client, the Yuanbao chatbot, and the core WeChat ecosystem.

At the same time, Tencent moved to secure monopoly control over sovereign agent infrastructure. After regulators in Beijing blocked Meta from acquiring the autonomous agent outfit Manus, Tencent stepped in to negotiate the controlling equity stake. That prospective deal gives Tencent the exact agent chassis it needs to deploy autonomous background workflows directly inside WeChat's billion-user messaging surface.

The race to disconnect response latency from deep inference has fractured into three competing engineering schools:

• Dedicated fast-slow orchestrators like OpenAI's GPT-Live, which splits instantaneous spoken patter from remote web queries and asynchronous compute.
• Pool-based routing frameworks like Sakana AI's Fugu, which operates as a lightweight supervisory model that dynamically summons secondary specialized models from an expanding bench.
• Autonomous background daemons, such as experimental OpenAI agents designed to assign themselves forward-looking follow-ups and initiate contact without an inbound wake word.

Human behavioral data shows why labs are desperate to get this balance right. Telemetry published by Anthropic revealed that seasoned developers interrupt Claude Code during roughly 9 percent of operational steps, while first-time users intervene only 5 percent of the time. Users who actually rely on agents treat them like erratic interns: they cut them off the moment they steer off course.

What people are saying

Tencent's research team framed the split architecture as the only pragmatic exit from the latency-reasoning bottleneck. Their technical findings argue that forcing a monolithic transformer to ponder multi-step logic while streaming low-latency phonemes inevitably degrades one capability or the other. By turning the speech module into a low-overhead scheduler, the team maintained that developers can freely swap in specialized coders like Claude Code or OpenAI Codex without touching the speech weights.

Industry engineers running commercial voice stacks view the results with far more skepticism. Survey data across enterprise conversational AI teams confirms that real-time latency and audio drift remain the primary operational failure points in production. To those builders, an 8 percent barge-in error rate is an improvement over GPT-Realtime, but a system that flunks object counting and misinterprets user instructions due to audio compression errors is an unacceptable liability.

Tencent's researchers acknowledged in their disclosures that evaluating mixed full-duplex agents remains broken across the industry. Standard language benchmarks test static textual reasoning, but there is no uniform standard that effectively scores the delicate social dynamics of when a voice agent should cut in, when it should hold back, and when it should stay completely silent during ambient room chatter.

What happens next

Tencent is preparing to drop the complete model weights and training datasets for Gander into the open-source wild once internal legal clearance finishes. The team already staged the bare GitHub repository and launched static product demonstrations on the project page. Releasing the core checkpoint weights gives open-source researchers an unvetted blueprint to wire their own local language models into an interruption-aware voice interface.

The commercial roadmap points straight to WeChat. If Tencent integrates Gander's cerebellum directly into the application runtime, WeChat users will get an assistant that listens to voice notes, monitors screen shares, and executes code tasks in the background while sustaining a live voice conversation.

Scaling the architecture up remains an unsolved technical roadblock. Expanding the two-minute conversational sliding memory without ballooning local hardware compute will require tighter compression layers inside the cerebellum. The Tencent group must also repair the perceptual regression that cripples Gander during visual counting and localization tasks before this dual-model design can survive outside controlled benchmarks.