Many spy movies feature iconic scenes where intelligence agents rely on innate talent to distinguish Morse code amidst countless radio statics, even identifying each operator's unique transmission habits, using their "sharp ears" to uncover enemy stations.
Those "sharp ears" from the stories have finally been replicated by AI in reality. For instance, in a classic spy movie clip, the protagonist dodges enemies on a street filled with diverse and complex sound sources. Such sounds are not only difficult for ordinary people to judge but also pose a challenging problem for intelligent voice systems. What exactly is happening on this street? How many people and objects are making sounds? What are the relationships between them?
For an intelligent voice system, "this problem" involves the perception, recognition, understanding, and comprehensive reasoning of various sound elements, essentially maximizing the difficulty index.
However, when such sound sources are input into the SparkAudio Foundation Large Model, it can easily complete the comprehensive perception and precise reasoning of human voices and various acoustic clues.
This raises a question: Intelligent voice is the most familiar AI capability, ubiquitous and ever-present. Can it evolve again?
iFlytek believes that the intelligent voice track still holds infinite potential. During the iFlytek Global 1024 Developer Conference in 2026, the company released its latest progress in intelligent voice technology, including the SparkAudio 1.0 voice foundation large model, as well as improvements in various voice technical capabilities based on this foundation model.
After decades of development, intelligent voice technology has achieved high adaptability in numerous scenarios. However, it still faces clear technical bottlenecks in complex real-world scenarios and advanced application requirements. Let us delve into this story of breaking through technical barriers.
The "Ear of All Spirits" forged by iFlytek is helping AI realize an endless journey from hearing to understanding in the AI era.
In 1952, Bell Labs released Audrey, the world's first speech recognition system. Since then, intelligent voice technology, as the most fundamental and important branch of AI, began its long seventy-year journey.
Over these seventy years, intelligent voice has gone through stages such as the Hidden Markov Model (HMM) era, Deep Neural Networks (DNN), and the end-to-end sequence modeling Transformer era. Today, our mobile phones, speakers, wearable devices, and home appliances all possess speech recognition and semantic understanding capabilities, building the interaction cornerstone of an intelligent society over decades.
But has this most widespread and foundational technology truly reached perfection? The answer is likely no.
The reason is that intelligent voice technology has only achieved "hearing," but remains distant from true "understanding." Three typical challenges stand in the way of advancing intelligent voice technology:
1. The challenge of parsing complex sound sources.
In the early days of intelligent voice technology, AI only needed to recognize speech in single-person silent environments. But as the industry developed to today, parsing complex sound sources has become a key requirement. In noisy or multi-speaker dialogue scenarios, humans can naturally focus on the target speaker's voice, but AI struggles to precisely separate target sounds from interference. Multi-speaker dialogues, complex background noises, and environmental sounds bring new tests to intelligent voice technology.
2. The barrier of semantic understanding.
For intelligent voice technology, "hearing" is one thing; "understanding" is another. In real natural language understanding scenarios, users often employ expressions involving contextual associations, topic switching, indirect references, self-corrections, colloquialisms, and disordered syntax. These issues make it difficult for models to accurately understand user intent, causing significant困扰 (trouble) in real-time interactions and simultaneous interpretation scenarios.
3. The dilemma of robustness.
Models exhibit extremely high accuracy in laboratory environments, but error rates skyrocket in real outdoor, factory, and automotive settings, revealing robustness issues. In applications with strict latency requirements like real-time translation, live subtitles, and in-car commands, weak model robustness is amplified. Another issue with poor robustness is that large-scale implementation requires specialized model adaptation and parameter tuning for numerous scenarios, leading to high costs and low promotion efficiency, ultimately hindering industrial application progress.
Ultimately, the real, complex acoustic world remains a wall of sighs for intelligent voice technology. Models are like flowers in a greenhouse, easily exposing their fragility when encountering "storms" of dense noise and complex sound sources.
Returning to the opening scene: How does intelligent voice solve the historical challenge of real-world domains and complex sounds?
In fact, iFlytek achieved a breakthrough in intelligent voice technology with the SparkAudio1.0 voice foundation large model. Revisiting the spy movie identification case discussed earlier. In this scenario, SparkAudio 1.0 must combine sound recognition, temporal relationships, semantic understanding, and scene inference to provide a holistic interpretation of audio containing multiple sound sources.
At the level of multi-source perception, the model must simultaneously capture music, human voices, mechanical operation sounds, metal collisions, and bells in the film's background, distinguishing different sound categories within the mixed audio.
At the level of voice content understanding, it needs to identify colloquial contents such as "Hey kid, clean that up," "Popcorn," and "Watch the car," differentiating between hawking, daily communication, and reminders.
In the stage of organizing temporal order, the model must arrange audio content according to the sequence of sound occurrences, describing how various sounds alternate or accompany each other, rather than just providing isolated labels.
Next, the model must model soundscape relationships, linking hawking calls, vehicle bells, and "watch out" reminders to understand that they may collectively form a soundscape of a street or mixed pedestrian-vehicle environment.
After this series of actions, the model provides a comprehensive judgment of the voice scene: integrating human voices, mechanical sounds, and environmental clues to infer the overall scenario of street vendors and vehicle activities. It is evident that truly understanding sound elements and relationships in complex environments requires meticulous analysis and layered reasoning, demanding precise perception and understanding of scenarios with overlapping sounds, dispersed information, and context-dependent interpretation.
What SparkAudio 1.0 achieves represents the shift from "hearing" to "understanding."
The AI's ability to replicate the intelligence agent's "innate talent" is backed by a comprehensive breakthrough achieved by the SparkAudio 1.0 voice foundation large model.
SparkAudio 1.0 supports text and voice modal inputs and text modal outputs, completing tasks such as speech transcription, multilingual translation, dialect recognition, environmental sound recognition, speaker diarization, sentiment analysis, and complex audio Q&A. Its performance is particularly outstanding in audio perception and reasoning capabilities for key scenarios like multilingual recognition, emotions, and sound events.
The key to achieving these goals lies in three major technical innovations implemented by SparkAudio 1.0, resulting in better performance, faster response, and lower cost intelligent voice capabilities.
1. Semi-supervised Audio Encoder (SemiEnc): During pre-training, the quantizer input is upgraded from raw acoustic features to enhanced representations extracted by specialized models built with supervised data. By introducing information priors from different dimensions and employing multi-task training, the encoder's modeling capability is significantly strengthened, improving robustness in tasks like speech recognition and enhancing overall task performance.
2. Dynamic Segmentation End-to-End Modeling (S-S2S): Based on end-to-end speech-to-speech modeling technology, it relies on a closed-loop feedback decision process fusing speech understanding and generation. This breaks the temporal barriers between recognition, understanding, and generation modules, supporting incremental streaming semantics-driven early generation and dynamic feedback correction. While ensuring effectiveness, it significantly reduces response latency for simultaneous interpretation and real-time voice interaction, meeting the low-latency demands of real-time voice services.
3. Dual-Paradigm Collaborative Decoding (CoDec): The audio encoder first performs parallel decoding via a non-autoregressive CTC Decoder to generate results at once. Then, an LLM autoregressive module uses speculative decoding to determine if intervention is needed and where to start autoregressive correction. This approach ensures effectiveness while drastically reducing the average inference latency and computational cost of the voice large model.
Based on these technical innovations, according to test data disclosed by the publisher, SparkAudio 1.0 achieved a 20% improvement in overall effectiveness, a 15% reduction in response time, and a 60% reduction in cost. This means better model performance, less waiting time, and lower barriers to scaled application.
Leveraging the upgrade of the foundation model, iFlytek's intelligent voice capabilities have seen multidimensional upgrades and expansions, introducing the Spark-ASR-2.0 recognition large model, the next-generation Spark-LiveTranslation simultaneous interpretation large model, and the Spark-Audio-Realtime interaction large model. This creates a three-dimensional landscape of comprehensive capability enhancement in recognition, simultaneous interpretation, and interaction, driven by the foundation model upgrade.
In terms of recognition, the Spark-ASR-2.0 speech recognition large model has made significant progress. Through key technical innovations such as non-autoregressive and LLM-enhanced autoregressive collaboration, joint enhancement of Chinese-English mixed text and acoustics, and dynamic context injection, overall speech recognition performance has improved substantially, with notable enhancements in general recognition, complex acoustic scenario recognition, contextual recognition, and text fluency and standardization. In terms of simultaneous interpretation, the Spark-LiveTranslation large model has achieved full-dimension improvements in scenario perception, long-history understanding, broadcast length control, and robustness in complex scenarios, synergistically enhancing translation quality and real-time experience. For example, its MOS score increased from 4.4 to 4.6, while S2S response time was shortened.
In terms of interaction, the Spark-Audio-Realtime voice interaction large model has revolutionized interaction, bringing significant improvements across various interactive scenarios.
The final result—more accurate recognition, superior simultaneous interpretation experience, and comprehensively iterated interaction capabilities—demonstrates that the barriers of intelligent voice have been broken anew. This technological journey has far from reached its end.
When AI can understand the real sounds of a complex world, the "Ear of All Spirits" of the Spark Large Model will eventually become the source of all industries.
In the process of AI driving productivity revolutions and lifestyle innovations, we have repeatedly witnessed the immense socio-economic value leap brought by intelligent voice. As technical barriers break again, the surging industrial value becomes inevitable.
The first step is that upgrades in intelligent voice technology will unlock potential through iFlytek's product layout. According to current information, products like iFlytek Hearing and Smart Office Tablets will integrate the voice foundation large model.
Take the newly upgraded iFlytek Smart Office Tablet powered by SparkAudio 1.0 as an example. On the product side, after handwritten annotation of speaker names, subsequent speeches automatically correspond; in meeting scenarios, it accurately identifies complex situations like interrupting, overlapping multi-speaker discussions; and it more accurately recognizes mixed Chinese-English speech and professional terminology. These upgrades in intelligent voice capabilities align perfectly with rigid demands in current AI trends, exemplifying the integration of AI technology into real office scenarios.
The newly released iFlytek AI Recording Ring brings voice capabilities to more portable recording scenarios, achieving two hours of recording transcription in just two minutes, with a comprehensive recognition accuracy exceeding 96%.
The second step looks toward broader industrial spaces. In automotive, robotics, outdoor, and industrial scenarios, intelligent voice capabilities in complex acoustic domains are key industrial hubs. For instance, in smart cockpits, precise identification of complex sound sources allows the vehicle system to more accurately recognize driver commands, enabling precise in-car intelligent services. As AI glasses become a new hardware trend, the ability to accurately identify sounds in outdoor spaces and perform high-quality simultaneous interpretation greatly enhances user experience, becoming a driving force for new hardware forms.
There are many such examples. Intelligent voice is the cornerstone of the intelligent world and an indispensable link in the industrial intelligence chain. The breakthroughs in intelligent voice completed by iFlytek will drive thousands of industries. This technological journey also implicitly builds industrial barriers.
Voice is humanity's most basic mode of communication and the starting point of AI exploration. This track has always held endless possibilities. Past decades of voice technology breakthroughs have made this technology highly usable and economically valuable, truly bringing AI to millions of households.
On this endless road, iFlytek has always served as the pioneer in the queue, undertaking the critical task of exploring the voice gold mine for Chinese technology. Walking the path of breaking technical barriers, empowering industries, and forging its own unique path.
iFlytek believes that increasingly developing application scenarios impose higher demands on the effectiveness and performance of voice technology. Only through systematic innovation, deeply integrating key technologies such as software-hardware integration, far-field high-noise multi-speaker voice enhancement and recognition, ultra-human-like voice synthesis and versatile voice cloning, and multimodal full-duplex interaction, can next-generation intelligent voice technology truly land in real-world scenarios.
At the current stage, the intelligent voice track, with higher technical requirements and stricter industrial demands, makes players capable of breakthroughs increasingly scarce. iFlytek, committed to the voice track, is both the earliest entrant and most extensively deployed enterprise in China's intelligent voice sector, while continuously deepening industrial accumulation and actively leading a new round of technological innovation.
In 2024, the "Key Technologies and Industrialization of Multilingual Intelligent Voice" project led by iFlytek won the First Prize of the National Science and Technology Progress Award, demonstrating significant value. Core intelligent voice technologies such as the voice foundation large model, recognition large model, ultra-human-like synthesis plus versatile voice cloning, and multimodal interaction systems continue to land. This year, iFlytek has won the CHiME-9 International Speech Recognition Challenge for six consecutive times and ranked first in TASK 1 – Multimodal Context-aware Recognition (MCoRec). It is safe to say that through its advantages in intelligent voice technology, iFlytek has constructed a full-chain advantage from foundation models and vertical models to product systems. Standing on these advantages and moving forward constantly has ultimately forged the "Ear of All Spirits."
Intelligent voice constitutes iFlytek's solid industrial barrier and technological moat, as well as its unwavering endless journey. Now, iFlytek has locked onto new travel destinations: more complex sounds and more realistic scenarios, while promoting the landing and expansion of more AI technologies through the fusion of voice and text large models.
When AI can truly "understand"—understanding the hustle and bustle in sounds, the most urgent commands from users, and the most real voices of industries—a new journey begins. We do not know where the endpoint of intelligent voice lies, but we can be certain that today it enables more wonderful changes.
With an Ear of All Spirits, we can hear every tremor of wind and chirp of insects in the intelligent world.
That might be the voice from the future.