Original Title: AI Begins to Take Over Data Production | Ifenxi Interview
Data production methods are being transformed by AI. In mature scenarios such as speech, language, and text, over 90% of data can now be automatically processed by models. As data agents further evolve, with one agent responsible for production and another for quality inspection, humans will gradually shift from inside the production closed-loop to outside it, moving from item-by-item annotation to rule-making, process supervision, and result acceptance. Meanwhile, in high-standard scenarios such as healthcare, education, and post-training for models, even if machines complete 99% of the work, the final 1% must still be backed up by experts.
Along with the changes in production methods, the boundaries of data services are also expanding simultaneously. In the past, data annotation mainly relied on humans labeling images, speech, and text; now, data vendors need to start from demand planning, completing collection, governance, annotation, quality inspection, and ongoing operations, ultimately delivering high-quality datasets that can directly enter model training. The value of data companies has also shifted towards building automated production capabilities, organizing industry experts, and steadily improving model performance. The predecessor of Feishu can be traced back to the speech production department established by iFLYTEK in 2001. In April 2024, the relevant data team incubated Anhui Feishu Data, transforming over twenty years of data production experience into an automated platform and an expert network.
Tan Chang, Executive President of Feishu, summarizes Feishu's capabilities as high difficulty, high knowledge density, and high efficiency. Currently, Feishu has reserved thousands of expert resources and has gradually extended its business from education and healthcare to scientific research and industrial manufacturing. Post-training for large models and embodied intelligence are bringing a new round of data demand. Tan Chang judges that large-scale basic data will be the first to experience a production peak, followed by a shift towards competition in hard cases, real-machine data, and high knowledge density data.
In this exchange with Ifenxi, Tan Chang systematically analyzed the judgment criteria for high-quality data, the division of labor between automation and experts, the market cycle of embodied data, and the path for Feishu to build long-term capabilities relying on its technology platform and expert network.
Core Insights
- Data annotation is upgrading to complete data production.
What AI enterprises need are data products that can be directly used for model training, and the scope of services has already covered planning, collection, governance, annotation, quality inspection, and ongoing operations.
- Model performance is the ultimate standard for high-quality data.
Accuracy, completeness, and timeliness are basic indicators; whether data can improve model performance determines its actual value.
- Machines can complete 99%, but experts must hold the final 1%.
Automation will continue to compress manual workload, but in high-standard scenarios, experts still need to set rules, handle long-tail errors, and accept results.
- Embodied data will shift from scale competition to knowledge density competition.
In the field of embodied data, basic data will first experience a production peak, followed by a shift towards hard cases, real-machine data, and data with high accuracy and high knowledge density.
- The data industry must be rooted in industries.
Local data bases need to be combined with advantageous industries to form long-term capabilities supported by real data, industry experts, and application scenarios.
The following is the transcript of this interview, slightly modified without changing the original meaning.
01. Data Annotation Is Upgrading to Data Production
Ifenxi: What was the consideration behind incubating Feishu from the iFLYTEK system?
Tan Chang: The predecessor of Feishu was data teams within iFLYTEK, such as the AI Resource Department and the Big Data Research Institute, which have a history of over 20 years. At the end of 2023, against the backdrop of the establishment of the National Data Administration and the increasingly clear policy orientation of data elements, iFLYTEK's decision-making layer independently incubated the data business and established Anhui Feishu in April 2024, focusing on the data field. From the very beginning, Feishu positioned itself as a data technology service provider. Its business chain covers the entire process of raw data collection, processing and governance, manual annotation, and deep processing of AI pre-annotation. The ultimate goal is to transform data resources into marketable data assets.
Ifenxi: How is the current data annotation industry different from the past?
Tan Chang: In the past, when mentioning data annotation, it was easy to understand it as humans labeling images, speech, or text. Now, the industry boundaries have expanded to the complete data processing process including screening, cleaning, classification, annotation, marking, and quality inspection. Traditional annotation, post-training data for large models, expert data, and embodied intelligence data all belong to this industry. What model vendors are looking for is no longer a single annotation supplier, but a data supplier. They expect suppliers to complete the processing of raw data and deliver data that can directly enter model training, minimizing subsequent work. Therefore, data annotation is upgrading to complete data production, and the final product is a high-quality dataset that can support model development and capability enhancement.
Ifenxi: What is the difference between Feishu and traditional labor-intensive annotation companies?
Tan Chang: Feishu will not take low-threshold, labor-intensive annotation as its main direction, but will leverage the accumulated advantages in technology, industry knowledge, and production organization. We summarize our capabilities into three highs: high difficulty, high knowledge density, and high efficiency. High difficulty and high knowledge density mean entering professional fields such as speech, multilingual, education, healthcare, and judiciary, where corresponding experts transform industry knowledge into data usable by models. High efficiency means quickly organizing the right people to complete tasks. A project manager in the past can now manage a production team of hundreds, or even coordinate more than a dozen suppliers. Some projects require organizing 300-500 people within a month, while some tasks have only 100 pieces of data but require finding a few professor-level experts. The two types of tasks differ in scale, both testing the organizational capabilities of data companies. After customers put forward their needs, Feishu can start from dataset planning, complete collection, governance, annotation, quality inspection, and delivery, and provide ongoing supplementation, correction, and optimization as models and scenarios change. High-quality datasets are usually not delivered at once, but operated alongside model iterations. Our positioning is a producer of high-quality data for AI. When customers express their demand intentions, Feishu starts from the planning and design of data solutions, completing the full lifecycle services from collection, governance, annotation, quality inspection to delivery, and providing ongoing operation optimization.
02. Model Performance Defines High-Quality Data
Ifenxi: What kind of data can be considered high-quality?
Tan Chang: Accuracy, completeness, and timeliness are basic indicators of data quality. A more critical standard is whether the data can be directly used for model development and training, and effectively improve model performance. Judging data quality requires starting with the end in mind. Accuracy, completeness, and timeliness are direct evaluation indicators, while the effect after model training is the ultimate evaluation. What truly determines whether a batch of data is high-quality is whether it can improve model performance. Whether data has been manually annotated does not directly determine if it is high-quality. Unannotated data, as long as it is properly processed, can be directly used for training and improve model performance, and can also become high-quality data. Therefore, the value of data cannot be judged independently of the model and application goals. I once joked that even if a piece of data seems somewhat incomprehensible to humans, as long as it actually improves model performance, its value to the model should be acknowledged. This statement is somewhat exaggerated, but it emphasizes that data must be evaluated in the context of model performance.
Ifenxi: How can industry vertical data be classified?
Tan Chang: Industry vertical data is roughly divided into three levels: general knowledge data, industry general knowledge data, and enterprise proprietary data. General knowledge data is mainly used for basic model pre-training. After learning, the basic model is like a high school student with a broad range of knowledge, but not deep enough in each field. Industry general knowledge data can make the model an entry-level expert in a certain industry. Through methods such as RAG (Retrieval-Augmented Generation) and SFT (Supervised Fine-Tuning), adding industry data such as chemical, steel, mathematics, or code, the model's capabilities will gradually focus on specific fields. Enterprise proprietary data comes from a company's operations, production, and management processes, including technical documents, process records, and internal experience. Only after learning this data can the model further become an expert who truly understands the company. The three levels deepen step by step, and the knowledge density and degree of customization also steadily increase.
Ifenxi: Which data can be standardized, and which must be customized?
Tan Chang: Public general knowledge data and some industry general knowledge data have strong reusability and can be produced into standardized products. For example, basic data for the steel industry formed through public channels can serve multiple steel enterprises. Enterprise proprietary data, on the other hand, is highly customized. The ten-year operation record of a certain blast furnace is only directly effective for specific enterprises and equipment, and also involves process capabilities and trade secrets, so it can only be produced and used within the enterprise. Whether data can be reused at scale depends not only on the universality of knowledge but also on property rights and authorization methods. Generally speaking, if enterprises regard data as a competitive advantage, it will remain in the private domain for a long time. However, scientific research institutions, through desensitization, processing, and authorization, may also transform internal data into industry data products. The depth of data application mainly depends on quality and knowledge density, while the breadth of application is affected by property rights, authorization methods, and the willingness of holders. In addition, standardized products and enterprise customized services will coexist for a long time.
03. Machines Complete 99%, Experts Hold the Final 1%
Ifenxi: To what extent can Feishu's data production achieve automation?
Tan Chang: The technological maturity of different data varies greatly. In mature scenarios such as speech, language, and text, over 90% of the data can now enter training as weakly supervised data without manual annotation or processing. Some pre-training corpora, after being automatically processed by models, can serve this purpose. Weakly supervised data lies between supervised and unsupervised data. It is not meticulously annotated item by item by humans, nor is it completely unlabeled; instead, basic processing and pre-annotation are completed by models. The requirements for post-training data are higher. Much fine-tuning and reinforcement learning data still needs to be pre-processed by models first, and then reviewed and calibrated by annotators or industry experts. The degree of automation ultimately depends on data types, model capabilities, and customer quality requirements, and cannot be summarized by a single ratio for all scenarios.
Ifenxi: Will manual annotation disappear after model capabilities improve?
Tan Chang: Human workload will decrease, but in high-standard, high-stakes scenarios, it is hard for manual backup to disappear. For example, printed OCR (Optical Character Recognition) can approach full automation when the recognition rate exceeds 95%, but handwritten recognition in education and healthcare may still require manual correction. Algorithms may reach 97%, 98%, or even 99% in the future, but as long as the model still makes mistakes, the final errors need to be handled by humans. In scenarios such as healthcare and education, machines can complete 99% of the work, but the remaining 1% may still determine whether the system can enter real business. For instance, if the handwritten recognition accuracy is only 90% and the business requirement is 95%, the gap in between requires manual correction. The experts' workload may drop to 1%, but the quality responsibility they bear will not disappear accordingly.
Ifenxi: If data production can achieve such high automation, why do model companies still need professional data service providers?
Tan Chang: First, model companies cannot maintain long-term reserves of talent in all fields. Today they need Japanese, tomorrow Spanish; it is difficult for in-house teams to cope with fluctuating demands, whereas data companies can concentrate professional talent and serve multiple clients simultaneously. A data team can reserve a hundred Japanese professionals and serve twenty model enterprises at the same time, thus forming the advantages of professional talent and scaled organization. Second, large-scale data processing requires a professional production system. Processing hundreds of millions of pieces of data or hundreds of TBs of data every day necessitates the establishment of dedicated teams, processes, and quality systems. Third, professional division of labor is forming between model training and data production. In the early days, iFLYTEK kept its algorithm and data teams in-house. Later, the data team gradually developed into a professional service company that can serve different model vendors and industry clients simultaneously. Models continue to iterate, business scenarios constantly change, and data also needs ongoing supplementation, correction, and optimization, so high-quality data construction will not be a one-time job.
Ifenxi: What role will humans play in the data production process in the future?
Tan Chang: Humans will gradually shift from inside the production closed-loop to outside it. One agent is responsible for production, and another agent with different ideas and architecture is responsible for quality inspection; the two continuously generate, check, and correct. The production and quality inspection agents cannot be completely isomorphic, otherwise they might make mistakes in the same place. After the two agents reach the preset standards, humans will conduct the final review. Data engineers are mainly responsible for two things: setting rules and accepting results. Generally, they first define the meta-rules for data production, and then conduct sampling checks and final acceptance of the results generated by data agents. For example, after a data agent generates 10,000 pieces of compliant data, the engineer no longer participates in production item by item, but checks the samples and overall quality to avoid humans becoming the slowest link in the assembly line. Machines are responsible for scaled production, while humans are responsible for defining quality and bearing ultimate responsibility. Future data engineers and industry experts will increasingly become designers, supervisors, and acceptors of the assembly line.
04. Embodied Data Will Shift from Scale Competition to Quality Competition
Ifenxi: What value does synthetic data have for data production? Can it replace real data?
Tan Chang: Traditional data production includes collection, governance, annotation, and quality inspection. Synthetic data first changes the collection method by directly generating data according to goals and rules, which can also reduce subsequent governance and annotation work. Quality inspection still cannot be canceled. Synthetic data may have errors, deviations, and unreasonable distributions. As long as the generative model cannot guarantee 100% accuracy, the final results need to be checked. Data engineers and industry experts will shift towards designing, operating, and accepting synthetic data pipelines. In the future, models with different architectures can be responsible for generation and inspection respectively, with humans outside the closed-loop setting standards, supervising operations, and accepting results. Synthetic data changes the production method but will not cancel quality responsibility.
Ifenxi: Has a consensus been formed on the technical route of embodied intelligence?
Tan Chang: The technical route is still developing, but a preliminary consensus has emerged at the data level. The first stage requires large-scale simulation and first-person perspective data as basic data similar to pre-training corpora; the second stage uses real-machine teleoperation and actual production data to complete post-training, fine-tuning, and capability enhancement. Currently, the number of real machines is limited, and related data is mostly collected by in-house teams of robotics enterprises, with little data flowing to the external market. External service providers mainly participate in large-scale simulation and non-embodied data production, with the focus of competition on cost and delivery efficiency. Some leading enterprises have begun to separate model teams and data teams, establishing dedicated data companies, but the overall field is still in its early stages.
Ifenxi: How will this market develop next?
Tan Chang: Large-scale basic data production will experience an obvious peak, but it will not necessarily maintain the same scale in the long term. After pre-training data crosses the scale threshold, the demand for newly added low-difficulty data will decline. The industry still has different judgments on the specific threshold, but the shift of basic data from rapid growth to falling demand is a repeatedly occurring rule in the field of data production. This rule is very common in the data industry. After new models and new scenarios emerge, basic data grows rapidly first; after basic capabilities are formed, demand shifts towards hard cases, long-tail scenarios, and high-precision data. Embodied intelligence competes on scale in the early stage, and on complex tasks, accuracy, and knowledge density in the later stage.
Ifenxi: How does Feishu cope with this phased change?
Tan Chang: Feishu will participate in the current large-scale collection and production, but will not build long-term capabilities solely on low-difficulty scaled production. Large-scale production helps the team establish processes and tools. After demand declines, the team needs to further shift to high-difficulty, high-knowledge-density tasks. Feishu pays more attention to whether the team can gradually upgrade from a scaled collection unit to a high-quality data production unit. The embodied data market competes on scale and cost in the early stage, but in the long run, it still depends on technology, experts, and quality control capabilities. Basic data has cycles, while high-difficulty data and ongoing optimization will exist for a long time.
05. The Data Industry Must Be Rooted in Industries
Ifenxi: How do you view the development prospects of this round of data annotation bases?
Tan Chang: In the early days, some data annotation bases did not have a clear industry direction. Various places mainly compared rent, subsidies, and labor costs, and ultimately introduced similar labor-intensive enterprises, which were prone to fall into homogeneous competition. High-quality data increasingly relies on industry knowledge. If a city's advantageous industry is steel, rather than introducing ten traditional autonomous driving annotation enterprises, it is better to introduce a data service provider that truly understands steel production. Only by entering the local industry can data service providers access real data, business experts, and specific scenarios, and transform them into industry data products. The data industry cannot just build parks; it must be rooted in industries. Previously, Feishu participated in the work of relevant data annotation bases in Hefei and Nanjing, with the focus also on promoting the combination of data technology, expert resources, and local industries.
Ifenxi: What is the relationship between public data operation and high-quality dataset construction?
Tan Chang: The two are already merging. Taking intelligent transportation as an example, a complete dataset requires both data held by enterprises and data on roads, vehicles, and public transportation held by government departments. Relying on only one type may result in incomplete data. There is also a large amount of public data in the healthcare and education fields. The construction of high-quality datasets needs to connect public data, enterprise-owned data, and market data under the premise of compliant authorization, safe use, and quality governance. This fusion has already begun to happen. Cities and enterprises undertaking related tasks are promoting public data operation and industry high-quality dataset construction within the same system.
Ifenxi: What does Feishu plan to rely on to build long-term barriers?
Tan Chang: The core barriers of Feishu are technology and talent. In terms of technology, we are promoting the construction of an intelligent data production platform, deeply integrating capabilities such as AI pre-annotation, automated quality inspection, process scheduling, and expert calibration into a unified platform. Through platform-based tools, we increase the proportion of automated processing in standardized links, releasing human resources to focus on core tasks with high difficulty and high knowledge density. At the same time, we strengthen the decision-making role of experts in rule-making, key calibration, and final acceptance, forming an efficiency barrier that attaches equal importance to scaled organizational capabilities and high-quality delivery capabilities. We will continue to build automated annotation, data production agents, and platforms. Enterprises with automated assembly lines will have advantages in efficiency, quality, and cost compared to those relying on manual workshops. In terms of talent, Feishu will consolidate and expand its high-knowledge-density expert network. We will steadily reserve high-quality expert resources in advantageous fields such as education and healthcare, forming rapid response capabilities to deal with high-difficulty and high-knowledge-density tasks, and building a positive flywheel where "the stronger the model capabilities, the deeper the demand for high-quality expert-annotated data."
Ifenxi: Which customers is Feishu currently focusing on serving?
Tan Chang: Currently, there are mainly three types of customers. The first type is AI model vendors, for whom Feishu provides customized services according to model training needs; the second type is universities, research institutes, and laboratories, for whom we produce relatively standardized high-quality data for scientific research large models; the third type is industrial manufacturing enterprises, around whom we build industry general knowledge data and enterprise proprietary data for industries such as chemical and metallurgy. Regardless of whether the demand comes from model vendors or end enterprises, the goal is to improve the performance of vertical models through industry data and enterprise data. Scientific research institutions prefer standardized datasets, while industrial enterprises need to transform their own processes, documents, and experience into enterprise proprietary data. The delivery methods for different customers vary, but all require ongoing data production and quality operations. In the future, Feishu will continue to provide data services around model and business performance. The technology platform improves production efficiency, and the expert network solves high-knowledge-density tasks; the two together form the foundation for long-term development.