Data Engineering, Automated Evaluation, and Knowledge Graph Integration in Large Model Development

1. Data Engineering for Large Models The discussion begins with the concept of data‑centric AI, describing training‑data pipelines (collection, labeling, preprocessing, augmentation), inference‑data development, and data maintenance, and outlines key questions such as data needs, sources, processing, evaluation, and versioning.

2. Review of Existing Large Models and Their Pre‑training Data It surveys the evolution of GPT‑1/2/3, LLaMA, BLOOM, PaLM, and other models, detailing parameter scales, data sources (Common Crawl, WebText, Wikipedia, books, The Pile, etc.), multilingual capabilities, and differences between English and Chinese corpora.

3. Automated Model Evaluation Three evaluation paradigms are introduced: human‑in‑the‑loop business assessment, downstream task benchmarks (e.g., BIG‑bench, MMLU, C‑EVAL), and automated scoring using LLMs like ChatGPT, complemented by crowd‑sourced voting systems such as Elo‑rated leaderboards.

4. Integration of Knowledge Graphs The article compares knowledge graphs and LLMs, emphasizing graph‑based reasoning, interpretability, and schema‑driven data organization, and describes how LLMs can generate schemas, annotate data, perform entity extraction, and enhance KG‑based QA.

5. End‑to‑End Solutions and Workflows Practical pipelines are presented, including data cleaning (deduplication, toxicity filtering), tokenization, quality scoring, site filtering, and multi‑stage pipelines for pre‑training, SFT, and post‑training interventions (prompt augmentation, knowledge verification, external tool integration via LangChain, search engines, and specialized knowledge bases).

6. Q&A Highlights The session concludes with questions on multimodal data handling (tables, images) and industry practices for mitigating hallucinations, stressing SFT for instruction understanding, knowledge injection during pre‑training, and external knowledge‑base augmentation.