What Real‑World LLM Researchers Face: Scaling Limits, Data Bottlenecks, and Deployment Challenges

Scaling and Resource Limits

Current research labs can realistically train dense LLMs up to ~100 B parameters. With personal funding, a mixture‑of‑experts (MoE) configuration can reach roughly 500 B parameters, but financial resources are expected to be exhausted by the next year.

Deployment Constraints

Relying on a single LLM to solve all tasks is impractical. Successful deployments must respect engineering constraints, industry requirements, and commercial logic, integrating the model as one component of a larger system.

Data‑Centric Iteration

Model performance is tightly coupled to data quality and volume. Data iteration still depends on manual inspection and heuristic adjustments. The core architecture remains the Transformer; occasional experiments with mamba or rmkv were not pursued due to limited resources. Hyper‑parameter tuning and continuous "babysitting" dominate the workflow.

Experiment Cost and Evaluation

High per‑experiment cost forces reliance on semi‑automatic or fully automatic evaluation pipelines, but these cannot be fully trusted. When combined with subjective assessments, SOP (standard‑operating‑procedure) lag becomes severe. Version and data management are often reduced to timestamps and locked evaluation checkpoints, leading to chaotic reproducibility.

Hardware Coupling

Access to more powerful ASICs would lower both training and inference costs, expanding the exploration space. On the inference side, tighter integration with hardware (e.g., wearables such as Ray‑Ban + Meta) is seen as a future direction, especially since embodied AI currently cannot leverage large models effectively.

Multimodal Input Expansion

LLM inputs are expected to incorporate additional modalities:

Vision‑language (VLM/VLA) for images and video.

Structured data streams (databases, sensor data) – exemplified by the TableGPT project.

Audio and speech signals.

Output Side Growth

Beyond generating text, code, and reasoning steps, LLMs will need to interface with hardware APIs and SDKs. Ensuring stability and engineering safeguards for these integrations is a short‑term priority.

Safety and Alignment

Aligning models to avoid out‑of‑box or unsafe behavior remains critical. Emerging approaches such as world models and verifier modules are viewed as promising solutions.

作者：@赵俊博 Jake
知乎：https://zhuanlan.zhihu.com/p/716420396