What I Learned After a Year Building Large Language Models: Wins, Losses, and Future Trends

Year in Review: The Gains and Losses of Working on Large‑Model Foundations

In the past year I dramatically reduced my salary to join a fledgling startup dedicated to large‑model research, driven by curiosity about why models generate fluent text and concern that only a few closed‑AI companies control this pivotal technology.

1. Early Exponential Curve – Nothing Visible at First

For most companies the early stage of an exponential growth curve looks indistinguishable from a linear one; the signal is hidden by noise. This mirrors personal career growth: large companies offer steady linear returns, while chasing non‑linear upside requires risk‑taking, and many such attempts end in exponential decay.

2. The Thrill and Frustration of Competing with Model Giants

Model developers are constantly benchmarked against ChatGPT or Claude. With far fewer resources than the big players, achieving differentiation starts with solid baseline capabilities. In 2023 open‑source models were weak; our team trained a 14‑billion‑parameter model from scratch, performed SFT and alignment, and spent countless hours testing it on complex customer scenarios.

Reading a flood of papers revealed incremental breakthroughs that, when combined, created a rapid knowledge‑accumulation effect across the community. Open‑source momentum accelerated large‑model development and reshaped the competitive landscape, but it also demanded better information‑filtering skills to avoid being lost in the literature deluge.

When every component of the pipeline is refined, the model eventually works in real‑world, complex settings. Maintaining a technical lead now requires continuous investment in compute and talent—companies with deep pockets like xAI or DeepSeek can explore aggressively, while resource‑constrained startups must treat revenue generation as a first‑class priority.

3. HR No Longer Questions My “Skill‑Tree” Choices

My career path has been unconventional: starting with model research, then moving to Google for engineering, and later returning to large‑model R&D. Once a model reaches sufficient scale, understanding both algorithms and systems becomes essential, but specialization eventually outweighs being a generalist.

Initial Judgments: What Was Right and What Was Wrong

❌ Underestimated Meta’s disruptive power – I feared that only OpenAI could master top‑tier models, yet open‑source progress proved rapid, and Meta, Qwen, and DeepSeek have become serious contenders.

❌ Didn’t anticipate Google’s catch‑up – I assumed Google would fall behind in the AI era, but its abundant data and talent have allowed it to regain momentum, albeit with slower promotion paths for data‑focused engineers.

✅ Correctly identified the large‑model wave – I was among the early observers of the GenAI transformation during the Stable Diffusion boom and pivoted accordingly.

✅ Saw multimodal and inference‑time scaling trends, but missed early execution – I recognized the potential of multimodal products and inference‑time scaling in early 2024, yet failed to allocate resources promptly.

Emerging Trends to Watch

Off‑shoring of Silicon‑Valley engineering talent: more mid‑size firms hire outside the US (Toronto, Poland, India). The new risk is not outsourcing work, but outsourcing to AI‑augmented services.

Marketing and sales remain locally anchored: despite technical tasks being outsource‑able, cultural, linguistic, and network factors keep these roles tied to specific regions.

Most people still under‑utilize GenAI: everyday AI usage among peers is low, indicating abundant opportunities for consumer‑facing applications.

Work and Family Balance

The intense focus on R&D strained my family life, creating financial and emotional pressure. My spouse’s support helped me shift from a scarcity mindset to a more sustainable approach.

Looking ahead to 2025, I plan to devote more time to family, recognizing that personal stability underpins long‑term professional success.

Final Thoughts

Do I regret abandoning a six‑figure salary to pursue large‑model research? Not yet. Long‑term investments often reveal their outcomes only after several years. The biggest gain has been learning how to navigate uncertainty, collaborate with like‑minded teammates, and experience the excitement of breakthrough moments—whether the journey ends in success or failure.