Why 2024 Will Be the Year of AI Engineers and LLM‑Driven Apps

Developers can expect that integrating large language models (LLMs) into applications will be one of the biggest tech trends of 2024.

Since 2023 many companies have used OpenAI’s proprietary models via APIs, but by the end of 2023 a variety of LLMs, including open‑source options, became available, reducing reliance on proprietary services.

1. The Emergence of AI Engineers

Developers now have a new career path: the “AI Engineer,” promoted by Shawn Wang ("@swyx"). This role is positioned as the next step after “instant engineers,” focusing on building applications that leverage LLMs and related tools such as LangChain and vector databases.

By late 2023 many companies were already hiring developers skilled in LLMs and associated tooling.

2. Evolution of the LLM Technology Stack

A new stack tailored for AI engineers is emerging, with the orchestration layer being crucial for connecting applications to LLMs. Tools like LangChain and LlamaIndex, introduced in 2023, help developers integrate and manage LLM interactions, and LangChain’s “chain” concept enables interoperability with other frameworks.

In May 2023 Cloudflare announced that LangChain supports its Workers platform.

3. Open‑Source Large Language Models

The most impactful development in AI engineering this year is the rise of open‑source LLMs. Meta’s Llama 2, released in July, already ranks near the top on Stanford’s HELM benchmark, offering developers a more adaptable alternative to proprietary models.

Open‑source models are rapidly catching up, challenging the dominance of OpenAI and Google.

4. Vector Databases

Vector databases, which store data as high‑dimensional vectors (embeddings), have become the most influential data technology for LLMs in 2023. They enable use cases such as personalized recommendation systems, anomaly detection, and natural‑language processing.

New solutions like Pinecone and open‑source projects such as Chroma are carving out significant market niches, while established database vendors (e.g., Redis Enterprise) are adding vector capabilities.

5. AI Agents

AI agents—software that uses LLMs to perform tasks autonomously—are a controversial yet promising trend, exemplified by AutoGPT released in March. While some view them as overhyped, they represent a step toward more capable, self‑directed AI systems.

Experts suggest that AI agents have not yet reached their full potential, largely due to challenges in creating reliable evaluation methods.

Conclusion

2024 is poised to be a breakthrough year for AI engineering, despite challenges such as governance issues and competition among major players like OpenAI, Meta, and Google. Emerging tools and open‑source models are expected to mature the LLM application ecosystem, while regulatory developments may shape the industry’s trajectory.