2023’s Leading Open-Source LLMs: LLaMA, Pythia, MPT, Falcon, BLOOM, Mistral

Since the launch of OpenAI's ChatGPT last year, interest in large language models (LLMs) has grown dramatically across industries.

While these AI‑generated tools promise huge profit potential, many small businesses and independent researchers remain cautious about proprietary models due to high operational costs, compute demands, data‑ownership and privacy concerns, and the tendency of models to produce hallucinated information.

Consequently, open‑source LLM alternatives have attracted attention over the past year; although they are often less powerful than their closed‑source counterparts, fine‑tuning can enable them to surpass proprietary models on specific tasks.

Below is a summary of the most impactful open‑source LLM competitors that emerged in 2023.

LLaMA / LLaMA 2

In February 2023, Meta released the first version of LLaMA, a 13‑billion‑parameter model that outperformed the 175‑billion‑parameter GPT‑3 on most benchmarks. The initial release was open‑source under a non‑commercial license, but the model and weights were quickly leaked online.

In July, Meta launched LLaMA 2, expanding the training data by 40% and introducing fine‑tuned variants such as LLaMA 2‑Chat for human‑like dialogue and LLaMA Code for code generation.

Although the openness of LLaMA 2 sparked debate, Meta later relaxed usage restrictions to include commercial applications, leading to a wave of LLaMA‑based open‑source derivatives such as Alpaca, Alpaca‑LoRA, Koala, QLoRA, llama.cpp, Vicuna, Giraffe, and StableBeluga.

In early December, Meta and IBM announced an AI Alliance of over 50 organizations to support open innovation and open science in AI.

LLaMA 2 URL: https://ai.meta.com/llama/

Pythia

Released in April by the nonprofit EleutherAI, Pythia is a suite of LLMs of varying scales trained on publicly available data.

The project aims to provide researchers with interpretability tools to better understand the training process and outcomes of large language models.

GitHub: https://github.com/EleutherAI/pythia

MPT (MosaicML)

Starting in May, MosaicML introduced the MPT series, beginning with a 7‑billion‑parameter model and later releasing a 30‑billion‑parameter version in June, which the company claims outperforms LLaMA and Falcon, especially on tasks requiring long prompts.

MPT incorporates recent advances to improve efficiency, extend context length, and enhance stability, reducing loss spikes.

URL: https://www.mosaicml.com/mpt

Falcon

Developed by the Technology Innovation Institute in Abu Dhabi, Falcon was released in June under the Apache 2.0 license.

The initial 40‑billion‑parameter model quickly gained popularity because the weights were publicly available.

In September, the institute announced a larger 180‑billion‑parameter Falcon model, making it one of the biggest open‑source models. The team asserts that, while slightly behind proprietary models like GPT‑4, Falcon 180B surpasses Meta’s LLaMA 2 and rivals Google’s PaLM 2 Large.

URL: https://www.tii.ae/about-us

BLOOM

BLOOM (BigScience Large Open‑science Open‑access Multilingual Language Model) was released in July 2022, but it remains a notable open‑source model in 2023.

Co‑developed under the coordination of Hugging Face and France’s GENCI, it involved over 1,000 AI researchers from 60 countries and 250 institutions.

The largest BLOOM model contains 178 billion parameters, trained on data from 46 human languages and 13 programming languages, making it the biggest open‑source multilingual LLM to date.

URL: https://huggingface.co/bigscience/bloom

Mistral

Mistral, founded by former Meta and Google R&D staff, launched its first 7‑billion‑parameter LLM in September.

The company claims Mistral 7B outperforms LLaMA 2 and other open‑source LLMs on many benchmarks. Later that month, they released Mixtral 8×7B via torrent, generating significant buzz.

URL: https://mistral.ai/

Conclusion

As the open‑source LLM ecosystem expands, many developers seek cost‑effective, transparent, and customizable alternatives to reduce reliance on OpenAI’s API.

Proprietary models may still hold slight advantages, but open‑source models are rapidly catching up, with some surpassing larger‑parameter peers, highlighting that training data quality often outweighs sheer model size.

The past year’s breakthroughs demonstrate that open‑source LLMs will continue to play a crucial role as the field evolves.