Open-Source ChatGPT Training: LAION, CarperAI, and Phil Wang’s RLHF Implementations

Artificial intelligence research groups LAION (https://laion.ai/) and CarperAI (https://carper.ai/) have released OpenAssistant and trlX, respectively—both open‑source implementations of human‑feedback reinforcement learning (RLHF) used to train ChatGPT‑style models.

Independent AI developer Phil Wang has also open‑sourced his own ChatGPT algorithm implementation.

LAION, a non‑profit machine‑learning research institute, provides AI models, datasets, and code to the public. In 2022 it released the LAION‑5B dataset containing over 5 billion image‑text pairs. Its current project, OpenAssistant, aims to make large language models accessible to everyone, building on the InstructGPT paper with instruction data, machine‑generated responses, human rankings, and RLHF.

We will not stop at merely copying ChatGPT; we aim to build future assistants that can write emails, draft cover letters, perform meaningful work, use APIs, conduct dynamic research, and be personalized and extensible for anyone. We strive to achieve this in an open and accessible way, creating a capable yet efficient assistant that can run on consumer hardware.

CarperAI, a new lab of the EleutherAI research group, focuses on improving large language model (LLM) performance and safety through reinforcement learning. In October 2022 the lab announced an RLHF‑based “instruction tuning” model and open‑sourced the Transformer Reinforcement Learning X (trlX) framework for fine‑tuning HuggingFace language models.

The project is a collaboration among several organizations, including HuggingFace, Scale, and Humanloop.

AI developer Phil Wang, known for open‑source implementations of models such as Imagen and Make‑A‑Video, shared his work on applying RLHF to the PaLM language model (PaLM + RLHF). He notes that without a pre‑trained model, users must train their own frameworks and encourages interested developers to join the LAION Discord channel.

Although these open‑source projects implement ChatGPT training methods, none currently provide a usable trained model; training such models can require millions of dollars in compute and data. LAION’s OpenAssistant roadmap outlines data collection and model training steps, but a release date for a trained model remains unclear.

CarperAI’s Twitter clarified that no formal RLHF model has been released yet, only small replication efforts and learning summaries shared in their Discord.

Community members have discussed these efforts: HuggingFace CTO Julien Chaumond predicts up to ten open‑source ChatGPT replicas within six months, while AI researcher Sebastian Raschka cautions that many implementations will lack high‑quality models due to the difficulty of labeling training data. StabilityAI founder Emad Mostaque highlighted governance as the toughest challenge for open‑source ChatGPT.

GitHub repositories for the projects are available:

https://github.com/lucidrains/PaLM-rlhf-pytorch

https://github.com/LAION-AI/Open-Assistant

https://github.com/CarperAI/trlx

Hope this information is useful for developers.

Edited by: 场长

Okta Private GitHub Repo Hacked, Source Code Leaked

CodeGPT: VSCode Extension with ChatGPT‑like Features

7 AI Technology Trends to Watch in 2023