Meta Launches Code Llama: An Advanced AI Coding Model

Meta has launched an AI coding tool called Code Llama, touted as the "most advanced large coding language model".

The model is built on the Llama 2 large language model and can be thought of as a "code‑focused version of Llama 2", capable of generating new code and debugging human‑written code; it is now available on GitHub.

According to reports, Code Llama will use the same community license as Llama 2 and will be free for research and commercial use.

Meta states that Code Llama can generate the code you want and related natural‑language explanations from code‑centric prompts, or refine and debug when pointed at specific code.

In addition to the base version, Meta released a Python‑specific version called Code Llama‑Python and an instruction‑following version called Code Llama‑Instrct.

In Meta’s benchmark tests, Code Llama outperformed the most advanced publicly available LLMs on programming tasks.

Meta says each specific Code Llama variant is not interchangeable; the company does not recommend using the base Code Llama or Code Llama‑Python for natural‑language instructions.

Meta released three model sizes—7B, 13B, and 34B parameters. Each model was trained on 500B code tokens and code‑related data; the 7B and 13B base and instruction models also underwent training with the fill‑in‑the‑middle (FIM) capability, allowing them to insert code into existing code, effectively enabling a "code completion plan".

These three models have different trade‑offs: the 7B model can run on a single GPU; the 34B model offers the best results and stronger coding assistance; the 7B and 13B models are faster than the 34B, making them suitable for low‑latency tasks.

Meta quoted in its blog: "Programmers are already using LLMs to assist with a range of tasks, from writing new software to debugging existing code. The goal is to make developers’ workflows more efficient so they can focus on the most human‑centric aspects of their work."

Meta claims Code Llama performed better than existing publicly available LLMs in benchmark tests, though it did not specify which models were compared.

The company reports that Code Llama scored 53.7% on the HumanEval code benchmark and 56.2% on MBPP, comparable to the state‑of‑the‑art ChatGPT.

It is worth noting that GitHub launched Copilot, based on GPT‑4, in March to help users quickly write and review code and can rewrite old code, but it faces legal lawsuits over alleged copyright infringement.

Additionally, Amazon AWS offers CodeWhisperer for code generation, checking, and updating, while Google’s AlphaCode also provides a coding tool, though it has not been publicly released.