2024 AI Programming: Key Advances, Tools, and Trends

Adoption rates

According to the 2024 Software R&D Large‑Model Survey (https://mp.weixin.qq.com/s?__biz=MjM5ODczMDc1Mw==&mid=2651861037&idx=1&sn=d745412b411abf23144e92460433e8a3), code‑generation adoption in Chinese teams ranges from 10 % to 40 % (see Figure 1).

Devin benchmark results

The AI programmer "Devin" can independently code, debug, locate and fix repository bugs, and deploy applications. On the SWE‑bench benchmark it solved 13.86 % of real GitHub issues. For the verified subset of 500 problems, success rose from 2.8 % in April to 53 %–54.2 % later in the year, demonstrating a marked capability increase.

Key factors driving progress

Model capability enhancements : Successive releases such as Claude 3 Opus, GPT‑4o, Claude 3.5 Sonnet and Claude 3.5 Haiku have steadily improved large‑model understanding of complex programming tasks.

AI agents : Agents invoke static analysis tools, search engines and APIs, building knowledge graphs of codebases to improve problem localization and patch generation. Reported performance jumps include:

RAG + GPT‑4 (1106) 2.8 % → SWE‑agent + GPT‑4 (1106) 22.4 %

RAG + Claude 3 Opus 7 % → SWE‑agent + Claude 3 Opus 18.2 %

Multimodal abilities : Multimodal LLMs (e.g., Claude‑3.5‑Sonnet) process both text and visual inputs, enabling agents to interpret UI screenshots, charts and highlighted code, which helps solve image‑rich GitHub issues.

Tool‑integration frameworks : Projects such as Composio SWE‑Kit combine file operations, code analysis, shell execution, knowledge‑base management and database access, boosting SWE‑bench verified scores to 48.6 %. OpenHands + CodeAct v2.1 unifies agent actions in a single code‑action space, achieving a verified leaderboard score of 53 % (excluding dev‑only cases).

Pre‑training data impact

Enriching pre‑training corpora with abstract syntax trees (AST) and code dependency graphs gives LLMs stronger context awareness. Consequently, AI‑powered coding assistants can retrieve the most relevant snippets based on function names, comments or partial code, reducing hallucinations and improving generation accuracy.

Future workflow concept

The paper Flows: Building Blocks of Reasoning and Collaborating AI proposes a "composite competitive coding flow" where developers specify requirements and LLMs plus agents autonomously produce, verify and iterate code.

Tool landscape

Domestically popular coding assistants include ChatGPT, GitHub Copilot, Tongyi Code, CodeGeeX, Wenxin Kuaima and Ant CodeFuse. Abroad, newer tools gaining traction are Codeium Windsurf IDE, Codeium IDE Cascade, Solver AI and Websim AI (see Figure 3).

Adoption metrics in Chinese enterprises

Over 80 % of engineers use AI programming tools daily.

Approximately 30 % of committed code originates from AI‑generated output.

Average code‑acceptance rate exceeds 40 % (some product lines reach 60 %).

For tasks that constitute 20‑30 % of overall work, development efficiency improves by 20‑30 %.

Development approach highlighted

ATDD (Acceptance‑Test‑Driven Development) is presented as a workflow where large‑model agents generate requirements, acceptance criteria, product code and test code that iteratively validate each other.

Future expectations

AI programming tools are expected to become more intelligent, explainable and language‑agnostic, leveraging reinforcement learning for self‑optimization. Development teams will need to continuously learn new technologies, refine processes and ensure high‑quality AI‑generated output.