How Safe Is AI-Generated Code? Real‑World Risks and Mitigation Strategies

Introduction

AI‑generated code is now a major contributor to software development. While it can increase productivity, it also introduces security risks that need systematic study. Tencent Wukong Code Security Team, Peking University Narwhal‑Lab, and Fudan University Systems Software and Security Lab analyzed open‑source projects and real CVE data to quantify usage trends, vulnerability rates, and the role of AI‑generated code throughout the vulnerability lifecycle.

Key Findings

Vulnerability prevalence : Veracode’s analysis of 100 large language models on 80 programming tasks shows that about 45 % of AI‑generated snippets contain security flaws , even though functional correctness is often high.

Academic benchmarks : Recent benchmarks such as A.S.E and SUSVIBES evaluate LLMs on multi‑file, cross‑context tasks and consistently find that AI‑generated code is less secure than expected.

Evolution of AI‑Generated Code in Open‑Source Projects

Explosive exploration : Early adoption driven by low‑friction IDE plugins leads to a rapid rise in AI‑generated code across many tasks.

Rational regression : As codebases grow, limitations in long‑context understanding and non‑functional requirements cause developers to reduce AI usage in high‑risk or core‑architecture areas.

Stable collaboration : AI settles into a supportive role, handling repetitive, pattern‑driven work (e.g., test scaffolding, documentation) while humans retain responsibility for system design, business logic, and security reviews.

Language Distribution

AI‑generated code is most common in languages with large open‑source ecosystems— Python, JavaScript, TypeScript . Enterprise‑focused languages such as Java and Go see moderate adoption, whereas system‑level languages like Rust and C++ have low adoption due to stricter type and memory‑safety constraints.

AI‑Generated Code in the Vulnerability Lifecycle

In 3–5 % of vulnerability fixes , AI‑generated code is replaced by manually written code, indicating a fallback to human control for security‑critical patches.

Conversely, in 9.4 % of fixes , previously human‑written code is substituted with AI‑generated snippets, showing that AI can accelerate remediation when used responsibly.

Thus AI can act both as a source of risk and as a remediation aid, depending on how it is applied.

Characteristics of AI‑Introduced Vulnerabilities

AI‑generated defects tend to be “pattern‑based”, reflecting the model’s imitation of training data rather than an understanding of security principles. The most frequent flaw categories are:

Improper input validation and data handling, leading to injection‑type risks.

Unsafe API usage and outdated cryptographic practices.

These vulnerabilities are usually shallow, localized code issues that static analysis tools can detect. Severity analysis shows that AI‑introduced bugs can be as severe as human‑written ones, often affecting network‑exposed components (APIs, web services) and expanding the remote attack surface.

Mitigation Framework

Evaluation benchmarks : Adopt LLM‑focused security benchmarks such as the A.S.E repository ( https://github.com/Tencent/AICGSecEval) which provides adversarial test suites across languages and scenarios. Use these benchmarks to quantify model safety before deployment.

Model‑level hardening :

During training, incorporate high‑quality, security‑annotated datasets and apply Reinforcement Learning from Human Feedback (RLHF) to align models with secure coding practices.

At inference time, employ Retrieval‑Augmented Generation (RAG) to inject up‑to‑date vulnerability intelligence.

Apply constrained decoding (e.g., AST‑level constraints) to prevent unsafe API calls and enforce syntactic safety.

Human‑AI collaborative governance :

Require mandatory human review of AI‑generated code, especially for critical logic, data flow, and permission boundaries.

Tag AI‑produced snippets with metadata for traceability and execute them in isolated environments.

Integrate static and dynamic analysis pipelines to catch residual risks before merge.

This layered approach—benchmarking, secure model engineering, and disciplined human oversight—preserves AI’s productivity while containing its security exposure.

References

Veracode research (2025):

https://www.businesswire.com/news/home/20250730694951/en/AI-Generated-Code-Poses-Major-Security-Risks-in-Nearly-Half-of-All-Development-Tasks-Veracode-Research-Reveals

A.S.E: A Repository‑Level Benchmark for Evaluating Security in AI‑Generated Code ( https://github.com/Tencent/AICGSecEval)

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent‑Generated Code in Real‑World Tasks