Large Models in Software Engineering: Capability Limits and Optimal Use Cases

Technical Core Bottlenecks

1. Dynamic Reasoning and State Management

Even with context windows expanded to 1,000K tokens (e.g., LongRoPE), large models still struggle with multi‑step state transitions such as dynamic programming and recursive backtracking. For example, on the LeetCode "Best Time to Buy and Sell Stock" problem, a university study reported LLM solution pass rates below 30% while human developers exceed 80%.

Dynamic programming problems : Models often miss hidden constraints like transaction limits or fees.

Concurrent programming challenges : In a producer‑consumer implementation, agent‑based frameworks (e.g., ChatDev) produce code with a 65% chance of race conditions or deadlocks due to insufficient understanding of OS scheduling and semaphore ordering.

2. Mathematical Modeling and Algorithm Design

LLMs show significant deficiencies in mathematically rigorous algorithm design.

Shortest‑path optimization : For Dijkstra on sparse graphs, LLM‑generated code degrades to O(N²) time, whereas human‑optimized versions achieve O(M+N log N) because the model lacks proof of greedy strategies.

Convex optimization in finance risk models : Generated risk‑assessment code often ignores convexity constraints, leading to sub‑optimal solutions. A large bank’s RAG system reduced hallucination to 1.2% but still required manual mathematical verification.

Engineering‑Practice Structural Shortcomings

1. Cross‑Language Engineering Barriers

Multi‑language context understanding : On the Multi‑SWE‑bench benchmark, LLMs solve Python tasks at ~50% success but drop below 10% on TypeScript and Java due to syntax differences (e.g., Rust ownership) and framework ecosystems (e.g., Spring Boot DI).

Legacy system reverse engineering : When migrating COBOL batch programs to microservices, LLM‑generated code fails to parse JCL definitions correctly in 70% of cases.

2. Security, Compliance, and Reliability

Vulnerability detection and defense : RAG tools like Vul‑RAG improve known‑vulnerability pattern recall by 12.96% on the PairVul benchmark, yet LLMs still mis‑respond to novel attacks (e.g., MIME‑encoding bypass) with error rates over 60% (garak tool).

Compliance code generation : In HIPAA‑regulated medical systems, LLM‑generated encryption modules often omit FIPS‑140‑2 certified algorithms, causing audit failures because of limited legal‑semantic understanding.

Current Benchmark Performance

1. Code Generation and Problem‑Solving

ByteDance’s Trae achieved a 75.2% success rate on SWE‑Bench‑Verified; the top‑8 models improved ~10% over the previous two months, with Claude 4 standing out while most models remain in the 40‑60% range, still far from human expert levels.

HumanEval shows top models reach 80‑90% on simple tasks but drop sharply on multi‑step reasoning and system integration tasks.

2. Production‑Environment Reality

GitHub Copilot data: development speed gains of 10‑55% (task‑dependent), code acceptance rate around 30‑40% (i.e., 60‑70% of suggestions are rejected), high developer satisfaction (>90%) mainly for repetitive work, limited assistance for complex business logic and architecture design.

Tasks LLMs Handle Well

Standard library usage and API calls.

Implementation of common design patterns.

Unit test generation for simple functions.

Docstring and comment generation.

Code formatting and basic refactoring.

Accuracy for simple function implementation ranges from 70‑85%; API usage examples achieve 60‑80% accuracy, but human review remains necessary.

Tasks LLMs Currently Cannot Perform Adequately

Complex system architecture design : lacks global optimization, non‑functional requirement handling, and long‑term technical debt assessment.

Production‑grade code quality assurance : frequent security vulnerabilities, inefficient algorithms, missing error handling, and hard‑to‑maintain structures.

Complex business‑logic implementation : fails to uncover implicit requirements, optimize business processes, and manage cross‑system integration.

Strategic technology planning : cannot perform end‑to‑end system design, forecast technology trends, or devise migration strategies.

Reasonable Outlook for Late 2025

1. Near‑Term Improvements (≈6 months)

Code‑generation accuracy may reach 85‑90%.

Larger context windows improve multi‑turn understanding.

Enhanced multimodal capabilities for interpreting diagrams and charts.

2. Persistent Challenges

End‑to‑end complex system design.

Reliable generation of security‑critical code.

Innovative implementation of deep business logic.

Large‑scale system performance optimization.

3. High‑Value Application Scenarios

Intelligent code assistants : assist developers without replacing them.

Rapid prototyping : support proof‑of‑concept and early development.

Documentation and test generation : automate auxiliary work.

Code and test review assistance : surface common issues and improvement suggestions.

Practical Recommendations

1. Tasks Suited for LLM Assistance

Highly standardized development tasks.

Repetitive coding work.

Exploratory scenario generation.

Learning and educational programming activities.

2. Areas Requiring Human Leadership

Core system architecture decisions.

Security‑critical and reliability‑critical code.

Complex business‑logic design.

Technology‑stack selection and evolution planning.

Team collaboration and project management.

3. Effective Human‑AI Collaboration Patterns

LLM generates initial solutions; humans evaluate and refine.

Humans define goals and constraints; LLM fills implementation details.

LLM performs code review; humans make final decisions.

LLM drafts documentation and tests; humans ensure quality.

Future Breakthrough Possibilities

1. Hybrid Augmented Architectures

Symbolic‑statistical integration : MetaGPT introduces an architect role, raising code‑generation success to 82%; mixing LLMs with theorem provers improves mathematical reasoning but increases training cost five‑fold.

Goal‑driven architectures : Qwen2.5’s “sub‑goal setting → reverse reasoning” outperforms traditional models by 23 percentage points on math tasks, showing promise in specialized domains like medical diagnosis, though generalization remains limited.

2. Domain‑Specific Optimization Paths

Vertical fine‑tuning : GemSUra‑7B improves medical code‑generation accuracy by 37% over generic models, but requires extensive domain data and expertise; similar gains observed in finance.

Dynamic knowledge injection : A large bank’s risk‑control system combines RAG with expert rules, lowering hallucination to 1.2% but demanding continuous knowledge‑base maintenance and incurring latency for new risk patterns.

Conclusion: Pragmatic Boundary Awareness

Based on 2025 data and real‑world experience, large models serve as excellent development assistants rather than autonomous software engineers. They boost efficiency on standardized, repetitive tasks, yet creative design, complex system thinking, and security‑critical decisions still demand human expertise. The key is to neither underestimate nor overestimate their abilities, to match the right application scenarios and collaboration modes, and to maximize AI/LLM value through balanced human‑machine cooperation.