AI in Software Engineering at Google: Progress and the Path Ahead

In recent years, software engineers have witnessed rapid advances in AI, especially in machine learning and deep learning, and by 2024 many are using AI‑based code completion tools, both internal at Google and commercial products.

The paper introduces Google’s latest AI‑driven improvements within its internal software development tools, discussing expected changes over the next five years and how to build AI products that add value to professional software development.

Key components of the engineering productivity team’s environment include internal loop interfaces (IDE, code review, code search) and external loop interfaces (error management, planning), whose enhancements directly impact developer productivity and satisfaction.

Challenges include the fast pace of AI research, the gap between technically feasible demos and successful productization, and the need to prioritize ideas based on feasibility and impact.

The team follows three principles: prioritize by technical feasibility and impact, iterate quickly while balancing user experience and model quality, and measure effectiveness by monitoring productivity and satisfaction metrics.

Applying LLMs to Software Development focuses on inline code completion, a natural application of LLMs that leverages code as training data. This feature has become the most popular AI use case in IDEs, with AI‑generated characters now matching manually typed characters, freeing developers to focus on design.

Improvements stem from larger models with coding ability, better context construction heuristics, and model adjustments based on usage logs. High‑quality internal software engineering activity logs are used to train models, capturing fine‑grained edits, build results, code copy‑paste, code review actions, and repository changes.

Future deployments include AI‑assisted code review comments, automatic code paste adjustments, natural‑language driven code edits, and predictive build‑failure fixes.

What We Learned includes the importance of seamless UX integration, balancing reviewer cost and added value, the necessity of rapid A/B experimentation, and the critical role of high‑quality engineer activity data for model quality.

The team emphasizes converting opportunities (user activity) into impact (AI assistance) by improving UX and model performance, and notes that missed opportunities arise from inaccurate predictions, latency, or lack of user attention.

Looking ahead, Google plans to combine the Gemini series of foundation models with developer data (part of DIDACT) to support existing and new ML applications in software engineering, extending benefits beyond code completion to testing, code understanding, and maintenance.

The article calls for community‑wide benchmarks covering broader software engineering tasks, such as code migration and production debugging, to drive progress in AI‑assisted development.

Original source: https://research.google/blog/ai-in-software-engineering-at-google-progress-and-the-path-ahead/