Boosting Aviator Script Development with AI—No Model Training Required

Abstract

This article describes a practice of implementing AI‑driven analysis, code generation, and code review for a niche development language (Aviator) without training a dedicated model. It records engineering‑level experiences such as RAG optimization, ranking pruning, small‑to‑big expansion, prompt tuning, few‑shot learning, and hallucination avoidance, aiming to help engineers without algorithmic backgrounds.

Business Background

In B‑side sales technical systems, the decision center supports rapid configuration of complex rule decision trees, enabling fast policy adjustments such as profit‑control, sales‑guidance, approval rules, flexible pricing, product bundling, and gray‑scale technical scenarios. The decision center’s architecture follows a hierarchy of strategy‑rule‑metric, where the metric layer is implemented using Aviator scripts.

Aviator is a high‑performance, lightweight Java‑based expression evaluation engine. Compared with Groovy or JRuby, Aviator is tiny (≈537 KB total, ≈70 KB without dependencies) and compiles expressions to JVM bytecode.

use java.io.File;</code>
<code>let io = require('io');</code>
<code>let files = count(ARGV) == 0 ? 5 : long(ARGV[0]);</code>
<code>let nums = count(ARGV) < 2? 1000 : long(ARGV[1]);</code>
<code>let temp_dir = "/tmp";</code>
<code>for i in range(0, files) {</code>
<code>  let file = "#{temp_dir}#{File.separator}data.#{i}";</code>
<code>  let writer = io.writer(io.file(file));</code>
<code>  for j in range(0, nums) {</code>
<code>    write(writer, "#{rand(100000000)}
");</code>
<code>  }</code>
<code>  io.close(writer);</code>
<code>  p("Generate #{nums} random numbers to file #{file}");</code>
<code>}</code>
<code>let testResult = indicator_execute("TEST", "TEST-INDICATOR", seq.map());</code>
<code>if (is_empty(testResult)) {</code>
<code>  log.log("测试指标调用结果为空");</code>
<code>}</code>
<code>let testNameSet = seq.set();</code>
<code>for testData in testResult {</code>
<code>  seq.add(testNameSet, testData.get("name"));</code>
<code>}</code>
<code>...

Business Problems

Developers face three main difficulties with Aviator scripts: (1) the language is obscure and lacks comprehensive documentation, leading to frequent syntax look‑ups; (2) the IDE support is weak, with most editing done in a browser where syntax errors are hard to spot; (3) maintaining scripts is painful, resulting in many poorly described or duplicated metrics.

Practical Solution

Leveraging AI capabilities (GitHub, Aone Copilot) for code generation, comment generation, and intelligent code review, an “AI Smart Development Assistant” is added to the decision‑center script editor to address the above challenges.

Challenges & Solution Ideas

1. Model knowledge is limited to data up to 2023, so the chosen model is Meta‑Llama‑3‑1‑405B‑Instruct, which performs well on complex reasoning benchmarks. 2. Large models have poor generalization for Aviator; prompt engineering and RAG are used to compensate. 3. The solution focuses on model selection, prompt optimization, and RAG knowledge enhancement.

RAG Implementation for Aviator Knowledge Q&A

Because Aviator documentation is scarce, the approach uses Yuque’s public search API to retrieve relevant document snippets instead of building a full vector store.

LLM Content Enhancement

Retrieved snippets are fed to the LLM with a carefully crafted prompt that instructs the model to answer only Aviator‑related questions and cite sources.

Next‑Step Optimizations

Rerank retrieved results to filter low‑relevance snippets.

Small‑to‑Big retrieval: first fetch summaries (small), then retrieve full documents (big) for precise matching.

LLM‑Generated Script Code from Functional Descriptions

Using few‑shot prompting, the model generates Aviator scripts based on metric name, functional description, and scenario, reusing existing similar scripts to improve consistency.

Code Review Automation

A prompt defines review rules (e.g., null‑check, naming conventions, indentation, comment style) and asks the LLM to produce review feedback and suggested fixes in Aviator syntax.

Code Formatting Assistance

Another prompt guides the model to suggest formatting changes (two‑space indentation, camelCase naming, proper comments) without altering logic.

Future Plans

Evaluation will include cross‑model scoring, automated compilation/execution testing, and human feedback loops. Successful metric‑level generation could extend to rule‑ and strategy‑level generation, achieving end‑to‑end intelligent policy creation.

Conclusion

The article records an engineering‑centric practice of applying LLMs, RAG, and prompt engineering to generate and review Aviator scripts without model training, highlighting challenges, solutions, and future directions for AI‑assisted development.