How to Extend AutoDev with Custom LLMs and Prompt Actions

Overview

AutoDev, an open‑source AI‑assisted programming tool, now supports deeper customization. Developers can plug in their own large language models (LLMs), tailor IntelliJ IDEA behavior, and define project‑specific coding standards, turning the assistant into a flexible development companion.

Custom Large Language Model Integration

The team built a unified LLM interface on top of the Gluon Meson platform and tested models such as Tsinghua's ChatGLM2‑6B and Alibaba Cloud's Qwen‑7B. Because these models are relatively small, they may lack strong context‑completion capabilities.

AutoDev currently streams responses from a server‑side FastAPI endpoint. A minimal FastAPI + EventSourceResponse example is provided in the source code.

Request Structure

The server expects a JSON payload similar to the following:

{
  "messages": [
    {"role": "user", "message": "I'm Nihillum."},
    {"role": "assistant", "message": "OK"},
    {"role": "user", "message": "What did I just say?"}
  ]
}

Future releases will allow configurable HTTP headers as discussed in issue #25.

Response Handling

To normalize outputs from different models, AutoDev uses JSONPath. A simple example extracts the generated text with the expression $.choices[0].content. The extracted content is then processed further as described in the README’s Custom LLM Server section.

Custom Prompt Actions

Inspired by JetBrains AI Assistant, AutoDev lets users define custom actions that send tailored prompts to the LLM. For instance, a Rust‑to‑Kotlin translation action can be added via a JSON configuration:

{
  "prompts": [
    {
      "title": "🇨🇳 Translate to Kotlin",
      "autoInvoke": false,
      "matchRegex": ".*",
      "priority": 1000,
      "template": "Translate the following code to Kotlin.
${SIMILAR_CHUNK}
Compare these snippets:
${METHOD_INPUT_OUTPUT}
Here is the code:
${SELECTION}"
    }
  ]
}

The priority field controls menu ordering, while template is the prompt sent to the LLM. Several context variables can be used inside the template: ${SELECTION}: the currently selected code. ${SIMILAR_CHUNK}: a similar code fragment from the project. ${METHOD_INPUT_OUTPUT}: method input and output examples.

Additional variables prefixed with SPEC_ pull values from the specification section of the configuration, e.g., ${SPEC_controller} loads the controller spec.

Custom Specification Rules

Action configurations can reference a spec object that defines coding conventions for different layers. An example spec includes rules for controllers, services, entities, repositories, and DDL:

{
  "spec": {
    "controller": "- Use BeanUtils.copyProperties in the Controller for DTO to Entity conversion.
- Avoid using Autowired.
- Use Swagger Annotations to indicate API meanings.
- Controller methods should capture and handle business exceptions, rather than throwing system exceptions.",
    "service": "- Service layer should use constructor injection or setter injection; avoid using the @Autowired annotation.",
    "entity": "- Entity classes should use JPA annotations for database mapping.
- The entity class name should match the corresponding database table name. Use @Id, @GeneratedValue, @Table, etc.",
    "repository": "- Repository interfaces should extend JpaRepository to inherit basic CRUD operations.",
    "ddl": "- Fields should be constrained with NOT NULL constraints to ensure data integrity."
  }
}

With these specifications, AutoDev can automatically apply the defined standards when generating code.

Conclusion

By configuring a custom LLM, defining request/response handling, and adding JSON‑based prompt actions and specifications, developers can turn AutoDev into a highly adaptable AI coding assistant. Issues, feature requests, and discussions continue on the project's GitHub repository.