Design and Implementation of a Kotlin Compiler Plugin (KCP) for Efficient Serialization and Faster Compilation

1. Background

There is a scarcity of information about compiler plugins, possibly due to implementation costs. In practice, Java has the Manifold project for reflection‑free meta‑programming, and Kotlin enjoys highly praised android-kotlin-synthetics and the KSP alternative to KAPT. These projects outperform traditional code‑gen solutions (APT, Transform) both functionally and in compilation performance.

Motivated by daily development pain points, I examined parts of the Kotlin compiler source, hand‑crafted a Kotlin-Compiler-Plugin (hereafter KCP). After integration, both code elegance and compilation speed improved significantly, so I share the experience for discussion.

2. Experiment and Results

Consider the following example:

@ProtoMessage("webcast.data.GiftTip")
 data class GiftTip(
    @SerializedName("display_text") @JvmField var displayText: Text? = null,
    @SerializedName("background_color") @JvmField var backgroundColor: String? = null,
    @SerializedName("prefix_image") @JvmField var perfixImage: ImageModel? = null
)

The above POJO, used to receive network data, suffers from several problems:

To keep a no‑arg constructor, avoid NPE, and support deserialization, the tiny class with three fields carries a total of seven annotations, causing severe visual noise.

All parameters are nullable, leading to a cascade of question marks or null‑checks in usage.

Fields must be declared as var to prevent the backingField from being marked final , which would block non‑reflective assignment outside the constructor.

These issues are hard to solve with pure Kotlin syntax because, aside from delegated properties, every property must have a default value. Usually the default is written after the declaration, e.g., val foo = "bar" . While acceptable in most cases, it becomes redundant when the class is used solely for deserialization.

The pain point is the lack of elegance; KCP can provide a syntactic sugar to improve readability.

Moreover, KAPT annotation processing slows compilation. In the example, @ProtoMessage is a custom annotation that generates Protobuf helper classes. KCP’s compilation speed is significantly better than KAPT, allowing us to replace the annotation processor with KCP for performance gains.

Experiment Results

Using KCP makes the code cleaner and speeds up compilation. The final effect is:

// Old solution (kapt)
@ProtoMessage("webcast.xxx.TestMessage")
class TestKotlin {
    @SerializedName("room_id") @JvmField var roomId: Long = 0L
    @SerializedName("display_text") @JvmField var displayText: Text? = null
    @SerializedName("url_list") @JvmField var urlList: MutableList
? = null
    @SerializedName("nickname") @JvmField var nickname: String = "defaultName"
}

// New solution (kcp)
@ProtoMessage("webcast.xxx.TestMessage")
class TestKotlin {
    val roomId by serialized
()
    val displayText by serialized
()
    val urlList by serialized
>()
    val nickname by serialized(defaultValue = "defaultName")
}

Performance-wise, compilation time decreased by 20% locally and 16.4% in CI (full module, non‑incremental, average):

Scenario

kapt (old)

kcp (new)

Local test

07:03

05:38

Online test

10:35

08:51

3. Kotlin (JVM) Compilation Overview

The following description applies only to the JVM target using the non‑IR backend (Kotlin version < 1.5). The IR backend became the default after Kotlin 1.5.

The Kotlin compilation flow is illustrated below:

The left side shows the internal kotlinc process, divided into prepare and compile steps; compile further splits into front‑end and back‑end. The right side lists the artifact types produced at each stage.

Preparation

Before building the syntax tree, Kotlin reuses the intellij‑core infrastructure, heavily sharing logic with the IDE. Interested readers can explore KotlinCoreEnvironment .

The syntax tree, like other IntelliJ‑supported languages, is based on the PSI system. See "What Is the PSI?" for details.

Source files are transformed into KtElements , which belongs to the compiler front‑end. This step also generates ASTNode and converts them to PsiElement objects.

This design makes it easy to share code between the IDE plugin and the compiler, e.g., for android-kotlin-synthetics .

Compiler Front‑end

The front‑end performs lexical and syntactic analysis, then traverses the syntax tree to report errors and convert Elements into Descriptors , storing them in a BindingContext map.

A Descriptor describes an element (its name, return type, whether it is inline , has a getter/setter, etc.). The BindingContext is essentially a nested map: Map<Type, Map<Key, Descriptor>> .

In the diagram, solid circles represent actual PsiElement nodes ( KtElement ), solid lines depict the tree structure, and dashed circles illustrate indirect relationships such as Call and Reference . All these eventually become Descriptor entries in the BindingContext .

Compiler Back‑end

The back‑end simply uses the BindingContext to emit bytecode for each KtFile . It walks the syntax tree and generates class files via DefaultCodegenFactory#generateModule .

Key components include:

MemberCodegen<T : KtPureElement> : entry point for generating bytecode of a specific element. Most often ImplementationBodyCodegen is used for classes/objects.

ClassBuilder : the abstraction that actually writes bytecode. It allows extensions and optimizations, and Kotlin’s built‑in bytecode tweaks are applied through proxy ClassBuilder implementations.

Utility classes such as FunctionCodegen , PropertyCodegen , ExpressionCodegen , and KotlinTypeMapper (maps Kotlin types to ASM Type ).

InstructionAdapter : a thin wrapper around ASM’s MethodVisitor for convenient instruction emission.

4. Practical Implementation

The goal is to use a concise syntax to declare a field’s type, serialized name, and default value while keeping the implementation extensible.

Syntax Design

Previously, the solution relied on KAPT/annotation processing, which required a two‑step process (object creation then field assignment). This caused several issues:

Fields could not be final because assignment happened after construction.

Fields had to be nullable or have a default value.

Fields could not be private because assignment occurs from outside the class.

Ideally, deserialization‑only data classes should have public final fields that work out‑of‑the‑box. To achieve this, we first tried a DSL that replaces field declarations entirely, but it introduced a steep learning curve and required an IDE plugin for code generation.

Delegation

Annotations are the simplest way to mark elements, but they become verbose when overused. Kotlin’s delegation feature offers a cleaner alternative: the getter and setter are delegated to another object, hiding the underlying logic.

Without a compiler plugin, delegating to serialized generates Java bytecode similar to:

public final class Test {
    @NotNull private final DecodeDelegate test$delegate = KtxKt.serialized("test");
    @NotNull public final String getTest() { /* invoke getValue */ }
}

Thus, a delegated property still has a backing field; the difference lies in the generated getter/setter. Our plugin will replace the delegated property with a normal protected backing field and a regular getter.

Data Parsing

All properties must support both JSON and Protobuf deserialization. JSON handling is straightforward: add @SerializedName to the hidden backing field and rely on Gson.

Protobuf requires a constructor that takes a ProtoReader (similar to JsonReader ) and performs stream‑based decoding. The generated Java code looks like:

public final class Test {
    @SerializedName("test")
    protected String _$$test;
    @NotNull public final String getTest() { return _$$test; }
    public Test(@NotNull ProtoReader reader) {
        for (/*...*/) {
            switch(tag) {
                case 0: _$$test = /* decode */; continue;
            }
            ProtoScalarTypeDecoder.skipUnknown(reader);
        }
        if (_$$test == null) _$$test = "";
    }
}

Core Logic & Implementation

The plugin performs the following steps in order:

Collect properties in the current class and its super‑classes that meet the criteria.

Generate a constructor that takes a ProtoReader based on the IDL definition.

Identify properties delegated to serialized and replace them with protected backing fields.

Generate a no‑arg constructor to ensure normal property initialization and execution of init blocks.

Information Collection & Error Detection

Using AnalysisHandlerExtension , we intercept the compiler front‑end to gather PsiElement and Descriptor information. This allows early detection of mismatches between Kotlin properties and the corresponding Protobuf IDL.

During analysisCompleted , we:

Parse all relevant .proto files to obtain message definitions.

Validate that the target class’s declarations conform to the IDL.

Collect configuration data for each field (key, strategy, default value) to be used later in code generation.

Field strategies are represented by the sealed class SerializeConfig.Strategy , which distinguishes between optional fields, fields with compile‑time constants, fields with runtime expressions, and legacy annotated fields.

ClassBuilder Proxy

Implementing ClassBuilderInterceptorExtension lets us proxy the built‑in ClassBuilder . We intercept:

defineClass to add a marker interface for runtime identification.

newField to suppress generation of fields belonging to delegated serialized properties.

newMethod to suppress the synthetic getter/setter generated for those delegated properties.

All other elements are delegated to the original builder.

Code Generation

The core generation consists of three parts:

Write the actual backing fields and accessors for delegated properties.

Rewrite existing constructors to assign values to both primary‑constructor parameters and delegated fields, then execute init blocks.

Insert a new constructor that accepts a ProtoReader and performs stream‑based decoding, handling collections, default values, and nullability according to the strategy rules.

For collection types (List, Set, Array, primitive arrays), the generated constructor first creates an empty collection, fills it while reading the stream, and finally assigns it to the backing field. Nullability handling follows these rules:

Non‑repeated, non‑map fields declared nullable fall back to null .

Non‑repeated, non‑map fields declared non‑nullable fall back to the specified default or the language default.

Repeated or map fields fall back to an empty collection.

5. Summary

Overall, developing a Kotlin compiler plugin is more challenging than typical Android development due to limited documentation. The following lessons may help future contributors:

Feature Design

Design features to be intuitive and ready‑to‑use out of the box.

Avoid heavy reliance on IDE plugins or changes to language syntax.

Minimize coupling with Kotlin’s internal code to reduce maintenance overhead.

Code Generation

Perform thorough syntax checks via AnalysisHandlerExtension before generating bytecode.

Encapsulate complex logic in runtime libraries; keep the generated bytecode simple and robust.

Familiarize yourself with ASM, as error messages can be cryptic.

Write unit tests using kotlin-compile-testing to compile snippets on the fly.

IR Backend

The IR backend became default after Kotlin 1.5. Since many projects still target 1.3.x, focusing on the classic front‑end/back‑end is sufficient for now.

Reference Sources

kotlin/plugins/android-extensions/android-extensions-compiler – Android extensions compiler plugin.

kotlin/plugins/kapt3/kapt3-compiler – KAPT source, demonstrates front‑end analysis.

kotlin/plugins/kotlin-serialization/kotlin-serialization-compiler – Kotlin serialization plugin, similar in spirit.

https://github.com/google/ksp – Google’s KSP, the next‑generation annotation processor.

---END---