How to Slash Android Build Times with Incremental Transform and DexMerge Tweaks

Problem Statement

In large Android applications the build time of incremental builds can be several minutes, which is unacceptable for rapid development. The default incremental compilation provided by the Android Gradle Plugin (AGP) does not handle the most time‑consuming tasks efficiently in medium‑to‑large projects.

Root Cause Analysis

The two dominant contributors to build latency in AGP 4.2.1 are:

Transform tasks – custom transforms for privacy scanning, automatic instrumentation, etc., often take minutes per incremental build.

dexMerge task – merging the dex files generated by previous steps consumes 35‑40 s for the mus project and up to 90‑100 s for the Cloud Music project.

Classification of Transforms

Transforms can be divided into two categories:

Functional transforms – they only affect their own functionality (e.g., instrumentation, privacy scanning) and do not change the final artifact. These can be disabled with build flags or debug/release checks.

Strong‑dependency transforms – they generate classes during the apt phase that are required at runtime. Flattening them with ByteX‑style tools is possible but introduces high intrusiveness.

The chosen approach modifies the inputs of the Transform pipeline instead of replacing the whole Transform implementation, achieving a lightweight incremental build.

Incremental Transform Implementation

Although many Transforms report isIncremental = true, they rarely implement incremental I/O. Consequently, unchanged inputs are still copied on every incremental build.

Typical (non‑incremental) Transform code:

input.jarInputs.each { JarInput jarInput ->
    File destFile = transformInvocation.getOutputProvider()
        .getContentLocation(destName, jarInput.contentTypes, jarInput.scopes, Format.JAR)
    FileUtils.copyFile(srcFile, destFile)
}

input.directoryInputs.each { DirectoryInput directoryInput ->
    File destFile = transformInvocation.getOutputProvider()
        .getContentLocation(directoryInput.name, directoryInput.contentTypes, directoryInput.scopes, Format.DIRECTORY)
    FileUtils.copyDirectory(directoryInput.file, destFile)
}

Incremental handling should copy only changed inputs:

// Pseudo‑code for incremental handling
if (!isIncremental) return

if (Status.ADDED == jarInput.status || Status.CHANGED == jarInput.status) {
    File destFile = transformInvocation.getOutputProvider()
        .getContentLocation(destName, jarInput.contentTypes, jarInput.scopes, Format.JAR)
    FileUtils.copyFile(srcFile, destFile)
}

if (Status.ADDED == dirInput.status || Status.CHANGED == dirInput.status) {
    dirInput.changedFiles.forEach {
        // instrumentation logic …
        copyFileToTarget(it, dest)
    }
}

For legacy Transforms that cannot be modified, a generic hook replaces the list returned by TransformInvocation#getInputs, filtering out unchanged JarInput and DirectoryInput entries.

dexMerge Incremental Optimization

AGP version differences affect where the time is spent: AGP 3.x spends most time in dexBuilder, while AGP 4.x (including 4.2+) spends it in dexMerger. The optimization therefore targets the dexMerger step.

Bucket‑Based Merging

The dexMerger groups inputs into a default of 16 buckets based on package name. Classes in the same package share a bucket.

fun getBucketNumber(relativePath: String, numberOfBuckets: Int): Int {
    val packagePath = File(relativePath).parent
    return if (packagePath.isNullOrEmpty()) {
        0
    } else {
        when (numberOfBuckets) {
            1 -> 0
            else -> {
                // Same‑package classes go to the same bucket
                val normalizedPackagePath = File(packagePath).invariantSeparatorsPath
                kotlin.math.abs(normalizedPackagePath.hashCode()) % (numberOfBuckets - 1) + 1
            }
        }
    }
}

val File.invariantSeparatorsPath: String
    get() = if (File.separatorChar != '/') path.replace(File.separatorChar, '/') else path

If a JAR changes (status CHANGED or REMOVED), all buckets are re‑merged. When only new JARs or changed directories appear, only the impacted buckets are re‑merged.

private fun getImpactedBuckets(fileChanges: SerializableFileChanges, numberOfBuckets: Int): Set<Int> {
    val hasModifiedRemovedJars = (fileChanges.modifiedFiles + fileChanges.removedFiles)
        .any { isJarFile(it.file) }
    if (hasModifiedRemovedJars) {
        // All buckets need full dexMerge
        return (0 until numberOfBuckets).toSet()
    }
    // New jars or changed classes – compute affected buckets only
    val addedJars = fileChanges.addedFiles.map { it.file }.filter { isJarFile(it) }
    val dexPathsInAddedJars = addedJars.flatMap { getSortedRelativePathsInJar(it, ::isDexFile) }
    val dexPathsInChangedDirs = fileChanges.fileChanges.map { it.normalizedPath }.filter { isDexFile(it) }
    return (dexPathsInAddedJars + dexPathsInChangedDirs)
        .map { getBucketNumber(it, numberOfBuckets) }
        .toSet()
}

Componentized projects often trigger a full re‑merge when any sub‑module changes. Two practical solutions are proposed:

Unpack all JARs into directories so that only the affected bucket is re‑merged (still limited by bucket granularity).

Hook the dexMerge task to accept only the changed inputs, moving incremental dex files to a temporary directory and feeding that directory as the sole input for the next merge.

Hot‑Update Workflow

Incremental dex files can be pushed to a device’s sdcard and loaded at runtime. The PathClassLoader loads dex elements in order; therefore the incremental dex files must be inserted at the front of the dexElements array.

Temporary Incremental Dex Directory

Changed dex files are moved to a folder named pulledMergeDex. Using adb, the folder on the device is cleared before the new dex files are pushed, guaranteeing that only the latest incremental dex files are present.

Runtime Dynamic Dex Loading

At runtime the dexElements array of the class loader is reordered so that the newly pushed dex files appear first, enabling immediate execution of patched code.

Dex Reordering in the APK

To avoid a full dex re‑merge on each build, empty slots are reserved in classes.dex, classes2.dex, etc. Incremental dex files are inserted after the original ones. The DexIncrementalRenameManager maintains a mapping file ( dex-renamer-state.txt) that records the relationship between original and incremental dex files.

public void updateFiles() throws IOException {
    // Calculate packagedFileUpdates
    List<PackagedFileUpdate> packagedFileUpdates = new ArrayList<>();
    // Add dex changes
    packagedFileUpdates.addAll(mDexRenamer.update(mChangedDexFiles));
    deleteFiles(packagedFileUpdates);
    addFiles(packagedFileUpdates);
}

private void deleteFiles(@NonNull Collection<PackagedFileUpdate> updates) throws IOException {
    Predicate<PackagedFileUpdate> deletePredicate =
        mApkCreatorType == ApkCreatorType.APK_FLINGER
            ? p -> p.getStatus() == REMOVED || p.getStatus() == CHANGED
            : p -> p.getStatus() == REMOVED;
    // ... delete logic ...
}

private void addFiles(@NonNull Collection<PackagedFileUpdate> updates) throws IOException {
    Predicate<PackagedFileUpdate> isNewOrChanged =
        pfu -> pfu.getStatus() == FileStatus.NEW || pfu.getStatus() == CHANGED;
    // ... add logic ...
}

The mapping file is updated after each incremental merge, ensuring the APK always contains the correct dex ordering.

Observed Performance Gains

After applying the incremental Transform and dexMerge optimizations, the dexMerge time for mus dropped from 35‑40 s to roughly 3 s. A typical incremental build that modifies a single Kotlin line now finishes in 10‑40 s depending on the machine, compared with the original 20‑60 s range.

Conclusion

Targeted incremental handling of Transform inputs and a bucket‑aware incremental dexMerger dramatically reduce Android build times for large, componentized projects. Further reductions are possible by optimizing Kotlin compilation, KAPT, and other tasks, and by integrating the presented hooks into the standard Gradle build pipeline.