How to Add On‑Device AI Scanning to Your Android App with ML Kit

Mobile development is undergoing an AI democratization wave. Functions that once required months of effort can now be accessed via ML Kit's simple APIs. This article shares practical experience integrating ML Kit into Android apps, covering its core advantages and step‑by‑step implementation.

Core advantages of ML Kit

Zero‑learning curve : Developers can use ML Kit without understanding neural networks or tuning models; the library handles model computation and image analysis as a “smart black box”.

Privacy‑by‑design : All processing runs on‑device, so sensitive data never leaves the phone, simplifying compliance.

Production‑grade reliability : Google’s flagship apps use the same underlying technology, and tests on low‑end devices show high recognition rates even in poor lighting.

Optimized for mobile : Automatic GPU/CPU adaptation yields fast responses (typically under 0.5 s) without draining battery.

Full development tutorial

Below is a practical Android example that adds barcode scanning and optical character recognition (OCR) using Jetpack Compose and CameraX.

Step 1 – Add ML Kit dependencies

Include only the required modules in build.gradle to keep the APK size small.

implementation("com.google.mlkit:barcode-scanning:16.2.0") // barcode
implementation("com.google.mlkit:text-recognition:16.1.0") // OCR
implementation("androidx.camera:camera-core:1.4.2")
implementation("androidx.camera:camera-camera2:1.4.2")
implementation("androidx.camera:camera-lifecycle:1.4.2")
implementation("androidx.camera:camera-view:1.4.2")

Models are downloaded automatically on first launch, so they are not bundled with the APK.

Step 2 – Build a basic barcode scanner

Configure the scanner with only the formats you need to improve speed and accuracy.

class BarcodeProcessor {
    private val configuredFormats = mutableSetOf(
        Barcode.FORMAT_QR_CODE,
        Barcode.FORMAT_UPC_A,
        Barcode.FORMAT_EAN_13,
        Barcode.FORMAT_PDF417,
        Barcode.FORMAT_CODABAR
    )
    private val scanner = BarcodeScanning.getClient(
        BarcodeScannerOptions.Builder()
            .setBarcodeFormats(configuredFormats)
            .build()
    )
    suspend fun process(image: InputImage): List<Barcode> = try {
        scanner.process(image).await()
    } catch (e: Exception) {
        emptyList()
    }
}

Remember to close imageProxy after each analysis to avoid memory leaks.

class RealtimeBarcodeAnalyzer(
    private val onBarcodesDetected: (List<Barcode>) -> Unit,
    private val onError: (Exception) -> Unit
) : ImageAnalysis.Analyzer {
    private var lastProcessTime = 0L
    private val processingInterval = 150L

    override fun analyze(imageProxy: ImageProxy) {
        val currentTime = System.currentTimeMillis()
        if (currentTime - lastProcessTime < processingInterval) {
            imageProxy.close()
            return
        }
        lastProcessTime = currentTime
        val mediaImage = imageProxy.image ?: run { imageProxy.close(); return }
        val image = InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
        processImage(image)
            .onSuccess { onBarcodesDetected(it) }
            .onFailure { onError(it) }
            .also { imageProxy.close() }
    }
}

Step 3 – Comprehensive error handling

Map various exceptions to user‑friendly messages.

class MLKitErrorHandler {
    fun mapExceptionToState(exception: Exception): ScanningState = when (exception) {
        is MlKitException -> when {
            exception.errorCode == MlKitException.UNAVAILABLE ->
                ScanningState.ModelLoading("Scanning component initializing…")
            exception.message?.contains("insufficient storage") == true ->
                ScanningState.StorageIssue("Please free storage to enable scanning")
            else -> ScanningState.NetworkRequired("First use requires network")
        }
        is SecurityException -> ScanningState.PermissionDenied("Camera permission required")
        is CameraAccessException -> ScanningState.CameraUnavailable("Camera is in use by another app")
        else -> ScanningState.GenericError("Scanning error: ${exception.localizedMessage}")
    }
}

Step 4 – CameraX integration

Set up a preview with the optimal resolution (1280 × 720) to balance speed and accuracy.

@Composable
fun BarcodeScannerScreen() {
    val context = LocalContext.current
    val lifecycleOwner = LocalLifecycleOwner.current
    val cameraProviderFuture = remember { ProcessCameraProvider.getInstance(context) }

    LaunchedEffect(cameraProviderFuture) {
        val cameraProvider = cameraProviderFuture.get()
        val imageAnalysis = ImageAnalysis.Builder()
            .setTargetResolution(Size(1280, 720))
            .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
            .build()
        val analyzer = RealtimeBarcodeAnalyzer(
            onBarcodesDetected = { /* handle results */ },
            onError = { /* show toast */ }
        )
        imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(context), analyzer)
        cameraProvider.unbindAll()
        cameraProvider.bindToLifecycle(
            lifecycleOwner,
            CameraSelector.DEFAULT_BACK_CAMERA,
            imageAnalysis
        )
    }
    Box(modifier = Modifier.fillMaxSize()) {
        Text("Align the barcode within the frame", color = Color.White,
            modifier = Modifier.align(Alignment.BottomCenter).padding(20.dp))
    }
}

Step 5 – OCR for text extraction

Use the text recognizer, optionally selecting language‑specific options.

class DocumentTextProcessor {
    private val recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)

    suspend fun extractTextFromImage(image: InputImage): TextExtractionResult = try {
        val result = recognizer.process(image).await()
        TextExtractionResult.Success(
            text = result.text,
            confidence = calculateAverageConfidence(result.textBlocks)
        )
    } catch (e: Exception) {
        TextExtractionResult.Failure(message = "Text recognition failed: ${e.message}")
    }

    private fun calculateAverageConfidence(textBlocks: List<Text.TextBlock>): Float {
        if (textBlocks.isEmpty()) return 0f
        val total = textBlocks.sumOf { block ->
            block.lines.sumOf { line ->
                line.elements.sumOf { it.confidence.toDouble() }
            }
        }
        val count = textBlocks.sumOf { it.lines.sumOf { line -> line.elements.size } }
        return if (count > 0) (total / count).toFloat() else 0f
    }
}
sealed class TextExtractionResult {
    data class Success(val text: String, val confidence: Float) : TextExtractionResult()
    data class Failure(val message: String) : TextExtractionResult()
}

ML Kit supports many languages, including Western scripts, Chinese (via ChineseTextRecognizerOptions), Indian scripts, Japanese, Korean, and more.

Step 6 – Performance tuning

Limit processing frequency and use the optimal 720 p resolution to keep CPU usage low and prevent overheating during prolonged scanning.

class PerformanceOptimizedScanner {
    companion object {
        private const val OPTIMAL_RESOLUTION = Size(1280, 720)
        private const val PROCESSING_THROTTLE_MS = 150L
    }
    // Scanner initialization with selected formats...
}

Following these steps lets you add on‑device AI capabilities such as barcode scanning and OCR to Android apps while maintaining privacy, reliability, and battery efficiency.