Mastering Android MediaCodec: From Creation to End‑of‑Stream Processing

In the previous article we covered MediaCodec fundamentals such as its operation, data types, and lifecycle. This article explains the practical usage flow from creation to termination.

Creation

Instantiate a MediaCodec with a specific MIME type and MediaFormat. Two typical scenarios are shown: creating an encoder for camera‑generated video and creating an encoder for re‑encoding extracted video streams.

//create MediaFormat by mime-type
MediaFormat format = MediaFormat.createVideoFormat("video/avc", nEncWidth, nEncHeight);
format.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface);
format.setInteger(MediaFormat.KEY_BIT_RATE, nEncBitsRate);
format.setInteger(MediaFormat.KEY_FRAME_RATE, nEncFrameRate);
format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 2);
//create Encoder
mEncoder = MediaCodec.createEncoderByType("video/avc");

To obtain the media format of a video file:

private void init() throws IOException {
    MediaExtractor extractor = new MediaExtractor();
    extractor.setDataSource("/sdcard/test.mp4");
    int trackCount = mExtractor.getTrackCount();
    for (int i = 0; i < trackCount; i++) {
        MediaFormat format = mExtractor.getTrackFormat(i);
        String mime = format.getString(MediaFormat.KEY_MIME);
        if (mime.startsWith("video/")) {
            mVideoFormat = format;
            mVideoTrackIndex = i;
        } else if (mime.startsWith("audio/")) {
            mAudioFormat = format;
            mAudioTrackIndex = i;
        }
    }
}

Initialization

After creating the codec, call configure. For raw video data, a simple configure is enough; to use a Surface, pass the surface parameter. Asynchronous processing can be enabled via setCallback. The codec can be set as encoder or decoder using appropriate flags.

To feed raw video into an encoder, create an input Surface with createInputSurface and link it to the producer via setInputSurface.

encoder = MediaCodec.createEncoderByType(MIME_TYPE);
encoder.configure(outputFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
inputSurface = new InputSurface(encoder.createInputSurface());
encoder.start();

decoder = MediaCodec.createDecoderByType(MIME_TYPE);
outputSurface = new CodecOutputSurface();
decoder.configure(mVideoFormat, outputSurface.getSurface(), null, 0);
decoder.start();

Data Processing

After start, data can be processed synchronously or asynchronously. Prior to API 23 only synchronous methods are available; from API 23 onward asynchronous callbacks are supported.

Synchronous processing using ByteBuffer (recommended for API 21+):

MediaCodec codec = MediaCodec.createByCodecName(name);
codec.configure(format, …);
codec.start();
for (;;) {
    int inputBufferId = codec.dequeueInputBuffer(timeoutUs);
    if (inputBufferId >= 0) {
        ByteBuffer inputBuffer = codec.getInputBuffer(...);
        // fill inputBuffer
        codec.queueInputBuffer(inputBufferId, …);
    }
    int outputBufferId = codec.dequeueOutputBuffer(...);
    if (outputBufferId >= 0) {
        ByteBuffer outputBuffer = codec.getOutputBuffer(outputBufferId);
        MediaFormat bufferFormat = codec.getOutputFormat(outputBufferId);
        // process outputBuffer
        codec.releaseOutputBuffer(outputBufferId, …);
    } else if (outputBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
        outputFormat = codec.getOutputFormat();
    }
}
codec.stop();
codec.release();

Synchronous processing using ByteBuffer arrays (pre‑API 21):

MediaCodec codec = MediaCodec.createByCodecName(name);
codec.configure(format, …);
codec.start();
ByteBuffer[] inputBuffers = codec.getInputBuffers();
ByteBuffer[] outputBuffers = codec.getOutputBuffers();
for (;;) {
    int inputBufferId = codec.dequeueInputBuffer(...);
    if (inputBufferId >= 0) {
        // fill inputBuffers[inputBufferId]
        codec.queueInputBuffer(inputBufferId, …);
    }
    int outputBufferId = codec.dequeueOutputBuffer(...);
    if (outputBufferId >= 0) {
        // process outputBuffers[outputBufferId]
        codec.releaseOutputBuffer(outputBufferId, …);
    } else if (outputBufferId == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
        outputBuffers = codec.getOutputBuffers();
    } else if (outputBufferId == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
        MediaFormat format = codec.getOutputFormat();
    }
}
codec.stop();
codec.release();

Asynchronous processing with callbacks:

MediaCodec codec = MediaCodec.createByCodecName(name);
codec.setCallback(new MediaCodec.Callback() {
    @Override
    void onInputBufferAvailable(MediaCodec mc, int inputBufferId) {
        ByteBuffer inputBuffer = codec.getInputBuffer(inputBufferId);
        // fill inputBuffer
        codec.queueInputBuffer(inputBufferId, …);
    }
    @Override
    void onOutputBufferAvailable(MediaCodec mc, int outputBufferId, MediaCodec.BufferInfo info) {
        ByteBuffer outputBuffer = codec.getOutputBuffer(outputBufferId);
        MediaFormat bufferFormat = codec.getOutputFormat(outputBufferId);
        // process outputBuffer
        codec.releaseOutputBuffer(outputBufferId, …);
    }
    @Override
    void onOutputFormatChanged(MediaCodec mc, MediaFormat format) {
        mOutputFormat = format;
    }
    @Override
    void onError(MediaCodec mc, MediaCodec.CodecException e) {
        // handle error
    }
});
codec.configure(format, …);
codec.start();
// wait for processing, then stop/release
codec.stop();
codec.release();

End‑of‑Stream Handling

When the input data ends, signal the codec by setting BUFFER_FLAG_END_OF_STREAM on the last input buffer (or submit an empty buffer with the flag). The codec will continue delivering output buffers until it reports end‑of‑stream via MediaCodec.BufferInfo or the callback. After signaling EOS, do not queue additional input buffers unless the codec is reset.

Improper state handling can cause IllegalStateException during encoding or decoding.

With these steps, the complete MediaCodec encode/decode lifecycle is covered. The next article will demonstrate a concrete use‑case.