Implementing Multi‑Stream Switching in ExoPlayer

The article discusses the evolution of audio‑video streaming in China and the need for flexible bitrate and track switching (4K/1080P/720P/480P, audio quality, original/ accompaniment) in modern applications. It reviews common stream‑switching approaches such as DASH/HLS adaptive switching, dual‑player switching, dual‑decoder switching, and player‑restart switching, highlighting their advantages and limitations.

It then focuses on ExoPlayer, an open‑source Android media player, and explains how ExoPlayer can implement low‑cost multi‑stream switching by restarting decoders. The core ideas include using MergingMediaSource to combine multiple media sources, and leveraging DefaultTrackSelector to select tracks dynamically.

Example code for building a merged media source:

val mediaDataSourceFactory: DataSource.Factory = DefaultDataSource.Factory(this)
var array = ArrayList<MediaItem>()
var mediaSources = ArrayList<MediaSource>()
// Add 480p video (audio + video tracks)
array.add(MediaItem.fromUri("asset://android_asset/viNM94G2aJw000_G/Video@MV@480/data"))
// Add 1080p video (audio + video tracks)
array.add(MediaItem.fromUri("asset://android_asset/viNM94G2aJw000_G/Video@MV@1080/data"))
// Add high‑quality accompaniment audio
array.add(MediaItem.fromUri("asset://android_asset/viNM94G2aJw000_G/Audio@ACC@Q_1/data"))
// Add original vocal audio
array.add(MediaItem.fromUri("asset://android_asset/viNM94G2aJw000_G/Audio@ORI@Q_1/data"))
val mediaSourceFactory = DefaultMediaSourceFactory(mediaDataSourceFactory)
array.forEach { mediaSources.add(mediaSourceFactory.createMediaSource(it)) }
val targetMergingMediaSource = MergingMediaSource(mediaSources[0], mediaSources[1], mediaSources[2])

Switching video tracks is performed via DefaultTrackSelector :

public static void switchToOtherVideoTrack(ExoPlayer exoPlayer, @NotNull Tracks tracks, int width, int height) {
    if (tracks == null || exoPlayer == null) return;
    ImmutableList
groups = tracks.getGroups();
    for (Tracks.Group group : groups) {
        if (group == null) continue;
        if (group.getType() != C.TRACK_TYPE_VIDEO) continue;
        boolean selected = group.isSelected();
        if (selected) continue;
        for (int trackIndex = 0; trackIndex < group.length; trackIndex++) {
            Format trackFormat = group.getTrackFormat(trackIndex);
            if (trackFormat.width != width || trackFormat.height != height) continue;
            TrackSelectionParameters params = exoPlayer.getTrackSelectionParameters()
                .buildUpon()
                .setOverrideForType(new TrackSelectionOverride(group.getMediaTrackGroup(), ImmutableList.of(trackIndex)))
                .setTrackTypeDisabled(group.getType(), false)
                .build();
            exoPlayer.setTrackSelectionParameters(params);
        }
    }
}

The article also provides the internal logic of ExoPlayer when re‑selecting tracks after a decoder restart. A simplified excerpt of the method reselectTracksInternal is shown below:

private void reselectTracksInternal() throws ExoPlaybackException {
    float playbackSpeed = mediaClock.getPlaybackParameters().speed;
    MediaPeriodHolder periodHolder = queue.getPlayingPeriod();
    MediaPeriodHolder readingPeriodHolder = queue.getReadingPeriod();
    boolean selectionsChangedForReadPeriod = true;
    TrackSelectorResult newTrackSelectorResult;
    while (true) {
        if (periodHolder == null || !periodHolder.prepared) return;
        newTrackSelectorResult = periodHolder.selectTracks(playbackSpeed, playbackInfo.timeline);
        if (!newTrackSelectorResult.isEquivalent(periodHolder.getTrackSelectorResult())) break;
        if (periodHolder == readingPeriodHolder) selectionsChangedForReadPeriod = false;
        periodHolder = periodHolder.getNext();
    }
    if (selectionsChangedForReadPeriod) {
        // Update streams, possibly recreate them, and reset renderer positions
        // ... (omitted for brevity) ...
    } else {
        // Early‑playback case: remove periods after the changed one and re‑prepare
        // ... (omitted for brevity) ...
    }
    handleLoadingMediaPeriodChanged(true);
    if (playbackInfo.playbackState != Player.STATE_ENDED) {
        maybeContinueLoading();
        updatePlaybackPositions();
        handler.sendEmptyMessage(MSG_DO_SOME_WORK);
    }
}

Alignment (synchronization) between audio and video is achieved without explicit seek calls. ExoPlayer computes a suitable SeekPoint (typically the nearest IDR frame) and resets all renderers to a unified playback position. The process includes resetting sample queues, marking early buffers with C.BUFFER_FLAG_DECODE_ONLY , and ensuring that only buffers aligned with the target timestamp are rendered.

for (SampleQueue sampleQueue : sampleQueues) {
    sampleQueue.setStartTimeUs(pendingResetPositionUs);
}
if (buffer.timeUs < startTimeUs) {
    buffer.addFlag(C.BUFFER_FLAG_DECODE_ONLY);
}
if (isDecodeOnlyBuffer && !isLastBuffer) {
    skipOutputBuffer(codec, bufferIndex, presentationTimeUs);
    return true;
}

The evaluation notes that ExoPlayer’s internal switching is fast, though it may exhibit slight stutter compared with native DASH/HLS adaptive switching. Nevertheless, for small teams lacking extensive resources, this method offers a practical solution.

In summary, ExoPlayer provides a robust, extensible framework for multi‑stream switching, supporting MergingMediaSource, DefaultTrackSelector‑based track changes, and decoder‑restart techniques. Developers are advised to prefer ExoPlayer over Android's native MediaPlayer for professional audio‑video applications.