Adaptive Stream Switching and Multi‑Track Selection in ExoPlayer

Adaptive stream switching is one of the methods for multi‑track switching. ExoPlayer, as a comprehensive MediaCodec wrapper, supports both MergingMediaSource‑based track merging and adaptive streaming based on protocols such as MPEG‑DASH, HLS and smoothing‑stream. The choice between the two approaches depends on team size, backend capabilities and resource constraints.

Main differences :

MergingMediaSource is suitable when manpower and backend services are limited; it works with a simple CDN deployment and incurs lower cost. Adaptive streaming requires more professional server setup, resource segmentation and encoding.

MergingMediaSource can combine streams of different encodings, while adaptive protocols (e.g., HLS) impose stricter requirements on TS segment encoding consistency to enable MediaCodec reuse.

In ExoPlayer, tracks of the same type inside a MergingMediaSource lack bitrate parameters, so FixedTrackSelection cannot automatically switch between them. Adaptive streaming can group Formats and create AdaptiveTrackSelection to manage dynamic switching.

Basic concepts :

Renderer – detects format support, registers and reuses decoders, reads samples and renders output.

MediaClock – handles audio‑video sync and playback progress (StandaloneMediaClock or Audio‑master clock).

Extractor – parses container metadata (Moov, sample tables, SPS/PPS, etc.) and prepares Tracks and Formats.

Decoder – performs actual decoding via MediaCodec, supporting hardware and software paths.

TrackSelector – core class for multi‑track switching; groups Formats into TrackGroups and matches them with Renderers.

DataSource – provides raw data.

MediaSource – abstracts a media stream; enables flexible multi‑track management.

SeekPoint – usually the start of an IDR frame.

FixedTrackSelection – keeps the selected track constant (used with MergingMediaSource).

AdaptiveTrackSelection – selects tracks dynamically based on bandwidth.

TrackGroup – groups Tracks of the same type.

MappedTrackInfo – holds Renderer‑TrackGroup mapping information.

Selection eligibility – SELECTION_ELIGIBILITY_FIXED / SELECTION_ELIGIBILITY_ADAPTIVE.

Bandwidth – measured by DefaultBandwidthMeter and used by AdaptiveTrackSelection.

DefaultMediaSourceFactory – creates the appropriate MediaSource (HlsMediaSource, DashMediaSource, etc.) from a DataSource.

Adaptive stream switching analysis

When network speed changes, ExoPlayer automatically switches to a media track whose bitrate fits the current bandwidth. The switch is driven by the playlist (manifest) which already defines the bitrate thresholds.

The default behavior does not require SeekPoint lookup; the next segment is selected directly based on bandwidth detection. Each segment’s duration and the start of its I‑frame must be strictly aligned.

Core logic steps :

Parse the playlist file.

Map Renderers to TrackGroups and Selection.

Start loading segments.

Detect bandwidth and let AdaptiveTrackSelection choose the appropriate segment.

Reuse or restart decoders.

Complete the switch.

Playlist parsing

ExoPlayer supports DASH, HLS and smoothing‑stream. The following excerpts show example HLS and DASH manifests.

#EXTM3U
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 1",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 2",AUTOSELECT=NO,DEFAULT=NO,URI="alternate_audio_aac/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",CHARACTERISTICS="public.accessibility.transcribes-spoken-dialog, public.accessibility.describes-music-and-sound",URI="subtitles/eng/prog_index.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=263851,CODECS="mp4a.40.2, avc1.4d400d",RESOLUTION=416x234,AUDIO="bipbop_audio",SUBTITLES="subs"
gear1/prog_index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=577610,CODECS="mp4a.40.2, avc1.4d401e",RESOLUTION=640x360,AUDIO="bipbop_audio",SUBTITLES="subs"
gear2/prog_index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=915905,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=960x540,AUDIO="bipbop_audio",SUBTITLES="subs"
gear3/prog_index.m3u8

tears_audio_eng.mp4
tears_h264_baseline_240p_800.mp4

Both manifests share common fields such as bandwidth , mimeType , codecs , width and height . Because the manifest provides format information before demuxing, ExoPlayer can decide track selection early.

The parsing flow uses HlsPlaylistParser for HLS, DashManifestParser for DASH, and SsManifestParser for smoothing‑stream.

Renderer‑TrackGroup‑Selection mapping

@Override
protected final Pair<@NullableType RendererConfiguration[], @NullableType ExoTrackSelection[]> selectTracks(
    MappedTrackInfo mappedTrackInfo,
    @Capabilities int[][][] rendererFormatSupports,
    @AdaptiveSupport int[] rendererMixedMimeTypeAdaptationSupport,
    MediaPeriodId mediaPeriodId,
    Timeline timeline) throws ExoPlaybackException {
  // ... (code omitted for brevity) ...
  ExoTrackSelection[] rendererTrackSelections =
      trackSelectionFactory.createTrackSelections(
          definitions, getBandwidthMeter(), mediaPeriodId, timeline);
  // ... configure tunneling, etc. ...
  return Pair.create(rendererConfigurations, rendererTrackSelections);
}

The factory creates either FixedTrackSelection (single track) or AdaptiveTrackSelection (multiple tracks):

@Override
public final @NullableType ExoTrackSelection[] createTrackSelections(
    @NullableType Definition[] definitions,
    BandwidthMeter bandwidthMeter,
    MediaPeriodId mediaPeriodId,
    Timeline timeline) {
  ImmutableList
> adaptationCheckpoints =
      getAdaptationCheckpoints(definitions);
  ExoTrackSelection[] selections = new ExoTrackSelection[definitions.length];
  for (int i = 0; i < definitions.length; i++) {
    @Nullable Definition definition = definitions[i];
    if (definition == null || definition.tracks.length == 0) continue;
    selections[i] = definition.tracks.length == 1
        ? new FixedTrackSelection(definition.group, /* track= */ definition.tracks[0], /* type= */ definition.type)
        : createAdaptiveTrackSelection(
            definition.group, definition.tracks, definition.type, bandwidthMeter, adaptationCheckpoints.get(i));
  }
  return selections;
}

Chunk loading

public boolean continueLoading(long positionUs) {
  if (loadingFinished || loader.isLoading() || loader.hasFatalError()) return false;
  boolean pendingReset = isPendingReset();
  List
chunkQueue;
  long loadPositionUs;
  if (pendingReset) {
    chunkQueue = Collections.emptyList();
    loadPositionUs = pendingResetPositionUs;
  } else {
    chunkQueue = readOnlyMediaChunks;
    loadPositionUs = getLastMediaChunk().endTimeUs;
  }
  chunkSource.getNextChunk(positionUs, loadPositionUs, chunkQueue, nextChunkHolder);
  // ... handle end of stream, start loading, etc. ...
  return true;
}

public final void getNextChunk(
    long playbackPositionUs,
    long loadPositionUs,
    List
queue,
    ChunkHolder out) {
  if (fatalError != null) return;
  StreamElement streamElement = manifest.streamElements[streamElementIndex];
  if (streamElement.chunkCount == 0) {
    out.endOfStream = !manifest.isLive;
    return;
  }
  int chunkIndex;
  if (queue.isEmpty()) {
    chunkIndex = streamElement.getChunkIndex(loadPositionUs);
  } else {
    chunkIndex = (int) (queue.get(queue.size() - 1).getNextChunkIndex() - currentManifestChunkOffset);
    if (chunkIndex < 0) {
      fatalError = new BehindLiveWindowException();
      return;
    }
  }
  // ... compute bufferedDurationUs, timeToLiveEdgeUs, build iterators ...
  trackSelection.updateSelectedTrack(
      playbackPositionUs, bufferedDurationUs, timeToLiveEdgeUs, queue, chunkIterators);
  // ... build Chunk and return ...
}

Bandwidth detection uses DefaultBandwidthMeter (or a custom implementation). The adaptive selection compares the estimated bitrate with the available bandwidth and applies buffering thresholds to avoid frequent switches.

private int determineIdealSelectedIndex(long nowMs, long chunkDurationUs) {
  long effectiveBitrate = getAllocatedBandwidth(chunkDurationUs);
  int lowestBitrateAllowedIndex = 0;
  for (int i = 0; i < length; i++) {
    if (nowMs == Long.MIN_VALUE || !isBlacklisted(i, nowMs)) {
      Format format = getFormat(i);
      if (canSelectFormat(format, format.bitrate, effectiveBitrate)) {
        return i;
      } else {
        lowestBitrateAllowedIndex = i;
      }
    }
  }
  return lowestBitrateAllowedIndex;
}

Decoder reuse logic decides whether the existing MediaCodec can be kept, flushed, re‑configured or must be recreated. Reuse reduces playback stalls during track switches.

protected DecoderReuseEvaluation onInputFormatChanged(FormatHolder formatHolder) throws ExoPlaybackException {
  Format newFormat = checkNotNull(formatHolder.format);
  if (newFormat.sampleMimeType == null) {
    throw createRendererException(new IllegalArgumentException(), newFormat, PlaybackException.ERROR_CODE_DECODING_FORMAT_UNSUPPORTED);
  }
  // ... DRM handling, codec initialization, reuse evaluation ...
  DecoderReuseEvaluation evaluation = canReuseCodec(codecInfo, oldFormat, newFormat);
  switch (evaluation.result) {
    case REUSE_RESULT_NO:
      drainAndReinitializeCodec();
      break;
    case REUSE_RESULT_YES_WITH_FLUSH:
      codec.flush();
      break;
    case REUSE_RESULT_YES_WITH_RECONFIGURATION:
      // update codec parameters, possibly re‑configure SPS/PPS
      break;
    case REUSE_RESULT_YES_WITHOUT_RECONFIGURATION:
      // reuse directly
      break;
    default:
      throw new IllegalStateException();
  }
  return evaluation;
}

Experiment

Purpose: manually switch between segments (e.g., 1920×1080 → 640×360) by controlling the bandwidth estimate.

Method: implement a custom QmBandwidthMeter that allows setting a specific bitrate, inject it into the ExoPlayer builder, and change the bitrate after a few seconds of playback.

private long bitrateEstimate;
private long specificBitrate = C.TIME_UNSET;
@Override
public synchronized long getBitrateEstimate() {
  if (specificBitrate == C.RATE_UNSET_INT) {
    return bitrateEstimate;
  }
  return specificBitrate;
}

// Builder configuration
bandwidthMeter = new QmBandwidthMeter.Builder(getApplicationContext()).build();
DefaultLoadControl loadControl = new DefaultLoadControl.Builder()
    .setAllocator(new DefaultAllocator(true, C.DEFAULT_BUFFER_SEGMENT_SIZE / 2))
    .setBufferDurationsMs(10000, 20000,
        DefaultLoadControl.DEFAULT_BUFFER_FOR_PLAYBACK_MS,
        DefaultLoadControl.DEFAULT_BUFFER_FOR_PLAYBACK_AFTER_REBUFFER_MS)
    .build();
ExoPlayer.Builder playerBuilder = new ExoPlayer.Builder(this)
    .setLoadControl(loadControl)
    .setBandwidthMeter(bandwidthMeter)
    .setMediaSourceFactory(createMediaSourceFactory());
player = playerBuilder.build();

During playback, the bitrate is set as follows (the division by 0.7 accounts for the safety margin used by the bandwidth meter):

// At start
bandwidthMeter.setSpecificBitrate((long) Math.ceil(1924009 / 0.7f));
// After 10 s
bandwidthMeter.setSpecificBitrate((long) Math.ceil(577610 / 0.7f));

Verification is done via the renderer callbacks AudioRendererEventListener#onAudioInputFormatChanged and VideoRendererEventListener#onVideoInputFormatChanged . The experiment confirmed that the player successfully switched to the lower‑bitrate track.

Conclusion

ExoPlayer supports both MergingMediaSource‑based multi‑track merging and adaptive streaming. By customizing the BandwidthMeter and AdaptiveTrackSelection parameters, developers can achieve smooth, user‑controlled bitrate switches. Important considerations include segment length, IDR‑frame alignment, consistent encoding (to enable decoder reuse), and buffering thresholds to avoid stalled switches.