Switching between HLS streams with AVKit

Recently, we explored HLS Streaming with AVKit and SwiftUI and how we can leverage different .m3u8 manifest files for media playlists. With that approach, videos can be made available in different resolutions and bandwidths to accommodate a variety of network conditions.

Even though HLS support intelligent detection of available bandwidth out of the box, sometimes it might be interesting to dig a little deeper and provide mechanisms to switch bandwidth based on user input as well as on our own estimation of the networking conditions.

One way to do that is by monitoring the buffer of the AVPlayer's current AVPlayerItem. The class provides multiple properties to determine the buffering status:

1. var isPlaybackLikelyToKeepUp: Bool
2. var isPlaybackBufferFull: Bool
3. var isPlaybackBufferEmpty: Bool

Most relevantly, isPlaybackBufferEmpty indicates whether all buffered media is consumed and that playback will stop, and isPlaybackLikelyToKeepUp indicates whether the item will play through without stalling.

Let's explore how that could work.

Creating a simplified Video Data Model

To keep things simple, we might use a Video struct with two properties, a name, and any number of streams as an array or Stream. Any Stream could have a resolution and a streamURL. In any real project, this model might be much more extensive and handle all kinds of metadata for any video, starting from the description, duration, poster, thumbnail, actors, directors, etc. To keep things simple, we will leave it as is. For specifying the resolution, we are using a Resolution enum and support 360p, 540p, 720p, 1080p.

struct Video {
    let name: String
    let streams: [Stream]
}

struct Stream {
    let resolution: Resolution
    let streamURL: URL
}

enum Resolution: Int, Identifiable, Comparable, CaseIterable {
    case p360 = 0
    case p540
    case p720
    case p1080
    
    var id: Int { rawValue }
    
    var displayValue: String {
        switch self {
        case .p360: return "360p"
        case .p540: return "540p"
        case .p720: return "720p"
        case .p1080: return "1080p"
        }
    }
    
    static func ==(lhs: Resolution, rhs: Resolution) -> Bool {
        lhs.rawValue == rhs.rawValue
    }
    
    static func <(lhs: Resolution, rhs: Resolution) -> Bool {
        lhs.rawValue < rhs.rawValue
    }
}

Creating a VideoPlayerViewModel

A simple implementation of a VideoPlayerViewModel might incorporate an AVPlayer and a Video property that could then have different features to select among the different streams offered inside the video item.

Making use of Combine, the ViewModel could use some @Published properties:

• A selectedResolution property of the type Resolution, to store the resolution the player is currently using.
• A shouldLowerResolution boolean property to indicate if the resolution should be lowered when the playback buffer is empty or playback is not likely to keep up.

The monitoring of the buffer could happen within a setObserver() function that periodically checks the buffer of the AVPlayerItem. If the resolution should be lowered, it is executed in the function lowerResolutionIfPossible().

The VideoPlayerViewModel could look like this:

import Foundation
import Combine
import AVKit

final class VideoPlayerViewModel: ObservableObject {
    @Published var selectedResolution: Resolution
    @Published private var shouldLowerResolution = false

    let player = AVPlayer()
    private let video: Video
    private var subscriptions: Set<AnyCancellable> = []
    private var timeObserverToken: Any?
    
    var name: String { video.name }
    var namePlusResolution: String { video.name + " at " + selectedResolution.displayValue }

    init(video: Video, initialResolution: Resolution) {
        self.video = video
        self.selectedResolution = initialResolution
        
        $shouldLowerResolution
            .dropFirst()
            .filter({ $0 == true })
            .sink(receiveValue: { [weak self] _ in
                guard let self = self else { return }
                self.lowerResolutionIfPossible()
            })
            .store(in: &subscriptions)
        
        $selectedResolution
            .sink(receiveValue: { [weak self] resolution in
                guard let self = self else { return }
                self.replaceItem(with: resolution)
                self.setObserver()
            })
            .store(in: &subscriptions)
    }
    
    deinit {
        if let timeObserverToken = timeObserverToken {
            player.removeTimeObserver(timeObserverToken)
        }
    }
    
    private func setObserver() {
        if let timeObserverToken = timeObserverToken {
            player.removeTimeObserver(timeObserverToken)
        }
        
        timeObserverToken = player.addPeriodicTimeObserver(forInterval: CMTimeMake(value: 1, timescale: 600), queue: DispatchQueue.main, using: { [weak self] time in
            guard let self = self,
                  let currentItem = self.player.currentItem else { return }
            
            guard currentItem.isPlaybackBufferFull == false else {
                self.shouldLowerResolution = false
                return
            }
            
            if currentItem.status == AVPlayerItem.Status.readyToPlay {
                self.shouldLowerResolution = (!currentItem.isPlaybackLikelyToKeepUp && !currentItem.isPlaybackBufferEmpty)
            }
        })
    }
    
    private func lowerResolutionIfPossible() {
        guard let newResolution = Resolution(rawValue: selectedResolution.rawValue - 1) else { return }
        selectedResolution = newResolution
    }
    
    private func replaceItem(with newResolution: Resolution) {
        guard let stream = self.video.streams.first(where: { $0.resolution == newResolution }) else { return }
        let currentTime: CMTime
        if let currentItem = player.currentItem {
            currentTime = currentItem.currentTime()
        } else {
            currentTime = .zero
        }
        
        player.replaceCurrentItem(with: AVPlayerItem(url: stream.streamURL))
        player.seek(to: currentTime, toleranceBefore: .zero, toleranceAfter: .zero)
    }
}

Switching between HLS streams with VideoPlayer and SwiftUI

An example implementation of a VideoPlayer in SwiftUI that makes use of the VideoPlayerViewModel described above could look like this:

import SwiftUI
import AVKit

struct BufferVideoPlayerView: View {
    
    @StateObject private var videoPlayerVM = VideoPlayerViewModel(video: Video(name: "Promo Video", streams: [
        Stream(resolution: .p360, streamURL: URL(string: "YOUR-VIDEO-360p.m3u8")!),
        Stream(resolution: .p540, streamURL: URL(string: "YOUR-VIDEO-540p.m3u8")!),
        Stream(resolution: .p720, streamURL: URL(string: "YOUR-VIDEO-720p.m3u8")!),
        Stream(resolution: .p1080, streamURL: URL(string: "YOUR-VIDEO-1080p.m3u8")!)
    ]), initialResolution: .p540)
 
    @State private var showResolutions = false
    
    var body: some View {
        ZStack {
            VStack(alignment: .leading) {
                Button("Change resolution") {
                    withAnimation {
                        showResolutions.toggle()
                    }
                }
                .font(Font.body.bold())
                VideoPlayer(player: videoPlayerVM.player) {
                    Text(videoPlayerVM.namePlusResolution)
                        .font(.subheadline)
                        .foregroundColor(.white)
                        .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading)
                        .padding(8)
                }
                .frame(height: 220)
                .onAppear() {
                    videoPlayerVM.player.play()
                }
            }
            .padding()
            if showResolutions {
                VStack(spacing: 20) {
                    Spacer()
                    ForEach(Resolution.allCases) { resolution in
                        Button(resolution.displayValue, action: {
                            withAnimation {
                                videoPlayerVM.selectedResolution = resolution
                                showResolutions.toggle()
                            }
                        })
                    }
                    Button(action: {
                        withAnimation {
                            showResolutions.toggle()
                        }
                    }, label: {
                        Image(systemName: "xmark.circle")
                            .imageScale(.large)
                    })
                        .padding(.top)
                    Spacer()
                }
                .frame(maxWidth: .infinity)
                .background(.thinMaterial)
                .transition(.move(edge: .bottom))
            }
        }
        .navigationBarTitle("Observing player's status")
    }
}

It uses VideoPlayerViewModel as a @StateObject and thus subscribes to any changes published by the view model while it's monitoring the AVPlayerItem during playback. Another @State property showResolutions is used to toggle a view that allows the user to switch resolutions.

To provide a deeper look at the implementation, have a look the corresponding Github repository with sample code to illustrate the VideoPlayer implementation in SwiftUI and how to monitor the playback buffer to switch to lower resolution streams based on network conditions.

The learn more about how to get started with HLS in SwiftUI, consider our article HLS Streaming with AVKit and SwiftUI. The approach was used in the Alles Neu Land Media Library app. To learn more about it, check out our article Developing A Media Streaming App Using SwiftUI In 7 Days. To learn more about how HTTP Live Streaming (HLS) works, explore the article Using HLS for live and on-demand audio and video.

This article is part of a series of articles derived from the presentation "How to create a media streaming app with SwiftUI and AWS" presented at the NSSpain 2021 conference on November 18-19, 2021.