Developing A Media Streaming App Using SwiftUI In 7 Days

These days, app development has become the web design of the early years of the millennia. Anyone can do it and the entry barrier is lower than ever, especially with Apple’s 2019 release of SwiftUI. It is the latest framework for declarative development of user interfaces on every platform in the Apple technology ecosystem. With the updates presented at WWDC 2020, it felt like a production ready environment for SwiftUI only app development for iPhone, iPad, Apple Watch, Apple TV or even the Mac.

In late 2020, I was working on the web platform for the Alles Neu Land online festival with live video streams in late 2020. I decided to augment the festival offering with a media library app. It would allow asynchronous streaming of recorded festival sessions for registered attendees. With time running out, I had to develop the app within few days to complete the review process of the App Store in due time before the festival launch on October 30, 2020. For all intends and purposes the goal was also to use SwiftUI exclusively for the interface and as many of the latest technologies as possible.

The entire development process took less than a week. I collaborated on the app with Marco Falanga(who also collaborated with me on the heart/work project). I estimate the total development hours at around 40-50 for both of us combined. For the media library app we heavily relied on SwiftUI. We implemented the backend service on AWS. Including the review process, the app went from inception to publication in 21 days. Most of the time however, we spent on the App Store Review process.

Talking Specifics

The app is pretty straight forward. It is a simple tab bar navigation app that serves a library view for the video content. Nested inside are various horizontal scroll views in a table view with different sections and detail views for each video. The detail views provide a video player with automatic bandwidth detection to pick stream quality. Of course, there are also video descriptions, presenter info and other meta data. The app also features a search view to query videos and presenters. In a favourite view users can save videos or presenters in a list. A presenter view with a grid layout and modal detail views, allows users to explore short presenter bios. In an account view, users can either enter their credentials and buy library access through an In-App Purchase subscription. All views rely on core functionality of the SwiftUI framework.

The app connects to a cloud based backend that provides authentication services and access to presenter and video data. The backend also serves HLS video streams in different qualities ranging from 360p up to 1080p. The backend service also uses means to limit account abuse. It limits the number of devices that can stream video content with the same credentials and manages the access rights. We localised both backend service and app in English, German and Italian.

SwiftUI and other latest technologies

We developed the app using only SwiftUI interface components and it runs on iOS for iPhone and iPad. With the latest MacOS Big Sur, the app can also run on the Mac, using the iPad layout. With only few adaptations it also runs on Apple TV, which is a natural extension of the app and its use case. Other latest technologies used in the app are Lazy Stacks and Grids, Combine, AppStorage, iCloud Storage, NSUbiquitousKeyValueStore, CryptoKit, Keychain, Sign in with Apple, Non-Renewing Subscriptions with In-App Purchase during the festival and Auto-Renewable Subscription afterwards, WidgetKit, MessageUI, AVKit, PictureInPicture and HTTP Live Streaming (HLS) technology with Low-Latency from Apple. For convenience, we moved some reusable elements into an external Swift Package for further usage in other projects using Swift Package Manager.

The backend service is running on cloud infrastructure using AWS. We chose S3 Storage, DynamoDB, CloudFront for storage and content delivery. We used API Gateway with AWS Lambdafor server-less computing with Swift AWS Lambda Runtime introduced at WWDC 2020. The video content was produced with Final Cut Pro and Compressor, transferred to S3 with MSP360 Explorer. This triggered Lambda scripts for AWS Elemental MediaConvert to create HLS video files. Manifest playlists for streaming at different resolutions and compressions to accommodate user’s different bandwidth requirements. Content is automatically added to the database and made available in the media library.

Access to the library is enabled either through valid In-App Purchase items procured through the App Store. During the festival period, also festival credentials from the festivals website’s payment system could be used. Following the privacy by design approach, no personal data except the email is saved in the app or the cloud. If desired, people can anonymise the email address when they Sign in with Apple. Anonymised and hashed IDs are used to identify valid credentials, returning users and their devices. Apart from anonymously registering and limiting the amount of devices using similar credentials, no data is collected about app usage. All data is transferred using Transport Layer Security (TLS), as is mandatorily requested by Apple’s App Transport Security (ATS)policy. Additionally, all user related data is stored encrypted both on the device and in the backend service.

User Interface Design

In an attempt to streamline the development process, we only spent limited time was on user interface design itself. We adopted icons, logos and colors from the Alles Neu Land brand guidelines without any time for mobile specific adjustments. The interface design, navigation and UI behaviour was modelled after best practices identified in a short UX research process. Also we tried to stay consistent with Apple’s Human Interface Guidelines for iOS. All views are using native interface components and rely on the core functionality of SwiftUI for building interfaces. For reasons of simplification, We used the default sans-serif typeface San Francisco throughout the app. All icons where chosen from the SF Symbols collection provided bye Apple. For a first MVP, we deemed the described approach sufficient. Future iterations of the UI might entail a more refined design and use of more graphic elements, coolers and fonts.

We modelled the main library view and video detail views within the current constraints of SwiftUI’s layout capabilities. They mimic the main interface components of the Netflix media library. Same as in similar media library apps, the video detail view provides access to presenter views. It also suggests additional videos of the same presenter or the same category of videos. User can add both presenters and videos to favourites and they appear in the My List tab of the app.

The search tab provides a grid view layout for presenters and table view layout for videos. It is populated with the latest videos and their presenters and dynamically updates as the search query is entered. The presenter tab shows all presenter profiles in a grid layout with modal detail views, containing bios, profile links etc. Lastly, in the settings tab users can either enter credentials or register using Sign in with Apple. After registration they can purchase a Non-Renewing Subscriptions with In-App Purchase asa festival pass. After the festival we replaced this mechanism with a monthly subscription pass. It allows access to the media library that will keep growing and periodically adds new videos for its users.

Also links to the app’s terms of service and its privacy policy are provided. We implemented the new privacy label announced in June 2020 for the apps page in the App Store. It was added proactively as soon as the feature became available, ahead of becoming a mandatory requirement. In an attempt to be as current as possible with latest tools and technologies, the app felt quite up to date from a formal perspective.

Obviously, there is room for improvements in terms of UI layout, colors and icons. Still, the established design matches the Alles Neu Land experience on the web and runs consistently across all Apple platforms.

Review Process

Main obstacle of the app development process for the Alles Neu Land Festival was the App Store review process. During the few days of development, we registered a new Apple Developer Account. The app was the first app submitted through this account. Consequently we had surpass all administrative obstacles in this period. Purchasing the developer licence, providing all required information about the company, adding banking and tax information etc. took some time.

App store review guidelines for using In-App Purchase options with a corresponding web service were quite demanding. More critically, we submitted the app in a period where this exact mechanism was under the microscope.

The review process occurred during a period of substantial criticism surrounding Apple’s requirements and practices in the App Store. It evolved around a lawsuit between Epic Games and Apple in August 2020 that challenged Apple’s 30% revenue cut. Epic implemented changes in their Fortnite game to bypass the App Store payment system. This resulted in the app being removed from the store and leading to the lawsuit.

The Alles Neu Land App had no interest in bypassing the App Store payment system. It was was compliant with the requirements from the initial submission. Still, the review process took 2 weeks. It led to 3 revisions of the app with tiny changes. For example, we have to remove a link to the festival website from within the app, to be accepted.

The usual period of 1 or 2 days for the review was stretched considerable. For the first submissions, the app went into review quite quickly with very short term responses. The final step however took 10 days. The app went into review within few hours. It then remained at this stage for 9 days until it was accepted for release. We don’t know what prompted the delay. But it seems as apps with this type of mechanism were looked at much more rigorously in this period.

How it went?

Luckily, the app was ready for release on Oct 24, well before the festival’s start on October 30, 2020. All in all, we worked on the app few hours per day for 7 days until it was ready for submission. The process was quite effective and I am very satisfied with the outcome. The development was distributed among the two of us. We used Github for the code repository and a simplified Kanban board on Trello. It was a somewhat lean and highly iterative process.

We integrated TestFlight from the very first day to gather feedback from other stakeholders. We were able to come up with a working prototype and functional backend within 2 days. Certainly, SwiftUI made this process as efficient as one could imagine. Most of the development time was spent on fixing bugs in the UI and making the UI behaviour consistent across all views in the app.

Considering the 14 days of review process, the app went from inception to release within 21 days. In the end, various features made it into the app that were not considered part of the MVP initially. Even though there is a lot more to be done to evolve this app and leverage its full potential, it is safe to say that it can take hours not weeks or months to produce something publishable for the Apple technology ecosystem. In this case a production ready media streaming library. With SwiftUI, usable and functional apps can be prototyped faster than ever before.

You can find more information about the Alles Neu Land festival and platform on their website. The Alles Neu Land app can be downloaded for free on the app store. Full access to the media library requires a valid subscription.

In due time I might publish more details on the development of the client app with SwiftUI. It might be interesting for anyone enthusiastic about declarative frameworks. Also the development of the backend service provides insights for Swift aficionados. Even though I am really enthusiastic about Server Side Swift, especially with the latest evolution of Vapor, the possibility to use Swift on AWS Lambda opened a new world for backend service development with AWS using only Swift. There are many valid alternatives to Swift for backend development of course, but I really enjoyed the opportunity to put Apple’s latest technologies to the test.

This article was originally published at moritzrecke.com on March 5, 2021.