Scoring the aesthetics of an image with the Vision framework

Evaluating the aesthetic quality is an image processing task more common than often assumed: it is used to highlight visually appealing content in applications like photo galleries, assist the user in the selection of the best photos from a set, or suggest images over others.

In this reference article, we are going to explore how to use correctly CalculateImageAestheticsScoresRequest, the API responsible for the analysis and the evaluation of the aesthetic quality of images, introduced with the redesign of the Vision framework.

Calculate the aesthetics score of an image

CalculateImageAestheticsScoresRequest leverages machine learning models to assess the image quality and distinguish memorable photos from utility ones. It analyzes those factors that concur in the visual appeal of an image - such as blur and exposure, color balance, composition, and subject matter - and represents them in a property, the overallScore, a float ranging from -1 to 1, where higher values indicate a more aesthetically pleasing image.

To start assessing an image quality, import the Vision framework.

Declare a function that receives the image to be analyzed as a parameter.

import Vision

func calculateAesthicsScore(image: UIImage) async throws -> ImageAestheticsScoresObservation? {
    // 1. Image to be processed
    guard let ciimage = CIImage(image: image) else { return nil }
    
    // 2. Set up the calculate image aesthetics scores request
    let request = CalculateImageAestheticsScoresRequest()
    
    // 3. Perform the request
    let observation = try await request.perform(on: ciimage)
    
    // 4. The resulting ImageAestheticsScoresObservation object
    return observation
}

The function calculateAesthicsScore(image:) works as follows:

1. It declares a constant image storing the image as CIImage to be processed.
2. It sets up the CalculateImageAestheticsScoresRequest object.
3. It performs the request on the image. For this request, the method perform(on:orientation:) doesn’t require the image's orientation to be specified as the resulting observation does not include the coordinates. In fact, the orientation parameter might be needed when the method is called by a request whose observation result returns coordinates too, which at that point must be normalized from the Vision’s coordinate system to the SwiftUI one.
4. The request returns an ImageAestheticsScoresObservation that stores the quality of the image in the overallScore property. It also includes the isUtility property, a bool that specifies whether it’s a utility image or not. Utility images are those images like screenshots, photos of receipts, documents or part of it, etc., that even though they may have a high overall score, are not considered memorable photos to share.

Integration with SwiftUI

import SwiftUI
import Vision

struct ContentView: View {

    @State var result: ImageAestheticsScoresObservation? = nil
    
    var body: some View {
    
        VStack {
            Image("docs")
                .resizable()
                .scaledToFit()
                .clipShape(.rect(cornerRadius: 15))
                .padding()
            
            Button(action: {
                self.calculateAesthetics()
            }, label: {
                Text("Calculate Aesthetics")
            })
            
            if let result = result {
                Text("Score is \\(result.overallScore)")
                Text(result.isUtility ? "It's a utility picture" :"This is a memorable photo")
            }
            
        }.padding()
    }
    
    private func calculateAesthetics() {
        Task{
            do{
                result = try await calculateAesthicsScores(image: UIImage(named: "docs")!)
            } catch {
                print("Error calculating image aesthetics scores: \\(error)")
            }
        }
    }
}

In this example, a button triggers the evaluation of the quality of the image and a text displays the overallScore result highlighting whether the image contains memorable images’ content.

CalculateImageAestheticsScoresRequest offers an easy and fast way to evaluate the aesthetic quality of images and detect memorable image content in applications. This request processes the image and returns aesthetic information in the properties of the ImageAestheticsScoresObservation returning object. In this way, developers can automate the evaluation of images based on a series of aesthetic criteria, making it easier to manage and present visually appealing content.