Generating images programmatically with Image Playground

By using the new ImageCreator API, developers can programmatically generate images using Apple’s local models without relying on the Image Playground interface. This provides greater flexibility and integration possibilities within applications while ensuring that image generation remains efficient and secure.

To generate images, we need to provide three parameters:

• A textual prompt for the image
• The desired style
• The maximum number of images to be generated

By understanding and optimizing these parameters, we can fully leverage the API’s capabilities to create high-quality, customized images that meet our needs. Let’s explore how to make the most of this powerful tool.

The ImagePlaygroundConcept Object

By using an ImagePlaygroundConcept object, we can define the concepts that should be extracted and used to guide the image creation process. This allows for more precise control over how the model interprets and represents the desired content.

There are different solutions for providing the prompt:

• text(_:): a type method that creates a concept structure that includes a short text description.
• extracted(from:title:): a type method that creates a concept structure from a long-form string and a title that guides the creation of the image.
• image(_:): a type method that creates a concept starting from a CGImage or URL of a local file of an image.
• drawing(_:): a type method that creates a concept starting from a PencilKit drawing.

Depending on your needs you can use one of these methods to create the concept.

The ImagePlaygroundStyle object

When you create images programmatically, you can ask the system to create images in a particular style. The generative model takes the requested style option and applies it to the content it generates.

Currently, the following styles are available:

• animation: generate the image in a 3D-looking style similar to animated movies.
• illustration: generate the image in a flat 2D style.
• sketch: generate the image in hand-drawn sketch style.

Possible errors

Since Apple Intelligence models run just on specific Apple devices is likely that some errors occurs during the process, failing the process of image generation.

This are all the possible error that the ImageCreator object can return:

• notSupported: an error message indicating that the device lacks the capability to generate images.
• unavailable: an error that indicates image creation is unavailable.
• creationCancelled: an error that occurs in response to cancellation of the parent task.
• faceInImageTooSmall: an error that indicates the system cannot use one of the source images because the face in it is too small.
• unsupportedLanguage: an error that indicates the input text uses an unsupported language.
• unsupportedInputImage: an error that indicates the system cannot use one of the specified source images.
• backgroundCreationForbidden: an error that indicates the app is hidden or in the background.
• creationFailed: an error that indicates a general failure occurred during image creation.

Implementing on a SwiftUI app

Now let's use the ImageCreator API in our SwiftUI app to generate an image and display it in a view using the images(for:style:limit:) method, by passing an array of ImagePlaygroundConcept objects, the desired ImagePlaygroundStyle and the number of images that needs to be generated.

import SwiftUI
import UIKit
import ImagePlayground

struct ContentView: View {

    @State var generatedImage: CGImage?
    @State var isGenerationStarted: Bool = false
    @State var prompt: String = ""
    
    var body: some View {
        VStack {
            if let image = generatedImage {
                Image(uiImage: UIImage(cgImage: image))
                    .resizable()
                    .frame(width: 200, height: 200)
            } else if isGenerationStarted {
                ProgressView()
            } else {
                ContentUnavailableView {
                    Label("Start creating beautiful images", systemImage: "apple.intelligence")
                } actions: {
                    TextField("Prompt:", text: $prompt)
                    
                    Button("Generate"){
                        isGenerationStarted.toggle()
                        
                        Task {
                            try await generateImage()
                        }
                    }
                    .buttonStyle(BorderedProminentButtonStyle())
                    .padding()
                }
            }
        }
    }
    
    func generateImage() async throws {
        do {
            let imageCreator = try await ImageCreator()
            let style = ImagePlaygroundStyle.animation
            
            let images = imageCreator.images(
                for: [.text("\(prompt)")],
                style: style,
                limit: 1
            )
            
            for try await image in images {
                generatedImage = image.cgImage
            }

        }
        catch ImageCreator.Error.notSupported {
            print("Image creation not supported on the current device.")
        }
    }
    
}

If we want to generate multiple images we need to adjust the limit parameter of the images(for:style:limit:) method.

import SwiftUI
import UIKit
import ImagePlayground


struct ContentView: View {
    @State var generatedImages: [CGImage]?
    @State var isGenerationStarted: Bool = false
    @State var prompt: String = ""
    
    var body: some View {
        VStack(alignment: .center) {
            if let image = generatedImages {
                VStack(){
                    ForEach(image, id: \.self){ selectedImage in
                        Image(uiImage: UIImage(cgImage: selectedImage))
                            .resizable()
                            .frame(width: 200, height: 200)
                    }
                }
            } else if isGenerationStarted {
                ProgressView()
            }
            else {
                ContentUnavailableView {
                    Label("Start creating beautiful images", systemImage: "apple.intelligence")
                } actions: {
                    TextField("Prompt:", text: $prompt)
                    Button("Generate"){
                        isGenerationStarted.toggle()
                        Task {
                            try await generateImage()
                        }
                    }
                    .buttonStyle(BorderedProminentButtonStyle())
                    .padding()
                }
            }
        }
    }
    
    func generateImage() async throws {
        do {
            let imageCreator = try await ImageCreator()
            let generationStyle = ImagePlaygroundStyle.animation
            
            
            let images = imageCreator.images(
                for: [.text("\(prompt)")],
                style: generationStyle,
                limit: 3)
            
            for try await image in images {
                if let generatedImages = generatedImages {
                    self.generatedImages = generatedImages + [image.cgImage]
                }
                else {
                    self.generatedImages = [image.cgImage]
                }
            }

        }
        catch ImageCreator.Error.notSupported {
            print("Image creation not supported on the current device.")
        }
    }
}