Sign in Subscribe
Bringing Image Playground to your app
Apple Intelligence

Bringing Image Playground to your app

Learn how to make your app able to generate images using Apple Intelligence models within Image Playgrounds.

Antonella Giugliano

• 8 min read

Besides being a standalone app, Image Playground is also exposed to developers as a framework enabling the integration of image generation into apps, embedding components that invoke and present the Image Playground interface, available both in UIKit and SwiftUI.

To dive deep into the topic, we are going to implement an app generating story covers out of prompted stories.

1. Getting Started

Handling feature availability in SwiftUI

To take advantage of Image Playground, whether you are developing with UIKit or SwiftUI, import the Image Playground framework.

import ImagePlayground

Second, be sure to set the minimum deployment to 18.1 for iOS and 15.1 for macOS to avoid Xcode errors and be able to build the app only on supported operating systems.

Alternatively, set the attributes to ensure the availability of Image Playgrounds in the operational system before using it.

Handle the entry point of the app according to the OS of the device.

import SwiftUI

@main
struct StoryCoverGenerator: App {
    var body: some Scene {
        WindowGroup {
            // 1. The available version
            if #available(iOS 18.1, macOS 15.1, *) {
                ContentView()
            // 2. Fallback on earlier versions
            } else {
                Text("Operating System not supported")
            }
        }
    }
}
  1. Check that the OS version is no lower than iOS 18.1 or macOS 15.1.
  2. Provide a fallback view for devices with minor versions.

Do the same for the ContentView, but now regarding the availability of the Image Playground feature.

import SwiftUI
import ImagePlayground

// 1. Define the OS availability
@available(iOS 18.1, macOS 15.1, *)
struct ContentView: View {
    
    // 2. Check the availability of the feature
    @Environment(\.supportsImagePlayground) private var supportsImagePlayground
    
    var body: some View {
        
        if supportsImagePlayground {
            // Shows the UI to use the feature
        } else {
            Text("This device does not support Image Generation features.")
        }
    }
}
  1. Add the @available attribute to specify to the compiler the OS availability of this struct.
  2. Check the dedicated environment variable to handle the display of the features according to the device type to avoid the component enabling the feature to be visible on those devices with the right OS but not supporting Apple Intelligence.

Handling feature availability in UIKit

Let’s explore how to take advantage of the attributes to handle the os version and feature availability when implementing in UIKit.

First, create a UIViewController to handle the view when the feature is unavailable.

import UIKit

class UnavailableViewController: UIViewController {
	// Create the UI as you see fit
}

Second, create a UIViewController responsible for the story cover generation and Set the os availability attributes.

import UIKit
import ImagePlayground

// The os availability attributes
@available(iOS 18.1,macOS 15.1, *)

// The view controller responsible for the Story Cover Generation
class StoryCoverGeneratorViewController: UIViewController {
	...
}

Now, we are ready to handle the scene to present it according to the availability of the feature.

import UIKit
import ImagePlayground

class SceneDelegate: UIResponder, UIWindowSceneDelegate {
    
    func scene(_ scene: UIScene, willConnectTo session: UISceneSession, options connectionOptions: UIScene.ConnectionOptions) {
        
        guard let _ = (scene as? UIWindowScene) else { return }
        // 1. The view controller
        let vc: UIViewController
        
        // 2. Check the os version and the feature availability
        if #available(iOS 18.1, macOS 15.1, *), ImagePlaygroundViewController.isAvailable {
            // a. ViewController for the Story Cover Generator
            vc = StoryCoverGeneratorViewController()
        } else {
            // b. ViewController if unavailable
            vc = UnavailableViewController()
        }
        
        let navigationController = UINavigationController(rootViewController: vc)
        self.window?.rootViewController = navigationController
        self.window?.makeKeyAndVisible()
    }
    
}
  1. Declare a variable where to store the view controller.
  2. Use the attribute to check the os version and the availability of the feature using ImagePlaygroundViewController.isAvailable .
  3. Initialize them according to whether the feature is available or not:
    1. assign an instance of the StoryCoverGeneratorViewController if Image Playground is available;
    2. assign an instance of the UnavailableViewController if it is not.

2. Integrating Image Playground

Integration in a SwiftUI app

The method imagePlaygroundSheet(isPresented:concept:sourceImageURL:onCompletion:onCancellation:) allows the presentation of the sheet where the user creates images from specified inputs.

It takes several parameters:

  1. isPresented is expecting a Bool value that enables the sheet presentation, resulting in the opening of the Image Playground interface;
  2. concept , a String that describes the content to generate the image accordingly;
  3. sourceImage is expecting a file URL depending on whether the prompt has to include an initial visual reference or not. Once it is open, it can be overridden directly from the Image Playground interface. 
    imagePlaygroundSheet(isPresented:concept:sourceImage:onCompletion:onCancellation:) method allows to do the same using an image instead of a URL.
  4. onCompletion , a closure with no return value that receives the generated image with these parameters:
    1. url , a URL pointing to the image file, temporarily stored
    2. onCancellation, a closure to handle the cancellation of the image creation process when the user exits the creation interface without selecting an image. Once this closure is executed, the system automatically dismisses the sheet.
import SwiftUI
import ImagePlayground

@available(iOS 18.1,macOS 15.1, *)
struct ContentView: View {
    // Enabling the sheet
    @State var isPresented: Bool = false
    
    var body: some View {
        
        VStack {
            // Application UI
        }
        
        // ImagePlayground Sheet Presenter
        .imagePlaygroundSheet(
            isPresented: $isPresented,
            concept: "a red cat with blue gloves"
        ) { url in
            // Use the URL
        }
    }
}

Another method that allows the description of the image with more concepts is imagePlaygroundSheet(isPresented:concepts:sourceImageURL:onCompletion:onCancellation:)which requires a [ImagePlaygroundConcept] object, a collection of text elements that describe the content to generate the image accordingly.

.imagePlaygroundSheet(
    isPresented: $isPresented,
    
    // Image playground concepts
    concepts: [
        ImagePlaygroundConcept.text("red cat with blue gloves"),
        ImagePlaygroundConcept.text("big park with a pink river")
    ]) { url in
        // Use the URL
    }

The method text(_:) creates image playground concepts out of short strings that describe the content of the image the user wants to generate. Keep in mind that:

  1. It works better with single words or short sentences;
  2. When the string is too long, according to the length the model supports, it automatically splits it into shorter concepts, selecting only the most important ones.

When it comes to longer strings, the method to use is extracted(from:title:) which is able to select multiple concepts.

// ImagePlayground Sheet Presenter
.imagePlaygroundSheet(
    isPresented: $isPresented,
    
    // Image playground concepts
    concepts: [
        ImagePlaygroundConcept.extracted(
            from: "A red cat, named Rusty, strutted through the park, with his blue gloves catching everyone’s attention. He waved at a squirrel juggling acorns and joined a duck painting by the pond. Spotting a kite tangled in a tree, Rusty climbed up, saved the kite, and became the park’s unexpected hero.",
            title: "Rusty, the blue gloves cat")
    ]) { url in
        // Use the URL
    }

This method returns an ImagePlaygroundConcept object and takes 2 parameters:

  1. The text, a long String object from which the image playground concepts are selected - if its length is not the minimum required by the model, the string could be used as it is;
  2. Optionally a title, a short String representing the text in a concise way, which helps the model to extract the salient concepts from it.

Now let’s integrate everything in our view to generate possible covers based on the story prompted by the user.

import SwiftUI
import ImagePlayground

@available(iOS 18.1, macOS 15.1, *)
struct ContentView: View {
    // Enabling the sheet
    @State var isPresented: Bool = false
    
    var storyPlaceholder: String = "Write your story here..."
    
    // URL of image created by AI
    @State var imageURL: URL?
    
    // Story prompt
    @State var story: String = ""
    
    // Determine the availability of the feature.
    @Environment(\.supportsImagePlayground) private var supportsImagePlayground
    
    var body: some View {
        
        if supportsImagePlayground {
            ScrollView {
                VStack(alignment: .center) {
                    ZStack {
                        // Image loaded from Image Playground generation
                        if let url = imageURL {
                            AsyncImage(url: url) { image in
                                image.resizable()
                                    .scaledToFill()
                                    .frame(width: 200, height: 300)
                                    .clipped()
                                    .cornerRadius(8)
                            } placeholder: {
                                ProgressView()
                            }
                        } else {
                            ZStack(alignment: .center){
                                RoundedRectangle(cornerRadius: 8)
                                    .fill(Color.red)
                                    .frame(width: 200, height: 300)
                            }
                            
                            if imageURL == nil {
                                Text("""
                                    Story
                                    Cover
                                    Generator 
                                """)
                                .font(.title)
                                .bold()
                            }
                        }
                    }
                    .shadow(color: .black.opacity(0.3), radius: 8, x: 4, y: 4)
                    .padding()
                    
                    VStack(alignment: .leading) {
                        Text("Story:")
                            .font(.title3)
                            .bold()
                            .padding(.bottom, 5)
                        Text("Describe your story in no more than 250 words")
                            .font(.subheadline)
                            .bold()
                            .foregroundStyle(.secondary)
                            .padding(.bottom)
                        TextField(storyPlaceholder, text: $story, axis: .vertical)
                            .lineLimit(15)
                            .padding(5)
                            .background(Color.clear)
                            .overlay(
                                RoundedRectangle(cornerRadius: 12)
                                    .stroke(Color.red.opacity(0.5), lineWidth: 1)
                            )
                    }
                    .padding()
                }
                .padding()
                
                Button {
                    isPresented = self.checkStoryLength(story: story)
                } label: {
                    Text("Generate")
                        .foregroundStyle(.red)
                }
                
                // ImagePlayground Sheet Presenter
                .imagePlaygroundSheet(
                    isPresented: $isPresented,
                    concepts: [ImagePlaygroundConcept.extracted(from: story, title: nil)]) { url in
                    imageURL = url
                }
                .padding()
                
            }
        } else {
            Text("This device does not support Image Generation feature.")
                .foregroundStyle(.red)
        }
    }
    
    // Check the lenght of Story
    private func checkStoryLength(story: String) -> Bool {
        if story.components(separatedBy: " ").count >= 250 {
            return false
        }
        
        return true
    }
}

In this app, users can input a story up to 250 words long into a text field and generate an image corresponding to the story.

When the user taps on the Generate button, it validates the story's length and then presents the Image Playground Interface. A conditional environment check ensures the feature is available on supported devices, displaying an error message otherwise.

0:00
/0:26

Integration in a UIKit app

To present the Image Playground Interface with UIKit, use the ImagePlaygroundViewController. Extend the StoryCoverGeneratorViewController class with a new method responsible for the Image Playground Sheet presentation.

extension StoryCoverGeneratorViewController {
    
    @IBAction
    private func openImagePlayground(with story: String) {
        // 1. Initialize the playground
        let playground = ImagePlaygroundViewController()
        
        // 2. Delegation
        playground.delegate = self
        
        // 2. Set extracted concepts from the story in the playground
        playground.concepts = [.extracted(from: story, title: nil)]
        
        // 3. Present the ImagePlaygroundViewController
        present(playground, animated: true, completion: nil)
    }
    
}

The openImagePlayground(with:) method receives the story as a String value works as follows:

  1. Initializes the ImagePlaygroundViewController;
  2. Sets the delegate property to capture the lifecycle events;
  3. Extracts the concepts from the parameter using the extracted(from:title:) method, and stores them in the conceptsproperty of the ImagePlaygroundViewController;
  4. Presents the ImagePlaygroundViewController.

Extend the StoryCoverGeneratorViewController class to conform to the ImagePlaygroundViewController.Delegate protocol.

// Conforming to ImagePlaygroundViewController.Delegate
extension StoryCoverGeneratorViewController: ImagePlaygroundViewController.Delegate {
    
    // 1. The delegate stub returning the generated image to the delegate
    func imagePlaygroundViewController(_ imagePlaygroundViewController: ImagePlaygroundViewController, didCreateImageAt imageURL: URL) {
        // 2. Set the image 
        if let image = UIImage(contentsOfFile: imageURL.path) {
            imageView.image = image
        } else {
            print("Error loading image from URL: \(imageURL)")
        }
        
        // 3. Dismiss the sheet
        dismiss(animated: true, completion: nil)
    }
    
}
  1. Implement the imagePlaygroundViewController(_:didCreateImageAt:) method, returning the URL of the generated image to the delegate;
  2. Set the image to be displayed on the UI;
  3. Dismiss the Image Playground sheet.

The integration in our Story Cover Generator app will look like this:

import UIKit
import ImagePlayground

class StoryCoverGeneratorViewController: UIViewController {
    
    // Image Generated View that will display the generated image
    private lazy var imageView: UIImageView = { ... }()
    
    // Cover Label
    private let coverLabel: UILabel = { ... }()
    
    // Headline
    private let storyLabel: UILabel = { ... }()
    
    // Subheadline
    private let instructionLabel: UILabel = { ... }()
    
    // TextField
    private let storyTextField: UITextField = { ... }()
    
    // Generate Button
    private let generateButton: UIButton = { ... }()
    
    // Lifecycle
    override func viewDidLoad() {
        super.viewDidLoad()
        setupUI()
    }
    
    // Setting up the UI: add the subviews and the layout constraints
    private func setupUI() { ... }
    
    // 1. Method triggered on the button tap
    @objc private func generateButtonTapped() {
        // 2. Check the length of story input
        if let storyText = storyTextField.text, storyText.components(separatedBy: " ").count <= 250 {
            // 3. Present the Image Playground interface
            openImagePlayground(with: storyText)
        } else {
            // 4. Present Error Alert
            let alert = UIAlertController(title: "Invalid Input", message: "Please limit your story to 250 words.", preferredStyle: .alert)
            alert.addAction(UIAlertAction(title: "OK", style: .default, handler: nil))
            present(alert, animated: true, completion: nil)
        }
    }
}

extension StoryCoverGeneratorViewController: ImagePlaygroundViewController.Delegate {
    
    // Initialize and present the Image Playground interface.
    @IBAction
    private func openImagePlayground(with story: String) {
        // Initialize the playground, get set up to be notified of lifecycle events.
        let playground = ImagePlaygroundViewController()
        playground.delegate = self
        // Set extracted concepts from the story in the playground
        playground.concepts = [.extracted(from: story, title: nil)]
        present(playground, animated: true, completion: nil)
    }
    
    // The delegate stub returning the generated image to the delegate
    func imagePlaygroundViewController(_ imagePlaygroundViewController: ImagePlaygroundViewController, didCreateImageAt imageURL: URL) {
        // Add the image to the image view
        if let image = UIImage(contentsOfFile: imageURL.path) {
            imageView.image = image
        } else {
            print("Error loading image from URL: \(imageURL)")
        }
        
        dismiss(animated: true, completion: nil)
    }
}
  1. Create the method that will be triggered when tapping on the “Generate” button.
  2. Check the length of the story input.
  3. Present the Image Playground interface with the method openImagePlayground(with story:).
  4. Present an error alert when the story is longer than expected.
import UIKit
import ImagePlayground

class StoryCoverGeneratorViewController: UIViewController {
    
    // Properties of the view controller
    
    // Generate Button
    private let generateButton: UIButton = {
        
        // Code that creates the button
        
        // 1. Add the action to the target
        button.addTarget(self, action: #selector(generateButtonTapped), for: .touchUpInside)
        return button
    }()
    
    // Methods of the view controller
}

Add this method as the action to the target of the Generate button.

Integrating Apple’s Image Playground with SwiftUI or UIKit opens new possibilities for creating dynamic visuals, interactive content, and advanced design tools. Developers can easily craft visually rich experiences while maintaining seamless functionality across app frameworks.

There are limitations to what can be achieved at the moment, while Apple rolls out its Apple Intelligence powered features throughout the year. What is promising is to think that even though these features are limited at the moment they will only improve from now on.

If your apps have use cases that could benefit from image generation capabilities it makes sense to start preparing for a future in which the quality of the results created by these tools will be much higher.

Sign up for more like this.

Enter your email
Subscribe