Getting started with creative coding using Swift and SwiftUI

For some, building apps that solve a problem is their calling, while others may be looking for a way to create something expressive rather than functional. Creative coding plays a huge role in that area and using code to create artwork is a well established practice that dates back to the early days of computers.

Creative coding defies the conventional boundaries between art, design and technology. It's not just about using digital tools for creative or artistic endeavors, it’s about engaging directly with computation as an expressive medium itself. Creative coders — whether they are artists, designers, architects, musicians, or poets — write their own custom software to create works that display behaviors and qualities that could not be easily achieved through other means. These practitioners design systems, processes, and behaviors that can evolve over time, develop in ways that the creator themselves didn’t predict, follow complex procedural rules where noise and randomness play a fundamental role, or even intentionally fail or produce "errors" or "glitches" as an aesthetic output.

Creative coding, sometimes known also "creative computing", represents a shift in how to approach both artistic expression and software development. Unlike common programming practices focused on solving practical problems, creative coding enables artists, designers, and creatives of all sorts to explore programming as a medium for artistic expression and cultural inquiry. For those without a technical background, creative coding offers an engaging gateway into programming fundamentals, while experienced programmers could discover that engaging with unconventional practices of code can broaden their perspectives and spark new creative possibilities.

Well known and established practitioners in the field like Lauren McCarthy, Mimi Onuoha, Zach Lieberman and Allison Parrish, position creative coding as a critical tool for examining and reimagining the role of technology in society and envisioning alternative futures for technological development. Just think about an organization as The School for Poetic Computation, co-founded by Lieberman, that starting from its name exemplifies such an approach by fostering an environment where code, art, and critical thinking intersect in unexpected ways.

Creative agencies and designers like Universal Everything or Tim Rodenbröker, to name a few, demonstrate the potential of creative computation in the context of commissioned work both for commercial and non-profit organizations, producing digital experiences that blur the boundaries between art, science and design, that wouldn’t have been possible without such a creative approach to coding and computation.

Artists like Golan Levin and Tega Brain push the boundaries of the field through their work and their educational efforts, as their book Code as Creative Medium. A Handbook for Computational Art and Design (MIT press) redefines the approach to learning coding through expressive or critical explorations.

Through these diverse approaches, practitioners from all around the world explore themes of procedure, connection, abstraction, and authorship, while raising important questions about the nature of digital technologies and their societal and political impact.

A bit of background

First glimpses of creative computation started appearing in the 1960s within research centers and laboratories. One of these early sparks happened at Bell Labs in 1966 when Leon Harmon and Ken Knowlton created "Studies in Perception #1", a portrait of the choreographer Deborah Hay made with ASCII symbols that made its way to the Museum of Modern Art's exhibition "The Machine as Seen at the End of the Mechanical Age".

In those same years, the development of the first computer-numeric controlled (CNC) milling machine in the 1950s, originally intended for military applications, found an unexpected artistic use with Charles "Chuck" Csuri that, around 1968, repurposed that technology to produce the abstract Numeric Milling sculpture series.

The late 1960s and early 1970s saw the emergence of pioneering artists who would define the field's fundamental approaches. Manfred Mohr transitioned from action painting and jazz to become a pioneer of algorithmic art in 1969. Working on a room-sized computer at the meteorological institute, Mohr programmed in FORTRAN IV, outputting his work - a mix of simple geometrical shapes altered through noise and randomness - through a Benson flatbed printer.

In Germany, artists like Frieder Nake and Georg Nees were exploring similar territory, pushing the boundaries of what computers could achieve aesthetically. Vera Molnár, created groundbreaking plotter drawings in 1974 that demonstrated the aesthetic possibilities of combining procedural iteration with randomized elements, establishing many of the principles that would become central to what then would be known as “generative art”.

The field continued to evolve through the following decades, with artists and programmers all around the world exploring new possibilities as computing technology advanced. Three decades later, a democratization of creative coding exploded also thanks to John Maeda's work at the MIT Media Lab laying crucial groundwork, leading to a pivotal moment in 2001 when Casey Reas and Ben Fry created Processing at MIT's Aesthetics + Computation Group.

This simplified Java-based graphics library and integrated development environment (IDE) built for the electronic arts not only pioneered a more intuitive approach to coding but also catalyzed a new generation of computational artists. Casey Reas himself produced influential works exploring emergent and agent based systems to generate intricate visual abstract forms, while other artists like Golan Levin started creating groundbreaking interactive performances merging code and gesture, or Camille Utterback who developed innovative art installations responding to human movement.

In the early 2010s, the web-based successor of Processing, p5.js, further expanded access to creative coding, fostering a diverse worldwide community of practitioners who share techniques and inspiration. Developed as a native JavaScript alternative by Lauren McCarthy, p5.js prioritized access and diversity in learning computer programming, making creative coding even more approachable. The democratization enabled by p5.js has fundamentally transformed how artists and designers approach computational creativity, leading to a rich ecosystem where code serves both as a tool and a medium for artistic expression.

Swift as a Creative Medium: Untapped Potential

Swift offers compelling possibilities for creative coding within Apple's ecosystem. The language's modern and natural syntax, strong performance, and deep integration with powerful native frameworks like Metal, SpriteKit and Core Graphics make it a capable tool for artistic expression through code.

The Swift developer community and the ecosystem of iOS apps, is full of great examples of creative software development. Just think about Andy Works' "Not Boring" suite of playful productivity apps that reimagine everyday tools and that is inspired by the work of conceptual artist John Baldessarri, or the mind-bending BlackBox game by Ryan McLeod, a sort of playable and experimental piece of art that push the boundaries of iOS as a creative platform, just to name a few well-known, award-winning creative developers.

However, even if full of extremely creative projects and practitioners, Swift remained somewhat peripheral to the broader creative coding “movement” and discourse. While languages and frameworks like Processing, p5.js, openFrameworks, and TouchDesigner have developed rich ecosystems specifically oriented toward artistic practice, Swift's creative coding community is still emerging. Just think about the very well-known Awesome Creative Coding GitHub repository, where Swift is barely mentioned among the extensive collection of tools and frameworks.

Swift's potential for creative coding shouldn't be underestimated. The language's clarity and approachability can provide an accessible entry point for artists and designers working within Apple's platforms, while its performance characteristics enable sophisticated real-time graphics and interactions. What's somehow missing is a larger and dedicated community - the rich network of knowledge sharing, collections of tutorials, examples, libraries, and shared resources that have made other platforms the natural place for creative coders or, more in general, non-programmers attracted by the artistic and expressive possibilities of code.

This gap presents both a challenge and an opportunity for Swift designers and developers interested in pushing the boundaries of creative expression through code. While they may need to look beyond Swift's immediate ecosystem for inspiration and techniques, they have the chance to help build a distinctive creative coding culture around Swift's unique strengths.

Creative Coding with Swift

However, the creative coding community's influence on Swift development has led to the emergence of thoughtful tools that bridge traditional creative coding approaches with Apple's ecosystem.

For those already in love with Processing or p5.js, the Swift community got them covered and created some Swift Packages that allow a person to take previous experience with Processing to Swift and create expressive sketches that run on all major platforms, such as iOS, macOS and even visionOS.

Among these, Swift Processing, created by Jonathan Kaufman, stands as a direct connection to Processing's legacy. Built with UIKit, it maintains Processing's commitment to ease of access, enabling artists, designers, and students to create expressive iOS experiences while preserving the familiar syntax that made Processing so popular. This continuity is particularly significant as it allows creative coders to transfer their existing knowledge to Apple's platforms while exploring new possibilities unique to iOS and macOS.

The ecosystem has further evolved with SwiftyCreatives, developed by Yuki Kuwashima, which takes a different approach by embracing SwiftUI and Metal for high-performance rendering. By incorporating a 3D canvas and leveraging Apple's modern frameworks, SwiftyCreatives demonstrates how Swift can provide fertile ground for creative coding experimentation. Its cross-platform support, extending even to visionOS, hints at the potential for creative coding to extend within the ever-evolving Apple ecosystem.

These frameworks are complemented by educational initiatives that echo Processing's original mission of making creative technology accessible to diverse audiences. Notable among these is Prof. Masood Kamandy's comprehensive course at the University of California Santa Barbara, which approaches creative coding with Swift, and Joel Gethin Lewis that run courses on creative computing including Swift at the Creative Computing Institute at the University of the Arts London.

What makes these examples particularly significant is how they're helping to establish a creative coding culture within the Swift community. While maintaining connections to the broader creative coding community through familiar paradigms and approaches, they leverage Swift's distinctive features and Apple's platforms to open new creative possibilities. This evolution suggests that Swift's role in creative coding, while still emerging, has the potential to contribute unique perspectives to the field's ongoing development.

A playground for creative coding: SwiftUI and the Canvas view

However, why make things more complicated than they are? While specialized frameworks and packages offer dedicated paths to creative coding, there's a wide space for exploring creative possibilities within “vanilla” SwiftUI and its native capabilities.

TimelineView(.animation) { timeline in
    Canvas { context, size in
        // Set background once for all layers
        context.fill(
            Path(CGRect(origin: .zero, size: size)),
            with: .color(LayeredNoiseCircle.colors.background)
        )
        
        // Draw layers from back to front
        for layer in 0..<LayeredNoiseCircle.numberOfLayers {
            drawLayer(context: context, size: size, layer: layer)
        }
        
        time += LayeredNoiseCircle.animationSpeed
    }
    .ignoresSafeArea()
}

Take the Canvas view for example: originally designed for custom graphics in user interfaces, reveals itself as an unexpected but powerful medium for creative computation. SwiftUI's Canvas view, with its support for immediate mode drawing, creates an environment remarkably similar to the concept of code "sketches" that made Processing and p5.js revolutionary in creative coding. It provides a space where artists and developers can quickly experiment with and visualize creative concepts, maintaining the spirit of rapid iteration and on-the-fly adjustments.

As a practical example, consider a creative sketch made in SwiftUI and inspired by some of the pioneers of computer art. Consider a grid of simple geometric shapes that echoes the systematic yet organic qualities found in Vera Molnár's plotter drawings or Manfred Mohr's early experiments with algorithmic art. Using SwiftUI's Canvas, we can create dynamic compositions using simple geometric shapes, turning their visual properties into parameters and continuously and procedurally altering them with controlled randomness.

In our example, a grid of simple circles evolves and transforms over time. Each circle responds to Perlin noise — a type of gradient noise that creates natural-looking randomness — affecting properties like size, rotation, thickness, color, shape-distortion, etc. This approach creates organic movement that feels both systematic and unpredictable, much like Molnár's exploration of patterns arising from the interaction between procedural iteration and randomizations.

class PerlinNoise {
    // Main noise function that generates a value between 0 and 1
    func noise(x: Double, y: Double) -> Double {
        // Hash coordinates to get grid cell coordinates
        let X = Int(floor(x)) & 255
        let Y = Int(floor(y)) & 255
        
        // Get decimal part of coordinates
        let xf = x - floor(x)
        let yf = y - floor(y)
        
        // Calculate fade curves
        let u = fade(xf)
        let v = fade(yf)
        
        // Generate hash values for corners of the cell
        let A = (X + Y) * 16807 % 256
        let AB = (X + Y + 1) * 16807 % 256
        let B = (X + 1 + Y) * 16807 % 256
        let BB = (X + 1 + Y + 1) * 16807 % 256
        
        // Interpolate between corner values
        return lerp(v,
                   lerp(u, grad(A, xf, yf), grad(B, xf-1, yf)),
                   lerp(u, grad(AB, xf, yf-1), grad(BB, xf-1, yf-1)))
    }
    
    // Fade function to smooth transitions
    private func fade(_ t: Double) -> Double {
        return t * t * t * (t * (t * 6 - 15) + 10)  // Smoothstep function
    }
    
    // Linear interpolation between two values
    private func lerp(_ t: Double, _ a: Double, _ b: Double) -> Double {
        return a + t * (b - a)
    }
    
    // Generate gradient value for a given hash
    private func grad(_ hash: Int, _ x: Double, _ y: Double) -> Double {
        let h = hash & 15
        let grad = 1.0 + Double(h & 7)
        let u = h < 8 ? x : y
        let v = h < 4 ? y : x
        return ((h & 1) != 0 ? -u : u) + ((h & 2) != 0 ? -2.0 * v : 2.0 * v)
    }
} 

The code demonstrates how we can control various aspects of the visualization—from the grid's overall layout to individual circle properties like stroke width, fill-opacity, and distortion amount. This granular control, combined with SwiftUI's efficient rendering, makes it possible to create complex, visually rich, animated patterns that run smoothly. And, in the pure tradition of generative art, what is exciting is that the outcome is not predictable by the author, it evolves and emerges autonomously, showing surprising and unexpected results.

Conclusion

The example provided with SwiftUI and Canvas demonstrates how creative coding can be naturally accessible to developers who are already familiar with Swift and SwiftUI. Rather than learning a new framework, they can extend their existing knowledge into creative territory, experimenting with algorithmic or generative art using tools they already understand. At the same time, creative practitioners coming from environments like p5.js or OpenFrameworks might find in Swift and SwiftUI a fresh perspective on their practice, one that combines the freedom of creative coding with the possibilities of mobile development and Apple ecosystem, its hardware features, and emerging territories like visionOS.

Swift, with its combination of access to devices’ native capabilities and specialized frameworks, offers multiple entry points for different types of creators. Whether through Processing-inspired tools like Swift Processing and SwiftyCreatives, or through direct engagement with SwiftUI's Canvas, the barriers to entry continue to lower. This multiplicity of approaches ensures that everyone—from experienced developers to artists without a coding background, from students to professional designers—can find their path into creative coding.

Whether it's reimagining productivity apps through an artistic lens, creating generative art that lives within the device and responds to sensors and other data input, or developing new forms of digital experience for emerging platforms, Swift provides an exciting territory for a new wave of creative coders and reminds us that code itself is a medium of expression, one that becomes richer and more vibrant as more diverse voices and perspectives join the conversation.