Gradient Meshes in SceneKit

These days, you see gradient meshes in a lot of places from the Instagram app icon to the Framer website and find implementations for Sketch, Figma and of course Illustrator.

Conceptually, gradient meshes generalize linear gradients by interpolating between colors that can be placed along two dimensions on a plane.

But we don't want to limit ourselves to displaying pre-exported assets. To make your apps truly responsive and to dynamically react to changes in color and user preference, we need to be able to generate these gradient meshes on the fly.

So let's throw our hat in the ring and take a stab at implementing them ourselves.

¶ Humble Beginnings

Let's start with the humble square and assign one color to each corner. With SceneKit, we can use SCNGeometry­Source and it will interpolate colors for a given output pixel.

Because the rectangle is made out of two triangles, we need to specify the corner points and color they have in common twice.

That's a pretty nice effect for very little work! We can generalize this to more than four colors by combining multiple patches into a larger surface.

First, we define a Control­Point struct that keeps track of the color values.

struct ControlPoint {
    var color: simd_float3
}

Next we create a Grid of multiple control points. We're using a straightforward two-dimensional array to store them and assign individual colors. You can find the specifics in the playground.

var grid = Grid(
    repeating: ControlPoint(),
    width: 3,
    height: 3
)


grid[x, y].color = myFavoriteColor

Now we can generate vertices by looking at the four control points that make up the corners of a patch. We derive the x and y coordinates implicitly from their position on the grid using linear interpolation (affectionately called lerp) from -1 to 1.

For the colors of the vertices, we directly copy over the color vector from the control point. The resulting gradient is even more colorful.

This works, but our simple approach is starting to show its limits — there's pronounced banding along some of the vertices.

Because it is formed of triangles, only three colors ever contribute to an output pixel and whatever is between them is simply interpolated linearly.

But let's not give up yet. Maybe once we found a way to deform the mesh, we can work around this problem through careful mis art-direction?

¶ Bend it like Bézier

You're likely familiar with a similar problem of naive interpolation looking very artificial: animations. By varying the speed at which you step through an animation, you can turn an abrupt change in velocity into a smooth acceleration, resulting in a more pleasant transition.

Turns out, we can use a similar approach for our gradient — by deforming the surface on which we distribute our colors, we can vary the distance between colors and create more interesting shapes and blends.

Another tool that you likely already have in your toolbelt from working with animations or graphics are Cubic Bézier Curves, available in Core Animation as CAMedia­Timing­Function and as one of the possible segments of a CGPath.

If we take four Cubic Bézier Curves and arrange them to form a square, the resulting shape is what's called a Cubic Bézier Patch.

Much like the two-dimensional Bézier Curve is described by its four control points, the three-dimensional Bézier Patch is described by a set of sixteen control points (That's 4².)

Another representation of such a parametric surface is the closely related Bicubic Hermite Patch. For our gradient mesh, we will use this representation but keep in mind that this is just a different way to describe the same surface and you can freely convert between Hermite and Bézier forms.

A Bicubic Hermite Patch is defined by its boundary conditions, that is its four corner points, a pair of tangent vectors along either edge for each of the corner points as well as corner twist vectors. That's sixteen data points per dimension, the same as for Bézier patches.

A point on the surface of this patch is defined as

where

• u and v are the unit coordinates along the respective edges.

• U and V are vectors of the 3rd to 0th powers of u and v.

• B is the boundary contition matrix for the current dimension, we'll need one for the X and Y coordinates of our surface.

• And finally, H is the Hermite matrix. This one is constant.

This may sound scarier than it is. The Accelerate framework handles the vector and matrix math for us, all we need to do is plug in the right values.

In Swift, it ends up looking like this:

let H = simd_float4x4(rows: [
    simd_float4( 2, -2,  1,  1),
    simd_float4(-3,  3, -2, -1),
    simd_float4( 0,  0,  1,  0),
    simd_float4( 1,  0,  0,  0)
])


let H_T = H.transpose


func surfacePoint(
    u: Float,
    v: Float,
    X: simd_float4x4,
    Y: simd_float4x4
) -> simd_float2 {
    let U = simd_float4(u * u * u, u * u, u, 1)
    let V = simd_float4(v * v * v, v * v, v, 1)


    return simd_float2(
        dot(V, U * H * X * H_T),
        dot(V, U * H * Y * H_T)
    )
}

To call this function, we will need to assemble the coefficient matrices X and Y from our control points.

First, we add location, u­Tangent and v­Tangent properties to Control­Point, setting aside the corner twist vectors for this article.

struct ControlPoint {
    var color: simd_float3 = simd_float3(0, 0, 0)


    var location: simd_float2 = simd_float2(0, 0)
    var uTangent: simd_float2 = simd_float2(0, 0)
    var vTangent: simd_float2 = simd_float2(0, 0)
}

Filling out the coefficient matrix is now just a matter of putting the right control point data in the right spot. Swift's Key­Path and a few helper functions make this a breeze:

func meshCoefficients(
    _ p00: ControlPoint,
    _ p01: ControlPoint,
    _ p10: ControlPoint,
    _ p11: ControlPoint,
    axis: KeyPath<simd_float2, Float>
) -> simd_float4x4 {
    func l(_ controlPoint: ControlPoint) -> Float {
        controlPoint.location[keyPath: axis]
    }


    func u(_ controlPoint: ControlPoint) -> Float {
        controlPoint.uTangent[keyPath: axis]
    }


    func v(_ controlPoint: ControlPoint) -> Float {
        controlPoint.vTangent[keyPath: axis]
    }


    return simd_float4x4(rows: [
        simd_float4(l(p00), l(p01), v(p00), v(p01)),
        simd_float4(l(p10), l(p11), v(p10), v(p11)),
        simd_float4(u(p00), u(p01),      0,      0),
        simd_float4(u(p10), u(p11),      0,      0)
    ])
}

We can now walk the grid of control points along the x and y axis to obtain the four neighboring control points of a patch. Control points in hand, we can then obtain the coefficient matrices which in turn allow us to resolve an arbitray point on the surface.

let p00 = grid[    x,     y]
let p01 = grid[    x, y + 1]
let p10 = grid[x + 1,     y]
let p11 = grid[x + 1, y + 1]


let X = meshCoefficients(p00, p01, p10, p11, axis: \.x)
let Y = meshCoefficients(p00, p01, p10, p11, axis: \.y)

¶ Splitting the difference

So far, we've only dealt with low-polygon meshes completely defined by our control points. Now that we can derive any point within a patch, we can subdivide it arbitrarily to form a smooth surface.

Once we have determined a suitable subdivision factor, performing the actual subdivision is just a matter of looping over the current patch in fractional steps, then updating our output points according to the patch coordinate as well as the fractional one.

We'll also store these points in their own Grid and generate the SCNNode from that separately. You can find the specifics in the playground.

let subdivisions = 15


for u in 0 ..< subdivisions {
    for v in 0 ..< subdivisions {
        points[x * subdivisions + u, y * subdivisions + v] =
            surfacePoint(
                u: Float(u) / Float(subdivisions - 1),
                v: Float(v) / Float(subdivisions - 1),
                X: X,
                Y: Y
            )
    }
}

We still haven't assigned any colors to our vertices though. While we could bilinearly interpolate between the four control point colors, that would still produce banding in areas where the mesh remains mostly undistorted, which is something we set out to avoid initially.

But since we already went through the trouble of setting up a parametric surface in the coordinate space of our mesh, let's try what happens when we expand this concept to its color space as well.

To do this, we treat each color channel as its own dimension and build up a new coefficient matrix for each. We will leave the tangent and corner twist vectors at zero, which results in a perceptually pleasing ease-in-ease-out look where bilinear interpolation might produce discontinuities, for example when going from black to white to black.

func colorCoefficients(
    _ p00: ControlPoint,
    _ p01: ControlPoint,
    _ p10: ControlPoint,
    _ p11: ControlPoint,
    axis: KeyPath<simd_float3, Float>
) -> simd_float4x4 {
    func l(_ point: ControlPoint) -> Float {
        point.color[keyPath: axis]
    }


    return simd_float4x4(rows: [
        simd_float4(l(p00), l(p01), 0, 0),
        simd_float4(l(p10), l(p11), 0, 0),
        simd_float4(     0,      0, 0, 0),
        simd_float4(     0,      0, 0, 0)
    ])
}

Working out a color point in RGB space looks very similar to surface­Point, too:

func colorPoint(
    u: Float,
    v: Float,
    R: simd_float4x4,
    G: simd_float4x4,
    B: simd_float4x4
) -> simd_float3 {
    let U = simd_float4(u * u * u, u * u, u, 1)
    let V = simd_float4(v * v * v, v * v, v, 1)


    return simd_float3(
        dot(V, U * H * R * H_T),
        dot(V, U * H * G * H_T),
        dot(V, U * H * B * H_T)
    )
}

Now we derive a color value much like we did for a coordinate point:

let R = colorCoefficients(p00, p01, p10, p11, axis: \.x)
let G = colorCoefficients(p00, p01, p10, p11, axis: \.y)
let B = colorCoefficients(p00, p01, p10, p11, axis: \.z)


for u in 0 ..< subdivisions {
    for v in 0 ..< subdivisions {
    points[x * subdivisions + u, y * subdivisions + v] =
        colorPoint(
            u: Float(u) / Float(subdivisions - 1),
            v: Float(v) / Float(subdivisions - 1),
            R: R,
            G: G,
            B: B
        )
    }
}

Now we have a functional gradient mesh. With up to 4 coordinate pairs per control point and 3 color components, there's a huge number of dials to tweak to achieve all kinds of effects.

¶ We've barely scratched the surface

Of course, there's a lot more to do to make a reusable SwiftUI component for your app but we hope you feel encouraged to play around with gradient meshes yourself.

There's plenty more to talk about in the future: How to build non-rectangular meshes, how to animate them or how to generate them randomly while maintaining some guarantees for the overall aesthetic. You may have also noticed that our interpolation scheme used the RGB colorspace – maybe in the future we can investigate perceptual colorspaces such as CIELAB or Oklab and how they affect the colors within our gradient.