Cyril Mottier

“It’s the little details that are vital. Little things make big things happen.” – John Wooden

Fun With Cutouts

Note: I started writing this article based on Android P preview 1. Android P preview 2, released at Google I/O 2018, actually removed some (most?) of the APIs used extensively in this blog post. Even though none of the UI tricks described below are possible with Android P preview 2, I thought it was still interesting to share them. Indeed, both the technical and design approaches may still be cool and reusable. Let’s hope this article will motivate Google engineers to bring back the required APIs once they will see what kind of interesting stuff they were enabling with Android P preview 1…

One of the new feature in Android P is the support for display cutouts. A display cutout (a.k.a. notch) is a small cut in a portion of the screen displaying no UI. Generally, notches are at the top edge of the screen and hold the cameras and sensors. The very first device I discovered featuring a cutout was the Essential Phone. Then came the iPhoneX and its large notch. More recently, we’ve seen this “disease” spread out to a lot of other Android OEMs…

As you may have understood, I’m not fan of cutouts in general. First, from a developer point of view, as they require quite a lot of work to deal with them properly: we’ve always been used to rectangle-shaped windows1. Secondly from a UI designer point of view as they clutter the UI and make it directional. Finally from a user point of view as they are not intelligible nor visually attractive.

Cutouts look like a temporary solution to technical impossibilities. I don’t consider them as viable in the long term and can’t wait to see full edge-to-edge screens. But they help in the mean time. As a consequence, we need to deal with them and that’s probably why Android P brings support for it. I took some time to look at the framework additions in Android P preview 1 and started to have fun with notches rather than complain about their introduction. So let’s have fun with cutouts!

Sliding cutout

The first concept I thought about was to trick the user about the actual purpose and origin of the cutout. In other words, we want to fully embrace the cutout shape in the app UI. Indeed, it would let the user think the cutout is actually part of the design rather than part of the device. Here is a what it looks like:

The effect here consists on 2 main steps:

  1. render the content of the app under the status bar and cutout area
  2. drawing a special “cutout” background behind the app content

Fullscreen rendering

The ability to render your app content under the status bar has been introduced in KitKatLollipop so I won’t spend much time explaining how to do it. I strongly recommand you to look at the View documentation and in particular everything related to “system UI visibility”. Chris Banes’s terrific presentation about “Becoming a master window fitter” might also be extremely helpful. The only difference in Android P is the introduction of a new layoutInDisplayCutoutMode on WindowManager.LayoutParams. This field defines how the Window is laid out when there is a display cutout. As we always want our content to be laid out under the cutout area, we simply need to initialize the mode in our Activity’s onCreate():

1
2

window.attributes.layoutInDisplayCutoutMode =
        WindowManager.LayoutParams.LAYOUT_IN_DISPLAY_CUTOUT_MODE_ALWAYS

DP1 vs DP2: The flag LAYOUT_IN_DISPLAY_CUTOUT_MODE_ALWAYS has been renamed to LAYOUT_IN_DISPLAY_CUTOUT_MODE_SHORT_EDGES.

Background computation

In order to draw the blue background, we need to know about the shape of the cutout. Android P preview 1 exposes a new DisplayCutout getDisplayCutout() method on WindowInsets. The DisplayCutout class provides developers with several interesting dimensions like the safe area and the cutout region. In particular, calling getBounds() returns the Region defining the cutout. Knowing a Region also exposes its path definition thanks to the getBoundaryPath, we can easily retrieve the display cutout outline stroke in a cutoutPath property:

1
2
3
4
5
6

setOnApplyWindowInsetsListener { _, insets ->
    insets.displayCutout?.let {
        it.bounds.getBoundaryPath(cutoutPath)
    }
    insets.consumeSystemWindowInsets()
}

DP1 vs DP2: Region getBounds() has unfortunately been removed and a new List<Rect> getBoundingRects() has been added. In other words, there is no way to get the actual shape of the cutout in DP2 - which makes this effect impossible :(. You can only get the bounding rectangle. On the other side, DP2 supports multiple cutout areas.

You can then easily build the background path of your content View in its onSizeChanged callback and draw it. Most of the trick consists on using Path.Op:

1
2
3
4
5
6
7
8
9
10
11

override fun onSizeChanged(w: Int, h: Int, oldw: Int, oldh: Int) {
    super.onSizeChanged(w, h, oldw, oldh)
    backgroundPath.reset()
    backgroundPath.addRect(0f, 0f, width.toFloat(), height.toFloat(), Path.Direction.CW)
    backgroundPath.op(cutoutPath, Path.Op.DIFFERENCE)
}

override fun onDraw(canvas: Canvas) {
    super.onDraw(canvas)
    canvas.drawPath(backgroundPath, backgroundPaint)
}

While this looks pretty cool, a lot of improvements can be made. Indeed, the effect would look better if we were taking into account several device-specific particularities. First, some recent devices feature rounded corners. Tweaking the shape of the background accordingly would make the effect really shine. Similarly, this entire demo relies on the assumption the cutout is always black. But we could imagine a device with a white cutout and would have to deal with it.

As you may have noticed, really leveraging the cutout is quite painful and requires having actual hardware information about the device (screen rounded corners radius, cutout color, etc.). Unfortunately, as far as I know, the Android framework doesn’t offer such information which are rather OEM specific.

Cutout progress

The other concept I thought about was to create a progress indicator. Users generally expect an indicator to show their current position in a content that is larger than the entire screen. The old - but still perfectly accurate - concept of “scroll bars” respond to that issue. This UI effect consists on modifying the concept a bit to simply follow the horizontal contour of the cutout area instead of a straight vertical line.

Even though, the concept looks pretty simple from a UI point of view, implementing it was actually pretty tricky. Here are the main steps we need to complete in order to implement it:

  1. Building the stroke path supporting the progress indicator
  2. Computing the current progress
  3. Drawing the indicator at the given progress

Building the stroke path

The very first step of the implementation is to build the path that will be supporting the progress indicator. In a nutshell, we need to compute an open path knowing the contour of the cutout area. The figure below shows the required transition from the cutout path (retrieved via getBounds() as explained above) to the wanted final path:

At first, I though doing so would be easy using path ops. Unfortunately, path ops only work with closed contours. For example, if you try to add a vertical line to an horizontal one, you will get an empty path rather than a cross. Similarly, doing the difference between a disc and a line crossing it won’t split the disc in two distinct parts. Clearly, path ops were not the solution.

The second option was to analyse the path and only keep a subset of it. A nice way to do this would be to read the drawing commands or verbs of the Path (lineTo, cubicTo, etc.) and only keep the needed ones, potentially changing their direction. Unfortunately, Path is a pretty opaque type. It lets you add drawing commands to an existing Path but there is no way to retrieve the drawing commands.

The final option I went for was to build the Path entirely on my own. The Android framework offers a handy approximate way to flatten the Path with a series of segments. The API is pretty low-level (the result is an array of 3-floats components per point) and not really Kotlin/Java friendly but you can use Path.flatten from Android KTX to get a list of more comprehensible PathSegments. Here is the code I quickly wrote to extract only the wanted points of the path.

1
2
3
4
5
6
7
8

var points = ArrayList<PointF>()
for (i in 0..(approximate.size - 1) step 3) {
  points.add(PointF(approximate[i + 1], approximate[i + 2]))
}

var finalPoints = points
        .filter { it.y > 0f }
        .sortedBy { it.x }

Once you get all the approximated points, you can loop over them to build a Path made of multiple segments:

1
2
3
4
5

path.reset()
finalPoints.forEach {
   path.lineTo(it.x, it.y)
}
path.lineTo(width.toFloat(), 0f)

When running the code above you might notice some artifacts on the rounded portions of the cutout area. These glitches, shown below, appear in Skia (the rendering engine dealing with paths on Android) generally when drawing larges strokes on a non-smooth path (i.e. not continuously differentiable) or on zero length contours2.

In order to workaround these glitches, I used several algorithms to build a Path mostly made of Bezier curves rather than segments. I started by implementing the Ramer-Douglas-Peucker algorithm. The main purpose of the algorithm is to find a similar curve with fewer points reducing the complexity of the paths. The counter part is obviously a loss of precision. In practice, running the RDP (with an epsilon of 1) on my set of 186 points reduced it to 26 points. Because it helped getting rid of the zero-length segments, it almost removed the glitches entirely:

The second technique I used to smooth the stroke was to compute a series of Bezier paths. While this is a pretty common problem in computer graphics, I didn’t know much about it. After some research on the web I found this paper dealing with cubic splines curves. Put simply, this paper explains how to compute a cubic Bezier curve (2 points + 2 control points) for each segments of the original path. After implementing the algorithm and using it to build the final path, I ended up with a pretty solid result. The stroke now appears smooth on curved portions of the shape:

Computing the current progress

If you read this article a while ago, you probably already know how to compute the current progress based on the content scroll offset. Rather than computing the actual height in pixels of the content and the position in pixels of the visible window, the idea is to rely on a framework feature. Indeed, most scrolling container on Android (implicitly or explicitly) implement ScrollingView. The methods provided by this interface expose 3 different dimension-less values:

  • the range: the size of the content
  • the extend: the size of the window in the content
  • the offset: the position at which the window is

The current progress (between 0 and 1) can then be computed with the formula: progress = offset / (range - extend). When using a RecyclerView as our scrolling container, we end up with the following code:

1
2
3
4
5
6
7

recyclerView.addOnScrollListener(object : RecyclerView.OnScrollListener() {
    override fun onScrolled(rv: RecyclerView, dx: Int, dy: Int) {
        progressIndicator.progress =
                        rv.computeVerticalScrollOffset().toFloat() /
                        (rv.computeVerticalScrollRange() - rv.computeVerticalScrollExtent())
    }
})

Drawing the current progress

In order to draw the current progress, we need to draw a portion of the stroke. This can be done using the phase parameter of DashPathEffect. Romain Guy explained in details how to do it in this article but here is the simplified code:

1
2
3
4
5
6
7
8

val length = PathMeasure(path, false).length

paint.pathEffect = DashPathEffect(
    floatArrayOf(length, length),
    length * (1 - progress)
)

canvas.drawPath(path, paint)

And we’re done! Whenever the RecylerView scrolls, we refresh the progress indicator current progress value, forcing it to be redrawn at the given progression.

Conclusion

While these implementations rely on a lot of assumptions that may be wrong in the future (cutout color, cutout position, etc.), it was nice to see what kind of visual effects could be implemented around the display cutout. It was also a great opportunity to discover how UIs can deal with notches and what it implies for both developers and users. I really don’t like display cutouts as a user. But I had fun with cutouts writing this article! If you do too, feel free to show me your tricks on Twitter: @cyrilmottier

1: To be honest, this is not the first time developers have to deal with weird screen shapes. The Moto 360 “flat tire” and most of the recent Android Wear devices featuring a round screen are some great examples.

2: The stroke not behind anti-aliased on its top edge has nothing to do with the way we render the stroke. Indeed, the stroke is actually twice as large as the visible portion and hence renders way beyond the screen edge. This is actually due to the cutout shape. Indeed, when doing a display cutout, hardware manufacturers have only two options: remove the pixel or keep it. The result is similar on rounded corners screens and that’s why I’ve seen some anti-aliasing being done on the software side to smooth curves or corners a bit.