UITests and accessibility tests on iOS using performAccessibilityAudit

9 min readJul 17, 2023

Performing accessibility tests is one of the most important parts of making your app inclusive. In this article, we’ll cover ways to automate accessibility testing of your iOS app using the new performAccessibilityAudit method, introduced at WWDC 2023.

What is the Accessibility Inspector?

First of all you need to familiarize yourself with the Accessibility Inspector. In Apple’s words:

The Accessibility Inspector enables you to identify parts of your app that are not accessible. It provides feedback on how you can make them accessible, as well as simulating voice-over to help you identify what a Voice Over user would experience.

With Accessibility Inspector you can check the characteristics of your app’s views, such as Label, Value, Traits, Identifier and Hint. You can also perform main element actions (like a tap on a UIButton) or Custom Actions, as well as being able to visualize the hierarchy of views. But the most important thing for us right now is the second tab in the top right corner, where you can run an accessibility audit on your app’s current screen.

When clicking Run Audit, a battery of tests will be performed on the current simulator screen, taking into account the Options selected in the menu

In the third tab, located in the upper right corner, you will also find some accessibility options, such as color inversion, transparency reduction, dynamic font size, among others.

As the focus of the article is different, I will stop here. You can find more information about the Accessibility Inspector on Apple’s own website and also in this WWDC 2019 video.

UITests

With the Xcode 15.0, Apple released a new way to do accessibility UITests, using the performAccessibilityAudit method. What it does is basically run the same tests as the Run Audit button on the second tab of the Accessibility Inspector, but in an automated way.

I created this ugly app to show you how we can perform these automated tests.

You can replicate it by downloading the final project on this GitHub repo or creating these views in your UIViewController:

// Background ImageView
let backgroundImageView = UIImageView(image: UIImage(named: "space-background"))
backgroundImageView.translatesAutoresizingMaskIntoConstraints = false
backgroundImageView.contentMode = .scaleAspectFill
view.addSubview(backgroundImageView)
view.addConstraints([
    backgroundImageView.leadingAnchor.constraint(equalTo: view.leadingAnchor),
    backgroundImageView.topAnchor.constraint(equalTo: view.topAnchor),
    backgroundImageView.trailingAnchor.constraint(equalTo: view.trailingAnchor),
    backgroundImageView.bottomAnchor.constraint(equalTo: view.bottomAnchor)
])

// Confirm Button
let confirmButton = UIButton()
confirmButton.translatesAutoresizingMaskIntoConstraints = false
confirmButton.backgroundColor = .red
confirmButton.setTitle("Tap here to confirm", for: .normal)
view.addSubview(confirmButton)
view.addConstraints([
    confirmButton.centerXAnchor.constraint(equalTo: view.centerXAnchor),
    confirmButton.centerYAnchor.constraint(equalTo: view.centerYAnchor),
    confirmButton.leadingAnchor.constraint(equalTo: view.leadingAnchor),
    confirmButton.trailingAnchor.constraint(equalTo: view.trailingAnchor)
])

// Checkbox Button
let checkboxButton = UIButton()
checkboxButton.translatesAutoresizingMaskIntoConstraints = false
checkboxButton.setImage(UIImage(named: "checkbox_icon"), for: .normal)
view.addSubview(checkboxButton)
view.addConstraints([
    checkboxButton.widthAnchor.constraint(equalToConstant: 40),
    checkboxButton.heightAnchor.constraint(equalToConstant: 40),
    checkboxButton.leadingAnchor.constraint(equalTo: view.leadingAnchor),
    checkboxButton.bottomAnchor.constraint(equalTo: confirmButton.topAnchor)
])

// Terms Label
let termsLabel = UILabel()
termsLabel.translatesAutoresizingMaskIntoConstraints = false
termsLabel.backgroundColor = .yellow
termsLabel.textColor = .black
termsLabel.text = "By selecting this checkbox, you are confirming that you have read and agree with the Terms of Use and Privacy Policy."
view.addSubview(termsLabel)
view.addConstraints([
    termsLabel.bottomAnchor.constraint(equalTo: confirmButton.topAnchor, constant: -8),
    termsLabel.leadingAnchor.constraint(equalTo: checkboxButton.trailingAnchor),
    termsLabel.trailingAnchor.constraint(equalTo: view.trailingAnchor)
])

Now that our ugly app is ready, let’s run the tests. In your UITests class, in the setUpWithError() func, let’s set continueAfterFailure = true for the purposes of this article. That way, when we run the test it will show all the UI errors, not just the first one to be identified.

override func setUpWithError() throws {
    continueAfterFailure = true
}

Finally we can create our accessibility test using the performAccessibilityAudit function. To do this, we just need to navigate to the screen where we want to run the tests and call the function. In the case of our app, the screen to be audited is the app’s main one, so we just need to call the launch() function and then performAccessibilityAudit itself.

func testAccessibility() throws {
    let app = XCUIApplication()
    app.launch()
    
    try app.performAccessibilityAudit()
}

When running this test, you will see that not coincidentally the output will be the same as running the Audit from the Accessibility Inspector app.

With the errors in hand, we need to understand what each error message is about. To do so, click on the Report navigator (1) > Tests (2) > Error you want to investigate (3) > Element Screenshot (4).

Now that we know where the problems are, we need to learn how to fix them.

Let’s start with the termsLabel (the UILabel with the yellow background). It has 2 errors: “Dynamic Type font sizes are unsupported” and “Text clipped”. To fix the first one, we need to assign the font of this UILabel and also the adjustsFontForContentSizeCategory; for the second one, we need to set numberOfLines = 0 so that the label has the size that is necessary to display all the text. Like this:

// Fix "Dynamic Text font sizes are unsupported"
termsLabel.font = .preferredFont(forTextStyle: .body)
termsLabel.adjustsFontForContentSizeCategory = true
// Fix "Text clipped"
termsLabel.numberOfLines = 0

Now the button “Tap here to confirm” (confirmButton). It has the same termsLabel font problem, so let’s solve the problem in the same way:

// Fix "Dynamic Text font sizes are unsupported"
confirmButton.titleLabel?.font = .preferredFont(forTextStyle: .body)
confirmButton.titleLabel?.adjustsFontForContentSizeCategory = true

However, if you run the test again, you will notice that another error appeared in the confirmButton: “Text clipped”. Now that there is the possibility of the font changing its size, there is also the possibility that the text does not fit the default size of the button. To fix this, let’s do exactly what we did in the termsLabel again (after all, the UIButton’s title is nothing more than a UILabel):

// Fix "Text clipped"
confirmButton.titleLabel?.numberOfLines = 0

Now all that’s missing is the error “Image name used in description” from the checkboxButton. In a UIButton that has no title, just an image, and that has not had its accessibilityLabel explicitly defined, iOS will consider the asset’s name as a accessibilityLabel, which is normally not very clear. Imagine that you had named this checkbox asset “cb-icon”, for example. VoiceOver would announce “cb-icon, button”. It’s not clear to the user, is it? For this reason, we need to explicitly define which accessibilityLabel we want for this button:

checkboxButton.accessibilityLabel = "Checkbox"

Ok. We solved all the problems pointed out by performAccessibilityAudit and our test is running successfully. But what if you for some reason need to ignore one of these tests, or want to restrict the tests that will run? To show you how we can do this, let’s comment out the line of code where we fixed the termsLabel “Text clipped” error.

// Fix "Text clipped"
// termsLabel.numberOfLines = 0

When you run the test again, you will notice that the previously mentioned error has returned. Now let’s assume that for some reason you don’t want “Text clipped” to be considered in tests. To do this, just pass the options you want to include in the test as a parameter:

try app.performAccessibilityAudit(for: [.contrast, .dynamicType, .elementDetection, .hitRegion, .sufficientElementDescription, .trait])
// or
try app.performAccessibilityAudit(for: .all.subtracting(.textClipped))

However, this way you are excluding the test textClipped from all screen elements. You can also filter what you want to ignore in the tests. For example, if you want to bypass the textClipped test for a specific element (termsLabel in this case):

try app.performAccessibilityAudit(for: .all) { issue in
    var ignore = false
    if let element = issue.element,
       issue.auditType == .textClipped,
       element.label.contains("By selecting this checkbox") { // termsLabel
        ignore = true
    }
    return ignore
}

To wrap up this part of the audit, I’d like to give you a tip: when identifying failures, the Accessibility Inspector also gives you suggestions on how you can fix the error. Sometimes it can be just what you needed.

Testing using the performAccessibilityAudit function does not guarantee that our app is 100% accessible, and even though these automated tests are meant to bring even greater coverage to your app, unit and manual testing are still essential to ensure your app is working as it should.

Now let’s leave the accessibility part aside for a moment and let’s do a simple UI test to verify that all the elements we want are actually showing up on the screen. For this, we will need to put a identifier on each element so that they are identified in the test:

backgroundImageView.accessibilityIdentifier = "BG_IMAGE"
confirmButton.accessibilityIdentifier = "CONFIRM_BUTTON"
checkboxButton.accessibilityIdentifier = "CHECKBOX_BUTTON"
termsLabel.accessibilityIdentifier = "TERMS_TEXT"

Now, we just create our UI test to verify that the elements are displaying as expected:

func testAllElementsExists() throws {
    let app = XCUIApplication()
    app.launch()
        
    XCTAssert(app.images["BG_IMAGE"].exists)
    XCTAssert(app.buttons["CHECKBOX_BUTTON"].exists)
    XCTAssert(app.staticTexts["TERMS_TEXT"].exists)
    XCTAssert(app.buttons["CONFIRM_BUTTON"].exists)
}

For this case it was simple, but eventually you need to define the accessibilityElements of the view and some element ends up not being exposed for the test. In this case, the backgroundImageView is just decorative and doesn’t need (and shouldn’t) be an accessible element for assistive technology, so we could ignore it by adding the following line to our ViewController:

view.accessibilityElements = [confirmButton, checkboxButton, termsLabel]

When you run the test again you will see that the first Assert will fail. For that, as of this new release we have the automationElements property, where we can define the views that will be exposed for UI tests.

view.automationElements = [backgroundImageView, confirmButton, checkboxButton, termsLabel]

Running the test again you will see that the result is no longer failing. We are exposing to the tests which elements we want to be visible, without the need to keep changing the accessibilityElements or declaring an accessible element just to be able to perform the UI test. This new property will have a huge impact for UITests and also for app accessibility. Apple, you took too long, but thanks!

Unit tests

To further expand coverage, you can also unit test your ViewControllers and components. Let’s assume you have the following component in your app:

final class CheckboxButton: UIButton {
    
    private var customAccessibilityTraits: UIAccessibilityTraits?
    
    override init(frame: CGRect) {
        super.init(frame: frame)
        isAccessibilityElement = true
    }
    
    required init?(coder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }
    
    override var accessibilityTraits: UIAccessibilityTraits {
        get {
            return customAccessibilityTraits ?? (isSelected ? [.button, .selected] : [.button])
        }
        set {
            customAccessibilityTraits = newValue
        }
    }
}

We can create some accessibility unit tests for it:

func testCheckboxButtonAccessibility() throws {
    let myButton = CheckboxButton(frame: CGRect(x: 0, y: 0, width: 200, height: 80))
    
    XCTAssertTrue(myButton.isAccessibilityElement)
    XCTAssertEqual(myButton.accessibilityTraits, .button)
    
    myButton.isSelected = true
    XCTAssertEqual(myButton.accessibilityTraits, [.button, .selected])
    
    myButton.accessibilityTraits = .notEnabled
    XCTAssertEqual(myButton.accessibilityTraits, .notEnabled)
}

In this case, the component is a Checkbox and has selected and not selected states. For each case, the accessibilityTraits must behave differently, that’s why I made this test. We can configure and check several other accessibility parameters. The more we test it, the better it will be.

Conclusion

The new performAccessibilityAudit function has a huge power to automate all your accessibility UITests. The automationElements property also came to help a lot with handling views in tests, avoiding the insertion of accessibility bugs when programming UITests. However, despite all the power of performAccessibilityAudit, it represents just one more way to test your app, and it is still of paramount importance that you also do manual, regressive, unit tests and everything you can to ensure that your app delivers best possible quality in all areas.

I’ll leave here some useful links in case you want to study a little about accessibility:

Article: Accessibility and UI Testing in iOS
Article: Verifying VoiceOver: Accessibility Inspector
Article: iOS Accessibility
Article: Testing UIKit accessibility in unit tests
Orange A11y Guidelines
Appt Org — Code samples
Daniel Devesa — Accessibility expert iOS developer, author of Developing Accessible iOS Apps.

I hope this article has been helpful to you. If you liked what you read, follow me, give your like and share the article that will help me a lot.

LinkedIn: https://www.linkedin.com/in/victorcatao/

UITests and accessibility tests on iOS using performAccessibilityAudit

What is the Accessibility Inspector?

UITests

Unit tests

Conclusion

Written by Victor Catão