I thought about doing this in Swift but decided to take this one thing at a time (I haven’t written any Swift yet). My strategy: 1) get tests working and then 2) convert them to Swift. This post focuses on (1). As for what tests to write, I had a full suite of KIF tests before I did a visual refresh, so my starting point was replacing those.
Note: I initially got errors from swift complaining about deployment target needing to be 9.0. This didn’t seem to be the case once I moved to Objective-C.
Issue 1: “No target application path specified”
Resolution: I had been trying to put my UI tests in the same target as my unit tests. Apparently you can fix this with some build settings, but for now I opted to put them in a separate target – I want my unit tests to be fast enough to develop against, and I don’t know how long the UI tests will take to run. I figure I can move my unit tests into the UI target later if they are fast enough.
Issue 2: Build errors from KIF
Resolution: Remove KIF. I’m not going to spend too much time on this since KIF is going anyway.
Test 1: Rotate home screen
Simple test that rotates the phone four times (to result in a full rotation), and after each rotation checks that the home screen buttons are still there.
I use record (launches the app and inserts the programatic equivalent of your actions) to get the code, and then refactor it to my taste. To verify each rotation, in KIF I had a method that waited for the buttons, here I use XCTest to assert the button exists, and that they are “hittable”.
Note: I declare all strings in the file “SHAStrings.h”, which means I can reference them from UI tests and don’t have to change my tests with copy changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – (void)testRotateHomeScreen { | |
| XCUIDevice *device = [XCUIDevice sharedDevice]; | |
| [self verifyHomePageButtons]; | |
| [device setOrientation:UIDeviceOrientationLandscapeRight]; | |
| [self verifyHomePageButtons]; | |
| [device setOrientation:UIDeviceOrientationPortraitUpsideDown]; | |
| [self verifyHomePageButtons]; | |
| [device setOrientation:UIDeviceOrientationLandscapeLeft]; | |
| [self verifyHomePageButtons]; | |
| [device setOrientation:UIDeviceOrientationPortrait]; | |
| [self verifyHomePageButtons]; | |
| } |
Test 2: Press inspire button
Test that taps one of the buttons on the home screen, and launches a web view with the tumblr page. I found a bug writing this test, so added in code that would wait for the webpage title to load (found here).
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – (void)testOpenInspireView { | |
| [[app_ buttons][[SHAStrings inspireButtonTitleString]] tap]; | |
| // Verify page load by checking the title. | |
| XCUIElement *title = [app_ otherElements][@"Show and Hide"]; | |
| NSPredicate *predicate = [NSPredicate predicateWithFormat:@"exists == 1"]; | |
| [self expectationForPredicate:predicate evaluatedWithObject:title handler:nil]; | |
| [self waitForExpectationsWithTimeout:2.0 handler:nil]; | |
| // Go back. | |
| [[[[[app_ navigationBars][@"SHAInspireView"] childrenMatchingType:XCUIElementTypeButton] | |
| matchingIdentifier:@"Back"] elementBoundByIndex:0] tap]; | |
| // Should be at Home Page. | |
| [self verifyHomePageButtons]; | |
| } |
Issue 3: Blocked HTTP Request
I actually found a bug writing this test (I love it when that happens!): “App Transport Security has blocked a cleartext HTTP (http://) resource load since it is insecure. Temporary exceptions can be configured via your app’s Info.plist file.” I fixed that following these instructions (also: need to be consistent with use of www in the plist and in the URL).
Test 3: Open the gallery and cancel
Tests that the gallery opens and then closes again.
Previously any cancel button was tapped, but the nice thing about XCUI is that I can specify the cancel button in the navigation bar.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – (void)testOpenGalleryAndCancel { | |
| [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap]; | |
| [[[[[app_ navigationBars][@"Photos"] childrenMatchingType:XCUIElementTypeButton] | |
| matchingIdentifier:@"Cancel"] elementBoundByIndex:0] tap]; | |
| // Should be at Home Page. | |
| [self verifyHomePageButtons]; | |
| } |
Test 4: Open the gallery, select an image, and go back
Tests that a photo can be selected, the edit view opens, and goes back.
Using the Photos stuff was one of the most annoying things about working with KIF, to pick a photo I had to tap on the screen at a place where there was usually a photo, which could be a little flaky (the “usually” is the clue there). I couldn’t find anything googling, and then I remembered the “record” button and voila: painless. Yay!
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – (void)testChooseImageAndGoBack { | |
| [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap]; | |
| [[[app_ tables] buttons][@"Moments"] tap]; | |
| [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap]; | |
| [self verifyEditPageButtonsVisible]; | |
| // Go back to Home Page. | |
| [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton] | |
| matchingIdentifier:@"Back"] elementBoundByIndex:0] tap]; | |
| [self verifyHomePageButtons]; | |
| } |
Test 5: Test rotation on the edit view
Test open the gallery, select an image, rotate and verify the controls disappear in landscape and re-appear in portrait.
Here I assert the buttons exist, but aren’t hittable.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – (void)testChooseImageAndRotate { | |
| XCUIDevice *device = [XCUIDevice sharedDevice]; | |
| [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap]; | |
| [[[app_ tables] buttons][@"Moments"] tap]; | |
| [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap]; | |
| [self verifyEditPageButtonsVisible]; | |
| [device setOrientation:UIDeviceOrientationLandscapeRight]; | |
| [self verifyEditPageButtonsNotVisible]; | |
| [device setOrientation:UIDeviceOrientationPortrait]; | |
| [self verifyEditPageButtonsVisible]; | |
| [device setOrientation:UIDeviceOrientationLandscapeLeft]; | |
| [self verifyEditPageButtonsNotVisible]; | |
| // Return to portrait. | |
| [device setOrientation:UIDeviceOrientationPortrait]; | |
| // Go back to Home Page. | |
| [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton] | |
| matchingIdentifier:@"Back"] elementBoundByIndex:0] tap]; | |
| [self verifyHomePageButtons]; | |
| } |
Test 5 and 6: Tap the respective sliders
Opens gallery, selects image, taps slider, goes back.
I never actually got anything happening on the sliders using KIF, and they are still not changing value, but the tap should trigger the image being regenerated, which is something.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – (void)testChangeAccuracySlider { | |
| [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap]; | |
| [[[app_ tables] buttons][@"Moments"] tap]; | |
| [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap]; | |
| XCUIElement *slider = [app_ sliders][[SHAStrings accuracySliderA11yLabel]]; | |
| [slider tap]; | |
| // Go back to Home Page. | |
| [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton] | |
| matchingIdentifier:@"Back"] elementBoundByIndex:0] tap]; | |
| [self verifyHomePageButtons]; | |
| } | |
| – (void)testChangeToleranceSlider { | |
| [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap]; | |
| [[[app_ tables] buttons][@"Moments"] tap]; | |
| [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap]; | |
| [self verifyEditPageButtonsVisible]; | |
| XCUIElement *slider = [app_ sliders][[SHAStrings toleranceSliderA11yLabel]]; | |
| [slider tap]; | |
| // Go back to Home Page. | |
| [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton] | |
| matchingIdentifier:@"Back"] elementBoundByIndex:0] tap]; | |
| [self verifyHomePageButtons]; | |
| } |
Test 7: Change the picture from the edit page
Opens gallery, selects image, on edit page selects another image and goes back.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – (void)testChangeImageFromEditPage { | |
| [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap]; | |
| [[[app_ tables] buttons][@"Moments"] tap]; | |
| [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap]; | |
| [self verifyEditPageButtonsVisible]; | |
| XCUIElement *navBar = [app_ navigationBars][[SHAStrings imageEditingViewControllerTitle]]; | |
| [[navBar buttons][[SHAStrings galleryButtonA11yLabel]] tap]; | |
| [[[app_ tables] buttons][@"Camera Roll"] tap]; | |
| [[[app_ collectionViews] cells][@"Photo, Landscape, March 12, 2011, 4:17 PM"] tap]; | |
| [self verifyEditPageButtonsVisible]; | |
| // Go back to Home Page. | |
| [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton] | |
| matchingIdentifier:@"Back"] elementBoundByIndex:0] tap]; | |
| [self verifyHomePageButtons]; | |
| } |
Test 8: Test Swipe
Opens gallery, selects image, swipes back and forth, goes back.
Recording didn’t give me the swipe gesture, so instead tapped on the indicator instead of swiping. Then looked it up and used [app_ swipe].
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| – (void)testSwipe { | |
| [[app_ buttons][[SHAStrings galleryButtonTitleString]] tap]; | |
| [[[app_ tables] buttons][@"Moments"] tap]; | |
| [[[app_ collectionViews] cells][@"Photo, Landscape, August 08, 2012, 7:52 PM"] tap]; | |
| [self verifyEditPageButtonsVisible]; | |
| XCUIElement *element = [[[app_ scrollViews] childrenMatchingType:XCUIElementTypeOther] element]; | |
| XCUIElement *imageButton = [[[element childrenMatchingType:XCUIElementTypeOther] | |
| elementBoundByIndex:0] buttons][[SHAStrings processedImageA11yLabel]]; | |
| [imageButton pressForDuration:1]; | |
| XCUIElement *page1Of2PageIndicator = [app_ pageIndicators][@"page 1 of 2"]; | |
| [page1Of2PageIndicator tap]; | |
| XCUIElement *page2Of2PageIndicator = [app_ pageIndicators][@"page 2 of 2"]; | |
| [page2Of2PageIndicator tap]; | |
| [app_ swipeLeft]; | |
| XCTAssertTrue([page2Of2PageIndicator isHittable]); | |
| [app_ swipeRight]; | |
| XCTAssertTrue([page1Of2PageIndicator isHittable]); | |
| // Go back to Home Page. | |
| [[[[[app_ navigationBars][@"Edit"] childrenMatchingType:XCUIElementTypeButton] | |
| matchingIdentifier:@"Back"] elementBoundByIndex:0] tap]; | |
| [self verifyHomePageButtons]; | |
| } |
Test 9: Test share.
Opens gallery, selects image, selects share, goes back.
Space and Time
According to RescueTime: 2h 23m of software development + 30 minutes in notes (writing what turned into this post).
Code: KIF file deleted: 232 loc. UI tests added: 234 loc. Bonus: the UI tests have better coverage. Although the biggest benefit of switching over is removing a dependency, especially one that has been a bit annoying to maintain.
Observations
- Record makes a great starting point for tests, once I got into the habit of using it things went faster. I ended up rewriting a lot of the code it produced, though.
- I think these tests run faster than KIF, but not by that much. Worth keeping a separate target. Bonus: means the inner workings of your app are not exposed (for me, just that strings file).
- Sometimes a bit flakey – I think this might be because there’s some asynchronicity and my computer was running slowly.
- So far the images have renamed themselves (with a different timestamp – weird) once. I changed it to just pick the first image in the collection.
Resources
More on unit testing, with a focus on UI code, can be found in my unit testing workshop.
5 replies on “Replacing KIF Tests with XCUI Tests”
This is so great–thanks!! Two questions:
1. You mention that “the UI tests have better coverage”. Can you explain what you mean?
2. In what way were the tests flakey? How did they fail? Did XCUI fail to discover elements, despite them being on the screen?
[WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.
1. I was able to test some things that I couldn’t in KIF. Specifically – swiping the images.
2. No, I think things just timed out sometimes.
[WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.
Can you share your KIF Tests? It would be nice to see what changed
[WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.
You can find the KIF tests here: https://gist.github.com/catehstn/fc9a16e6e9017d106f49
Note: the UI is slightly different so they do not map exactly.
[WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.
Thanks for sharing. Very useful example.
[WORDPRESS HASHCASH] The poster sent us ‘0 which is not a hashcash value.