A Guide to XCode UI Test

Apple’s new UITest suite has some developers excited, and others disappointed in lost functionality. UITest works differently than the functional testing solutions developers have come to rely on, such as KIF. Instead of giving you access to elements themselves, UITest gives access to proxy elements with minimal parameters to interact with. Learning to separate unit testing and unit based uitesting from functional testing can be a frustrating experience, especially when working with a new framework that still has its own kinks to work out. It is my hope the following guide will help developers form some clarity into the methodology needed when working with UITest.

* This article was written in mid December 2015. At that point UITest was still experiencing several bugs. Many times I had tests fail that the next day would pass after no changes. It is recommend you clean and delete derived data often, as well as restart XCode from time to time. You may also have to find some creative workarounds. More in the “Gotchas” section below. When looking at the docset keep in mind that UITest is for both iOS and OSX testing and therefore there may be OSX specific items mentioned (click vs tap for example).

 

How It Works

UITest is unique in that it does not use the project code to test, it exists outside the app. UITest instead looks at what is available in the simulator and returns to us instances of XCUIElement based on what it finds, XCUIElement and XCUIApplication are the proxies that are used for this. A button is a XCUIElement of type XCUIElementTypeButton (or just .Button), for example. This means you are not able to access ObjC or Swift classes/objects nor their associated properties. You can only access XCUIElement properties. This also means there is no @testable or other such imports as in XCTest.

UITest includes a new Record feature which can be used to record actions and help speed up the writing of tests. UITests can be written without the help of the Record feature, and therefore can be part of test-driven development, however when writing tests after writing the code you are testing it is best to first use Record and then modify the results. This is because it is sometimes difficult to predict how an element will be exposed to UITest until you are more familiar with it. It also helps to identify issues you may have been unaware of such as multiple items with the same identifier.

XCUIApplication is the proxy that is used for testing. It launches a new instance of the app for every test giving you a clean slate to run your test on. As with XCTest there are setup and teardown methods and you can encapsulate common functionality such as writing a function in the test file to clear a text field which you can then call in each subsequent test that is needed. You can also make an extension on XCUIElement to reuse code in any part of the test suite.

XCUIApplication launches and terminates the app (launch and terminate, respectively). You can also pass arguments to the application on launch using launchArguments and pass an environment using launchEnvironment. From the header files / docset: “Unlike NSTask, it is legal to modify the environment [ and launch arguments ] after the application has been launched. These changes will not affect the current launch session, but will take effect the next time the application is launched.”

UITest is built on top of the XCTest Framework and includes new additions to the API:

 

There aren’t any official docs but Joe Masilotti has compiled docs based on the framework headers. The items above link directly to those docs. If the author takes the website down we have a copy of the docset forked on Github (as of Dec 8, 2015). The docset also includes the rest of the XCTest framework, not only UITest.

 

would return the XCUIElement of type .Table with the specified identifier vs the long form which would be app.containingType(.Table, identifier: "table identifier")) and one can use the long form or the short form depending on preference and need. Queries are chained together and variable names can be substituted throughout the chain:

let app = XCUIApplication()let cellQuery = app.tables.cells.containingType(.StaticText, identifier:"String Identifier")let helpButton = cellQuery.buttons["?"]helpButton.tap()

There are often multiple ways to reach an element, for instance a cell with identifier “Purchase Cell” that contains a button “Buy Item” and also a label with identifier “Purchase” could be found several ways and if you use Record you should be presented with a clickable token that allows you to choose which version you would like to use:

let cell = XCUIApplication().tables.cells["Purchase Cell"]let cell = XCUIApplication().tables.cells.containingType(.Button, identifier: "Buy Item")let cell = XCUIApplication().tables.cells.containingType(.StaticText, identifier: "Purchase")let cell = XCUIApplication().descendantsMatchingType(.Table).element.descendantsMatchingType(.Cell).matchingIdentifier("Purchase Cell")

If you were wanting to interact with the label or button rather than the cell you could do the following:

let buyButton = cell.buttons["Buy Item"]let purchaseLabel = cell.staticTexts["Purchase"]

These are short forms, the long form would be:

let buyButton = cell.childrenMatchingType(.Button, identifier: "Buy Item")

Or if you did not have a way to identify the cell but you could identify the table you can also do this:

let buyButton = table.descendantsMatchingType(.Button, identifier: "Buy Item")

If you had a cell that had two buttons with the same identifier you would need to identify which button you would like by it’s index value. Using Record is recommended as in practice I found the elements were not always at the expected indices. Note “elementAtIndex” will autocomplete but is already deprecated, use elementBoundByIndex instead.

let button = XCUIApplication().tables.cells["Purchase Cell"].buttons.elementBoundByIndex(1)

This would find the button at index 1 out of all buttons on the cell.

let button = XCUIApplication().tables.cells["Purchase Cell"].buttons["Buy Item"].elementBoundByIndex(1)

This would find the button at index 1 out of all buttons on the cell with identifier “Buy Item”.

If you have only one item of a certain type there is an additional shortcut, element. For example if you have a cell containing one button, that button can be accessed like so:

let button = cellQuery.buttons.element

For element identification there is additionally elementMatchingPredicate(NSPredicate) and elementMatchingType(XCUIElementType, identifier: String?). These are separate from

containingPredicate(NSPredicate) and containingType(XCUIElementType, identifier: String?) which are checking the element for items inside it whereas the elementMatching... options are checking the values on the element itself. Finding the correct element will often include combinations of several query attributes.

If you have difficulty interacting with an element, printing the accessibility hierarchy can help (print(app.debugDescription), and if that doesn’t work here is a tip from shinobicontrols.com:

---

 Sometimes when you tap on an element while recording, you’ll notice that the code produced doesn’t look quite right. This is usually because the element you are interacting with is not visible to Accessibility. To find out if this is the case, you can use XCode’s Accessibility Inspector.

Once it is open, if you hit CMD+F7 and hover over an element with your mouse in the simulator, then you’ll see comprehensive information about the element underneath the cursor. This should give you a clue about why Accessibility can’t find your element.

https://www.shinobicontrols.com/blog/ios9-day-by-day-day2-ui-testing

---

 

Queries Reference:
Query For Elements Directly *returns a query	objectForKeyedSubscript: / object["subscript'] = matches identifier only matchingIdentifier: matchingType:identifier: matchingPredicate: containingType:identifier: containingPredicate:
Query Elements in the Tree *returns a query	descendantsMatchingType: childrenMatchingType:
Access Elements Directly	element = returns an element elementBoundByIndex: = returns an element elementMatchingPredicate: = returns an element elementMatchingType:identifier: = returns an element allElementsBoundByAccessibilityElement = returns an array allElementsBoundByIndex = returns an array
Available Query Actions	count = returns number of matches found for the query debugDescription = can be used on an element or a query

 

Sharing Code Between App and UITest Targets

UITest is a completely separate entity from the app itself and therefore can only access UI elements however it may be beneficial to share code between the project and UITest target for writing your tests. To use code from the app target there are two options.

1.  Add the file to Compile Sources (or check off UITest target in member dependencies on the file). Must remember to also compile all files used by this file whether or not the test will use them.
2.  Take the shared code out of the app’s files and into a new file and share that file between the two. This eliminates the need to also compile dependent files.

A good example of when this would be useful is given by Big Nerd Ranch where they show how you can use shared code to check if a to-do item is marked as finished (strikethrough). While we can not access the object’s properties directly we can use shared code to return values that we can use for assertions.

 

Properties, Attributes, Methods -> XCUIElement

UITest uses the accessibility API to populate attributes for the XCUIElement under test. From Masilotti’s docs:

 

How-To’s and Gotchas

In the resource links below Joe Masilotti has several code examples including reordering cells, pull to refresh, check if an element is on screen, and handling system alerts.

The following are things myself and others from Metova have come across:

 

Frames For Off-Screen Elements

Queries will return elements which are offscreen. If the frame is needed for test logic or an assertion, before getting the frame make sure it is on screen, usually by tapping on it first.

 

Asserting Adjusted Slider Values

adjustToNormalizedSliderPosition does not guarantee accuracy but rather is an estimate. I found setting 0.0 or 1.0 does give 0% and 100% respectively but otherwise it is best to XCTAssertNotEqual to the previous value rather than test the new value.

 

Inconsistent Test Failures

UITest is still buggy. Scrolling to a cell does not work consistently. In my tests I had it fail 20-30% of the time one day and then on another day not at all. I witnessed the same thing happen to another developer as well. I also experienced tests failing one day and passing the next, with no changes to the code. If you can not figure out why a test is failing I recommend moving onto another task and returning to debug at the end. When you come back you may discover it is a passing test after all.

 

Record Returns Wrong Element

UITest will sometimes grab an image instead of a button when tapping on a button. I also noticed that UITest prefers to return Static Texts (labels) rather than buttons, this may have to do with the view hierarchy and what element is on top.

 

Slider Will Not Move

Slider only slides using adjustToNormalizedSliderPosition if it starts at 0. A work around is to use pressForDuration:thenDragToElement and find an element nearby. Remember this may not be accurate unless you are dragging to an element at the extreme beginning (0%) or end (100%)

 

Multiple Matches Failure with Recorded Code

Sometimes Record does not identify elements correctly and a test will fail due to multiple matches or no matches when Record gives you code indicating there is one unique match. I found the multiple matches error happened most often when there were multiple matches to the query but the other elements were off screen at the time of Recording. The best way around this is to assign a unique identifier or to determine which index the element is at.

 

Multiple Matches Failure with Recorded or Written Code // How to Wait for Expected Result

Another reason for a multiple matches error is that queries will sometimes not be complete when the test moves onto the next line. A delay should be able to resolve this problem. We tell it to wait for exactly one result (self.count = 1), or wait for an element to exist or not exist (element.exists == true / false) before moving on and give it a timeout value. This is supposed to be fixed for Xcode 7.1.0 however users were again reporting the problem in 7.1.1.

_ = self.expectationForPredicate(NSPredicate(format: "self.count = 1 || element.exists || etc"), evaluatedWithObject: XCUIApplication().tables, handler: nil)self.waitForExpectationsWithTimeout(5.0, handler: nil)

 

Recorded Code with Special Text/Characters Do Not Compile

Record also may not convert text properly. A button with a unicode identifier was extracted using “U2192Ufe0e” which results in a compiler error and had to be manually adjusted to “u{2192}u{fe0e}”. Be aware of this for all escaped characters and special texts such as those used in HTML Strings.

 

Record Fails To Capture Action

Record sometimes fails to capture actions. The best way to deal with this was to tap until an action was recorded, then do the action that you need and if it was recorded edit the test after to remove the unneeded actions. Several times I was not able to record an action for one element but was able to record it for the element next to it and then edit the test code to target the correct element.

 

Interacting With Elements Currently Off-Screen

You do not need to include scrolling actions to reach an element far down a page. Once the element is identified it can be interacted with no matter if it is on screen or not; the interaction should bring it on screen. When using Record scrolling actions will be recorded however these lines can be deleted from the test once the element itself is identified.

 

Work Arounds for Accessing Object Properties and States

As you can not access the object’s properties you can not for instance tell if a label is NSString or NSAttributedString (for example to check if text is strikethrough). This is because UITest only uses accessibility attributes and a person using voice-over would not be told text is strikethrough unless it is part of the object’s accessibility traits. Aside from using shared project code (as described above), a work around for this would be to change the item’s accessibilityLabel to be prefixed with “strikethrough” or if in a to-do list “done” would be more descriptive.

Sometimes you need to check that an element’s state has changed. You can achieve this usually by checking element.value before and after an action. If it is a label or image you can use the accessibility attributes to have it treated like a button.

 

Nested Collection Views

UITest was unable to identify a collectionViewCell nested in a collectionView nested in a tableViewCell. I was not able to test if a nested tableView has the same problem. It could see the element in the collectionView cell but not the cell itself so state changes on the cell could not be asserted.

 

Animations Causing UI Tests to Fail

In several cases, animation may cause the UI tests to fail (for example, waiting for a button to fade in and be clickable). In some cases, you may be able to wait until the button exists, but in some cases, it is better to disable your animations in your UI tests, and instead use unit testing to check for valid animations.

To disable animations in your test, add the following to your app delegate:

<code>if NSProcessInfo.processInfo().environment["animations"] == "0" {</code><code>UIView.setAnimationsEnabled(false)}</code>

And this to your set up for your UI test:

let app = XCUIApplication()app.launchEnvironment = ["animations": "0"]app.launch()

 

Frames For Off-Screen Elements

Queries will return elements which are offscreen. If the frame is needed for test logic or an assertion, before getting the frame make sure it is on screen, usually by tapping on it first.

 

Resources

Apple’s UI Testing in XCode
UI Testing in Xcode 7 Part 1: UI Testing Gotchas
iOS9 Day by Day: Day 2: UI Testing
UI Testing Xcode 7
UI Testing Cheat Sheet

 

SEE OUR JOB OPENINGS