Test-Driven iOS: Introduction to Feature Tests

Reading Time: 6 minutes

In past articles about test-driven iOS, I have talked about how to resolve some challenges of unit testing an iOS app. This time I’ll talk about higher level tests that validate how the view looks and behaves when users interact with it (UI tests) as well as the app’s API requests and response handling (integration tests).

A feature test validates both the UI and the API calls of an app to document how a feature should work.

Today we’ll look at an example feature test in Swift. I’ll start by explaining the structure and purpose of feature tests, but you can skip to the code if you prefer.

The point of a feature test

We use unit tests to make sure that the individual components of the classes in our app are working. We use integration tests to make sure that a collection of those classes work correctly together. More broadly, we use integration tests to check whether the classes in our app work correctly together with the API of another app that talks to our app. A UI test makes sure that the app responds correctly when users tap buttons, move dials, and otherwise interact with the app.

Our feature test brings the integration and UI tests together to create a test that describes how a feature works. It provides security that the app works as expected, which gives us the freedom to refactor and release with confidence.

Features are like onions

shrek

When we start writing a feature in our iOS app, we can begin by imagining how that feature should look and act when we finish writing it. For example, we can say “This feature will allow users to enter a string in a search field. Then, when the user taps the ‘search’ button, our app will call an outside service and display the results of the search.” When our app’s functionality meets that description, the feature is done.

We can do a similar thing with feature tests. Once we finish imagining how a feature should work, we can codify our understanding in a feature test—which will fail, because we have not written that functionality yet. As we write our code, we fulfill the requirements of the feature. When the feature test passes, we know we are done!

One programmer that I paired with recommended starting every feature with a feature test, then jumping all the way in to the innermost layer (say, a service that makes an API call) and writing unit tests and code in there. When those unit tests pass, we commit the change, minus the feature test. Then we move out to the ViewController, do unit tests, write code, commit. We continue this pattern until the feature test passes, and we commit the feature test last. So we start from the outside and go in, but commit from the inside out.

Feature tests in Swift

UI Tests came to XCode in 2015, but the Swift feature test APIs do not have much documentation. In this example, we will look at some of the options in the feature test API. This example serves as a starting point; it does not exhaustively catalog the capabilities of the API. The idea is to get you started so you can continue to explore the APIs according to your needs for testing your app.

An age-old integration test problem…solved by Pact?

When we write integration tests, we usually have to decide whether to integrate with the real API or hook up our tests to a mock. The tradeoff is this: with a mock API, we define the endpoints, so our tests will not fail if the real API goes down or changes. But if we use the real API, say, to test search results, and the data coming down changes, then our assertions will fail even though our integration itself is copacetic.

The Pact Consumer library is designed to resolve this either/or question for Swift and Objective C. Pact verifies the request object that our app is sending and, if that request works against the real API, sends our app a canned response containing data that we define. So, for example, if we are testing a search API, Pact could make sure our request fulfills a contract, hit the api with a request that fits that contract, and upon getting a 200 response send our app a list of search results containing predefined values that we can check in the UI display, et cetera.

I have used Pact in the following feature test example for an iOS app written in Swift.

An Example

First, let’s take a look at our feature test code. Then we can go through it by chunks and talk about what it does.


import XCTest
import Hamcrest
import PactConsumerSwift
class CandiesMasterDetailFeature: XCTestCase {
override func setUp() {
super.setUp()
continueAfterFailure = false
XCUIApplication().launch()
}
func testMasterDetail() {
let expectation = expectationWithDescription("Pact tests complete")
let candiesService = MockService(
candy: "Service API",
consumer: "Our App",
done: { result in
assertThat(result, equalTo(PactVerificationResult.Passed))
expectation.fulfill()
})
candiesService
.given("a list of candies")
.uponReceiving("a request for candies")
.withRequest(
method:.GET,
path: "/candies",
query: ["search-text": "fruit"])
.willRespondWith(
status:200,
headers: ["Content-Type": "application/json"],
body: ["candies": [
[
"id": "1",
"name": "Lemon Drops",
],
[
"id": "2",
"name": "Chocolate Covered Candied Ginger",
],
[
"id": "3",
"name": "Orange Slices",
],
]])
candiesService
.given("Orange slices")
.uponReceiving("a request for orange slice information")
.withRequest(
method:.GET,
path: "/candies/3")
.willRespondWith(
status:200,
headers: ["Content-Type": "application/json"],
body: ["candy": [
"name": "Orange Slices",
"price": 6.75,
"flavors": [
"orange",
"lemon",
"lime",
"cherry",
]
]])
candiesService.run { (testComplete) -> Void in
let app = XCUIApplication()
let searchBar = app
.otherElements["CandiesSearchTextBox"]
.childrenMatchingType(.SearchField)
.element
searchBar.tap()
searchBar.typeText("fruit")
app.keyboards.buttons["Search"].tap()
let orangeSlicesRow = app.tables.cells.elementBoundByIndex(2)
assertThat(orangeSlicesRow.staticTexts["Orange Slices"].exists, equalTo(true))
orangeSlicesRow.tap()
assertThat(app.staticTexts["Orange Slices"].exists, equalTo(true))
let candyNameDetailRow = app.tables.cells.elementBoundByIndex(0)
assertThat(candyNameDetailRow.images["OrangeSlices"].exists, equalTo(true))
app.swipeUp()
let priceRow = app.tables.cells.elementBoundByIndex(1)
assertThat(priceRow.staticTexts["$6.75"].exists, equalTo(true))
let flavorsRow = app.tables.cells.elementBoundByIndex(2)
assertThat(flavorsRow.staticTexts["orange, lemon, lime, cherry"].exists, equalTo(true))
testComplete()
}
waitForExpectationsWithTimeout(20, handler: nil)
}
}

How much of that made sense without any explanation? Ideally, these tests can clearly tell the story of how a feature works so developers can use them to learn about a code base or understand why something is not working.

Now I’ll provide the same feature test, with a lot of comments added in for explanation.


import XCTest
import Hamcrest
import PactConsumerSwift
class CandiesMasterDetailFeature: XCTestCase {
override func setUp() {
super.setUp()
continueAfterFailure = false //So the moment that an assertion in the test fails, we will not continue with the test
XCUIApplication().launch() //Launches the test in the iOS simulator. We can access the app with XCUIApplication()
}
func testMasterDetail() {
let expectation = expectationWithDescription("Pact tests complete") //Setting up our Pact Contract expectations
let candiesService = MockService( //This mock service will validate our contract expectations
candy: "Service API",
consumer: "Our App",
done: { result in
assertThat(result, equalTo(PactVerificationResult.Passed)) //Here is where the actual contract assertion takes place
expectation.fulfill() //And this is where Pact provides our predetermined responses if our request fits the contract
})
candiesService
.given("a list of candies")
.uponReceiving("a request for candies")
.withRequest( //asserts the details of the request that we are making to the API
method:.GET,
path: "/candies",
query: ["search-text": "fruit"])
.willRespondWith( //Here is the predetermined response that we ask Pact to give to our app
status:200,
headers: ["Content-Type": "application/json"],
body: ["candies": [
[
"id": "1",
"name": "Lemon Drops",
],
[
"id": "2",
"name": "Chocolate Covered Candied Ginger",
],
[
"id": "3",
"name": "Orange Slices",
],
]])
candiesService
.given("Orange slices")
.uponReceiving("a request for orange slice information")
.withRequest(
method:.GET,
path: "/candies/3")
.willRespondWith(
status:200,
headers: ["Content-Type": "application/json"],
body: ["candy": [
"name": "Orange Slices",
"price": 6.75,
"flavors": [
"orange",
"lemon",
"lime",
"cherry",
]
]])
candiesService.run { (testComplete) -> Void in //This block runs our assertions inside of our app.
let app = XCUIApplication() //Here we will give a variable name to our app for assertions
let searchBar = app //Look for a search field on the app's visible view
.otherElements["CandiesSearchTextBox"] //with the accessibilityIdentifier "CandiesSearchTextBox"
.childrenMatchingType(.SearchField)
.element
searchBar.tap() //tap that search bar
searchBar.typeText("fruit")
app.keyboards.buttons["Search"].tap() //Press a button on the keyboard that displays the "Search" text
let orangeSlicesRow = app.tables.cells.elementBoundByIndex(2) //Get the third row in the table visible on screen
assertThat(orangeSlicesRow.staticTexts["Orange Slices"].exists, equalTo(true)) //make sure it has the text "Orange Slices" in it
orangeSlicesRow.tap() //Tap that row.
assertThat(app.staticTexts["Orange Slices"].exists, equalTo(true)) //Check for the name of the candy on the detail page
let candyNameDetailRow = app.tables.cells.elementBoundByIndex(0) //Find the first element in our table of Orange Slices data
assertThat(candyNameDetailRow.images["OrangeSlices"].exists, equalTo(true)) //Make sure the orange slice image is visible
app.swipeUp() //Swipe upward on the screen (to scroll down)
let priceRow = app.tables.cells.elementBoundByIndex(1) //Find the second row in the table, which contains price data
assertThat(priceRow.staticTexts["$6.75"].exists, equalTo(true)) //Check that the price text is in that row
let flavorsRow = app.tables.cells.elementBoundByIndex(2) //Find the third row in the table, which contains flavors
assertThat(flavorsRow.staticTexts["orange, lemon, lime, cherry"].exists, equalTo(true)) //Check that the flavor text is in that row
testComplete() //if all the previous assertions pass and we arrive at this line, end the test.
}
waitForExpectationsWithTimeout(20, handler: nil) //If testComplete() is not called after 20 seconds of running, fail the test.
}
}

Working with feature tests like this will frequently require some experimentation. I have found it helpful to look into the classes providing the feature test API to figure out which methods might be helpful for my use cases. If feature tests do not appear to be running as expected, it can also be helpful to watch them run in the iOS simulator. All iOS tests run on the simulator, so you can watch the screen to see what might be happening. In some cases a button might not be getting focus because the view hasn’t loaded to tap it, or a static text might not show up because the screen size of the simulator is smaller than the screen size of the simulator that the feature test was designed against. Though the feature testing process has its hangups—especially at this relatively early stage in API development—the feature tests can provide an excellent way to define and track progress while building iOS apps.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.