In past articles about test-driven iOS, I have talked about how to resolve some challenges of unit testing an iOS app. This time I’ll talk about higher level tests that validate how the view looks and behaves when users interact with it (UI tests) as well as the app’s API requests and response handling (integration tests).
A feature test validates both the UI and the API calls of an app to document how a feature should work.
Today we’ll look at an example feature test in Swift. I’ll start by explaining the structure and purpose of feature tests, but you can skip to the code if you prefer.
The point of a feature test
We use unit tests to make sure that the individual components of the classes in our app are working. We use integration tests to make sure that a collection of those classes work correctly together. More broadly, we use integration tests to check whether the classes in our app work correctly together with the API of another app that talks to our app. A UI test makes sure that the app responds correctly when users tap buttons, move dials, and otherwise interact with the app.
Our feature test brings the integration and UI tests together to create a test that describes how a feature works. It provides security that the app works as expected, which gives us the freedom to refactor and release with confidence.
Features are like onions
When we start writing a feature in our iOS app, we can begin by imagining how that feature should look and act when we finish writing it. For example, we can say “This feature will allow users to enter a string in a search field. Then, when the user taps the ‘search’ button, our app will call an outside service and display the results of the search.” When our app’s functionality meets that description, the feature is done.
We can do a similar thing with feature tests. Once we finish imagining how a feature should work, we can codify our understanding in a feature test—which will fail, because we have not written that functionality yet. As we write our code, we fulfill the requirements of the feature. When the feature test passes, we know we are done!
One programmer that I paired with recommended starting every feature with a feature test, then jumping all the way in to the innermost layer (say, a service that makes an API call) and writing unit tests and code in there. When those unit tests pass, we commit the change, minus the feature test. Then we move out to the ViewController, do unit tests, write code, commit. We continue this pattern until the feature test passes, and we commit the feature test last. So we start from the outside and go in, but commit from the inside out.
Feature tests in Swift
UI Tests came to XCode in 2015, but the Swift feature test APIs do not have much documentation. In this example, we will look at some of the options in the feature test API. This example serves as a starting point; it does not exhaustively catalog the capabilities of the API. The idea is to get you started so you can continue to explore the APIs according to your needs for testing your app.
An age-old integration test problem…solved by Pact?
When we write integration tests, we usually have to decide whether to integrate with the real API or hook up our tests to a mock. The tradeoff is this: with a mock API, we define the endpoints, so our tests will not fail if the real API goes down or changes. But if we use the real API, say, to test search results, and the data coming down changes, then our assertions will fail even though our integration itself is copacetic.
The Pact Consumer library is designed to resolve this either/or question for Swift and Objective C. Pact verifies the request object that our app is sending and, if that request works against the real API, sends our app a canned response containing data that we define. So, for example, if we are testing a search API, Pact could make sure our request fulfills a contract, hit the api with a request that fits that contract, and upon getting a 200 response send our app a list of search results containing predefined values that we can check in the UI display, et cetera.
I have used Pact in the following feature test example for an iOS app written in Swift.
An Example
First, let’s take a look at our feature test code. Then we can go through it by chunks and talk about what it does.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import XCTest | |
| import Hamcrest | |
| import PactConsumerSwift | |
| class CandiesMasterDetailFeature: XCTestCase { | |
| override func setUp() { | |
| super.setUp() | |
| continueAfterFailure = false | |
| XCUIApplication().launch() | |
| } | |
| func testMasterDetail() { | |
| let expectation = expectationWithDescription("Pact tests complete") | |
| let candiesService = MockService( | |
| candy: "Service API", | |
| consumer: "Our App", | |
| done: { result in | |
| assertThat(result, equalTo(PactVerificationResult.Passed)) | |
| expectation.fulfill() | |
| }) | |
| candiesService | |
| .given("a list of candies") | |
| .uponReceiving("a request for candies") | |
| .withRequest( | |
| method:.GET, | |
| path: "/candies", | |
| query: ["search-text": "fruit"]) | |
| .willRespondWith( | |
| status:200, | |
| headers: ["Content-Type": "application/json"], | |
| body: ["candies": [ | |
| [ | |
| "id": "1", | |
| "name": "Lemon Drops", | |
| ], | |
| [ | |
| "id": "2", | |
| "name": "Chocolate Covered Candied Ginger", | |
| ], | |
| [ | |
| "id": "3", | |
| "name": "Orange Slices", | |
| ], | |
| ]]) | |
| candiesService | |
| .given("Orange slices") | |
| .uponReceiving("a request for orange slice information") | |
| .withRequest( | |
| method:.GET, | |
| path: "/candies/3") | |
| .willRespondWith( | |
| status:200, | |
| headers: ["Content-Type": "application/json"], | |
| body: ["candy": [ | |
| "name": "Orange Slices", | |
| "price": 6.75, | |
| "flavors": [ | |
| "orange", | |
| "lemon", | |
| "lime", | |
| "cherry", | |
| ] | |
| ]]) | |
| candiesService.run { (testComplete) -> Void in | |
| let app = XCUIApplication() | |
| let searchBar = app | |
| .otherElements["CandiesSearchTextBox"] | |
| .childrenMatchingType(.SearchField) | |
| .element | |
| searchBar.tap() | |
| searchBar.typeText("fruit") | |
| app.keyboards.buttons["Search"].tap() | |
| let orangeSlicesRow = app.tables.cells.elementBoundByIndex(2) | |
| assertThat(orangeSlicesRow.staticTexts["Orange Slices"].exists, equalTo(true)) | |
| orangeSlicesRow.tap() | |
| assertThat(app.staticTexts["Orange Slices"].exists, equalTo(true)) | |
| let candyNameDetailRow = app.tables.cells.elementBoundByIndex(0) | |
| assertThat(candyNameDetailRow.images["OrangeSlices"].exists, equalTo(true)) | |
| app.swipeUp() | |
| let priceRow = app.tables.cells.elementBoundByIndex(1) | |
| assertThat(priceRow.staticTexts["$6.75"].exists, equalTo(true)) | |
| let flavorsRow = app.tables.cells.elementBoundByIndex(2) | |
| assertThat(flavorsRow.staticTexts["orange, lemon, lime, cherry"].exists, equalTo(true)) | |
| testComplete() | |
| } | |
| waitForExpectationsWithTimeout(20, handler: nil) | |
| } | |
| } |
How much of that made sense without any explanation? Ideally, these tests can clearly tell the story of how a feature works so developers can use them to learn about a code base or understand why something is not working.
Now I’ll provide the same feature test, with a lot of comments added in for explanation.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import XCTest | |
| import Hamcrest | |
| import PactConsumerSwift | |
| class CandiesMasterDetailFeature: XCTestCase { | |
| override func setUp() { | |
| super.setUp() | |
| continueAfterFailure = false //So the moment that an assertion in the test fails, we will not continue with the test | |
| XCUIApplication().launch() //Launches the test in the iOS simulator. We can access the app with XCUIApplication() | |
| } | |
| func testMasterDetail() { | |
| let expectation = expectationWithDescription("Pact tests complete") //Setting up our Pact Contract expectations | |
| let candiesService = MockService( //This mock service will validate our contract expectations | |
| candy: "Service API", | |
| consumer: "Our App", | |
| done: { result in | |
| assertThat(result, equalTo(PactVerificationResult.Passed)) //Here is where the actual contract assertion takes place | |
| expectation.fulfill() //And this is where Pact provides our predetermined responses if our request fits the contract | |
| }) | |
| candiesService | |
| .given("a list of candies") | |
| .uponReceiving("a request for candies") | |
| .withRequest( //asserts the details of the request that we are making to the API | |
| method:.GET, | |
| path: "/candies", | |
| query: ["search-text": "fruit"]) | |
| .willRespondWith( //Here is the predetermined response that we ask Pact to give to our app | |
| status:200, | |
| headers: ["Content-Type": "application/json"], | |
| body: ["candies": [ | |
| [ | |
| "id": "1", | |
| "name": "Lemon Drops", | |
| ], | |
| [ | |
| "id": "2", | |
| "name": "Chocolate Covered Candied Ginger", | |
| ], | |
| [ | |
| "id": "3", | |
| "name": "Orange Slices", | |
| ], | |
| ]]) | |
| candiesService | |
| .given("Orange slices") | |
| .uponReceiving("a request for orange slice information") | |
| .withRequest( | |
| method:.GET, | |
| path: "/candies/3") | |
| .willRespondWith( | |
| status:200, | |
| headers: ["Content-Type": "application/json"], | |
| body: ["candy": [ | |
| "name": "Orange Slices", | |
| "price": 6.75, | |
| "flavors": [ | |
| "orange", | |
| "lemon", | |
| "lime", | |
| "cherry", | |
| ] | |
| ]]) | |
| candiesService.run { (testComplete) -> Void in //This block runs our assertions inside of our app. | |
| let app = XCUIApplication() //Here we will give a variable name to our app for assertions | |
| let searchBar = app //Look for a search field on the app's visible view | |
| .otherElements["CandiesSearchTextBox"] //with the accessibilityIdentifier "CandiesSearchTextBox" | |
| .childrenMatchingType(.SearchField) | |
| .element | |
| searchBar.tap() //tap that search bar | |
| searchBar.typeText("fruit") | |
| app.keyboards.buttons["Search"].tap() //Press a button on the keyboard that displays the "Search" text | |
| let orangeSlicesRow = app.tables.cells.elementBoundByIndex(2) //Get the third row in the table visible on screen | |
| assertThat(orangeSlicesRow.staticTexts["Orange Slices"].exists, equalTo(true)) //make sure it has the text "Orange Slices" in it | |
| orangeSlicesRow.tap() //Tap that row. | |
| assertThat(app.staticTexts["Orange Slices"].exists, equalTo(true)) //Check for the name of the candy on the detail page | |
| let candyNameDetailRow = app.tables.cells.elementBoundByIndex(0) //Find the first element in our table of Orange Slices data | |
| assertThat(candyNameDetailRow.images["OrangeSlices"].exists, equalTo(true)) //Make sure the orange slice image is visible | |
| app.swipeUp() //Swipe upward on the screen (to scroll down) | |
| let priceRow = app.tables.cells.elementBoundByIndex(1) //Find the second row in the table, which contains price data | |
| assertThat(priceRow.staticTexts["$6.75"].exists, equalTo(true)) //Check that the price text is in that row | |
| let flavorsRow = app.tables.cells.elementBoundByIndex(2) //Find the third row in the table, which contains flavors | |
| assertThat(flavorsRow.staticTexts["orange, lemon, lime, cherry"].exists, equalTo(true)) //Check that the flavor text is in that row | |
| testComplete() //if all the previous assertions pass and we arrive at this line, end the test. | |
| } | |
| waitForExpectationsWithTimeout(20, handler: nil) //If testComplete() is not called after 20 seconds of running, fail the test. | |
| } | |
| } |
Working with feature tests like this will frequently require some experimentation. I have found it helpful to look into the classes providing the feature test API to figure out which methods might be helpful for my use cases. If feature tests do not appear to be running as expected, it can also be helpful to watch them run in the iOS simulator. All iOS tests run on the simulator, so you can watch the screen to see what might be happening. In some cases a button might not be getting focus because the view hasn’t loaded to tap it, or a static text might not show up because the screen size of the simulator is smaller than the screen size of the simulator that the feature test was designed against. Though the feature testing process has its hangups—especially at this relatively early stage in API development—the feature tests can provide an excellent way to define and track progress while building iOS apps.