Apple just announced their new UI testing features at WWDC, and it looks awesome. However, until you start building with the iOS 9 SDK, KIF is the top UI automation option. KIF is a great automated UI testing framework, but sometimes, when KIF tests fail on your continuous integration server (or locally when you weren’t looking), it can be difficult to identify what went wrong. Luckily, KIF has a feature where it can take screenshots every time a test fails. This allows you to ensure that your UI automation testing is indeed automated. Let’s face it, if you have to sit there and watch your automation tests run so you can know what happened when it failed, it’s not fully automated.
Overview
In order to have KIF take screenshots on failures, there are a couple things we need to do.
– Create a directory for KIF to save the screenshots to.
– In order to prevent our git repo from accumulating unnecessary images, we’ll want to ignore the files in our screenshots directory.
– Add a KIF_SCREENSHOTS environment variable to our scheme. When this environment variable is present and its value is an existing directory, KIF will start saving screenshots to the directory.
– Add an extra run script build phase to our KIF tests target that removes old screenshots from the directory before each new run. This will eliminate the potential confusion of knowing which screenshots are for failures that are still occurring. Also, it will keep old screenshots from taking up extra space on your local machine or continuous integration server.
Step-by-Step Guide
Navigate to your project directory and create a KIF_SCREENSHOTS directory—This can be named anything you want, but I’ve chosen to simply name it after the environment variable that we’ll end up using later:
cd /workspace/my-appmkdir KIF_SCREENSHOTS
In order to commit this directory to our git repo, there must be a file in it. Since we also want to ignore the contents of this directory, we can create a new .gitignore inside this directory to kill two birds with one stone:
cd KIF_SCREENSHOTStouch .gitignore
Your new .gitignore should look like this:
# Ignore everything in this directory*# Except this file!.gitignore
In Xcode, click on your scheme, click on “Edit Scheme…”, select “Test” on the left panel, and add a new environment variable as shown below. This tells KIF that we want it to record screenshots on failures. It also tells KIF the name of the directory that it should save the screenshots to.
To prevent old screenshots from building up in our screenshots directory, we need to add a new run script build phase to remove them at the beginning of each build:
- Select your KIF tests target.
- Navigate to the “Build Phases” tab.
- Click on the “+” sign.
- Select “New Run Script Phase”
Add the following to remove old screenshots:
rm -f "KIF_SCREENSHOTS/"*.png
Caveats
The KIF screenshots feature won’t work when run on actual devices if the directory you’ve specified is outside of the app sandbox. Furthermore, there was a bug in the error handling code that caused the test suite to exit completely. The pull request with the fix has now been merged, but KIF v3.2.3 and below will still suffer from this issue. It still won’t be able to save screenshots to a directory outside of the sandbox, but it will at least allow your test suite to continue and yield the failed test result.
