Tuesday January 15, 2013

✦ UI Screen Shooter

Looking for more information about UI Automation? Check out my new book, published through the Pragmatic Programmers. Thanks for your support!

Like most iOS developers, I hate updating screen shots for the App Store. When the iPhone 5 came out I had to walk through the app to take all the screen shots again. For both new and old dimensions. In different languages.

That’s when I decided to buckle down and build UI Screen Shooter, a sample project with a bunch of scripts that use UI Automation to set up, walk through, and take screen shots of an app. Check out a video of it at work on one of my apps. To try it out yourself, pull down the project and simply run this from the terminal:

./run_screenshooter.sh ~/Desktop/shots

The final screen shots are named according to locale, device family, and screen orientation. Using these techniques it’s trivial to update all the screen shots at once.

The scripts themselves have a lot of moving parts. I don’t want to overwhelm you or duplicate too much of the information here that you can already glean from the comments in the project, but I do want to point you in the right direction. There are two parts to cover, the first is the actual automation script itself that walks through the app snapping pictures. The second is the shell script that automatically builds and runs the walkthrough automation script against all the device types and languages and extracts the screen shot PNGs into a destination directory.

UI Automation - The Easy Part

If you’ve never messed with UI Automation before, I’d recommend checking out the first part of my tutorial. That will bring you up to speed on how UI Automation works as part of Instruments. There’s nothing special you have to install, it just works out of the box with Apple’s tools.

All the automation scripts for this example are in the automation directory. The “driver” that actually controls the application is in the shoot_the_screen.js file. This is the automation script that is automatically run by the run_screenshooter.sh shell script. Let’s take a look at what it does for this app:

#import "capture.js"

var target = UIATarget.localTarget();
var window = target.frontMostApp().mainWindow();




We’re using pure UI Automation to tap a button and wait before snapping pics, but instead of using target.captureScreenWithName(...) we’re using a custom captureLocalizedScreenshot() function that is defined in the imported file, capture.js. This function takes the string and introspects the simulator type, orientation, and language to build up a screenshot filename that looks like this:


That name is then passed to target.captureScreenWithName(...). The screen shots are stored in the automation trace results directory. If you save the Instruments trace document somewhere you can reach it, you’ll see the PNG files nearby.

This automation script is where you’ll do the bulk of the tweaking. Walk through your own app setting up the proper state and then snap pics with captureLocalizedScreenshot(). I’d recommend checking the script in each locale and device you want to test while you’re tweaking it here in Instruments. Once you know everything is working, then we can move on to the hard part…automating the whole process.

Shell Scripting - The Complicated Part

Alas, there are a lot of moving parts to run_screenshooter.sh. I tried to pull as much as I could out into subscripts in the bin directory. The rest of the script needs to share information like where the app bundle lives.

The script automates the process of building the app, choosing the simulator device and language, running your screen shooting automation script, and extracting the screen shots out of the trace document directory and into the destination folder. Most of this shell script will just work with whatever app you throw at it, but there are three places you may need to tweak. I’ll itemize those below.

1. Choose Your Languages

The first thing you’ll want to change is the set of languages that the script loops over. At the top of the file, you’ll see a line like this:

languages="en fr ja"

This is simply a space separated list of locale identifiers. Adjust those to taste for your audience. That’s the easy step.

2. Choose Your Device Families

Switching simulators requires a bit of a dance. I’m using simple bit of AppleScript in bin/choose_sim_device to select a device type from the menu, but when launching an app with Instruments from the command line, it will always default to iPad if the app is universal. That means we need to rebuild the app to target the specific device families.

I tried to make the script easier to digest by breaking it up into bash functions. The main function is (unsurprisingly) the main entry point of the script. I have some cleanup code in there to check that the destination doesn’t exist and save the original language to restore it later, but then we reach the good stuff that does the build and run:

_xcode clean build TARGETED_DEVICE_FAMILY=1

bin/choose_sim_device "iPhone (Retina 3.5-inch)"

bin/choose_sim_device "iPhone (Retina 4-inch)"

To keep things organized for you, anything that starts with an underscore is a function within the run_screenshooter.sh and every command that starts with bin/ is a supplementary script in the bin directory.

The _xcode bash function, which we’ll dissect more in a moment, instructs Xcode to clean and build the app. By using the TARGETED_DEVICE_FAMILY config variable we instruct Xcode to alter the Info.plist file so that the app is forced to either iPhone (family 1) or iPad (family 2).

We then choose the sim device we want to test on, 3.5 inch retina display in this case, and run the _shoot_the_screens_for_all_languages function that loops over all the $languages, runs the automation scripts, and extracts the results. Do the same thing again for the 4-inch retina iPhone display.

To run on the iPad as well, we need to rebuild the app so that it targets the iPad device family, switch the simulator so it uses the retina iPad, and then shoot the screens:


bin/choose_sim_device "iPad (Retina)"

If your app isn’t universal, just use the code that makes sense for you.

3. Build Your App

This can be a very tricky part that depends quite a bit on the way you have your Xcode project or workspace set up. Let’s look at that _xcode bash function that does the building for us:

function _xcode {
  xcodebuild -sdk iphonesimulator \
    PRODUCT_NAME=app \

It’s simply a wrapper around the xcodebuild command to build a project from the command line that specifies some important configuration settings. First, note that we’re setting CONFIGURATION_BUILD_DIR to point to the temporary directory specified by the $build_dir environment variable. That part shouldn’t have to change for you. Also, by using the $* at the end of the command, any parameters passed to _xcode will be passed through to xcodebuild.

If you are using an Xcode workspace, you’ll have to specify a -workspace and a -scheme argument to the command here as well. Use man xcodebuild for more information about that if you need it.

The second bit relates to the way I’m setting the PRODUCT_NAME configuration variable. To keep things simple, I’m forcing Xcode to name the app bundle as app.app. That gets placed in the $build_dir directory so that we end up with our app bundle in /tmp/screen_shooter/app.app.

This is important because we need to know the full bundle path to pass in to the instruments command so we can run our automation scripts against it. And our bundle path CANNOT have any spaces in it. The Instruments command line tool will trip up if we specify a bundle path with a space, so I hard code it to a temporary directory out of the way and force the resulting bundle name to be app.app just in case.

The problem is, setting the PRODUCT_NAME configuration setting can break a build with more than one target. That’s because the product name affects all the targets that are being built. If your app is named Frodo and you have a static library target named Gandalf, then this will change the product name of both targets and the linker will get confused when it tries to link the Frodo product with a nonexistant Gandalf static library.

The only way I can see around this is to build your app without the PRODUCT_NAME config setting. Then move and rename the app bundle to /tmp/screen_shooter/app.app so the rest of the script knows what to do. The bad news is that you’ll need to write these steps yourself, the good news is that all the changes can be isolated in the _xcode bash function. The rest of the script will just chug along fine.

Bottom line: however you get it in there, you need the final app bundle built for the iOS simulator and in this path: /tmp/screen_shooter/app.app.

The Rest Of The Story

I’ve already spilled enough bits to send your head spinning so I’ll stop here. These steps above are all you have to do to get the shell script to automate the screen shots of your app. I put comments all over the place to describe what’s going on, and feel free to open and issue on github if you get stuck or think something is broken.

I’ve saved so much time with these scripts and I hope you do, too!

For more information about my UI Automation resources, check out my features page.