/Simon

BDD framework for iOS development.

Primary LanguageObjective-CBSD 3-Clause "New" or "Revised" LicenseBSD-3-Clause

(simple) Simon BDD framework

Simon is a BDD framework for testing apps on iOS simulators and devices. I’ve written it to be as simple to use and easy to add to your projects as possible. In fact, it’s so simple that all you need to do is to write a story file, include a single line macro to run Simon, and then include another single line macro on each step implementation.

Why use a BDD framework?

Customes don’t understand code. So showing then a Unit test run or raw code means little to them. That’s ok, that’s why we write the code and they don’t. However, customers do understand their own requirements and they usually can express those requirements quite well.

Behaviour Driven Development is an agile technique where we get the customer to express their requirements in simple story like terns. Then using a framework like Simon, we match executable code to the stories, thus proving that they are implemented. By doing this we can prove that the app meets the customers expectations.

Why did I write Simon?

There are other BDD style frameworks for iOS. But after looking at them, they each had something I wasn’t that fond of. Everyone required me to modify my Mac for accessibility. Some placed the code in the same files as the steps. And other things. So I sat down and thought about what I wanted as a developer. I came up with this list:

  • I wanted to use Objective C because that’s what every iOS developer already knows.
  • I wanted it to be easy to include and run without requiring modifications to the system.
  • I wanted to use as little as possible “Glue” code to relate story steps to Objective C methods.
  • I wanted the stories to be stored in seperate files from the code so they would be easier to modify and update and the customer could read them.
  • I wanted it to run on the simulator.
  • I wanted it – most of all – to be dead simple to run.

Where did the name come from?

When I was thinking about keeping things simple, I thought of the old nursery rhyme Simple Simon and the name stuck.

Ok how does it really work?

Here’s the basic sequence:

  1. The Customer and Developer work together to write stories in the form “As/Given …. then ….” For example “Given Simon is running Then I expect it to execute the stories”.
  2. You create a new target which runs your app as though you where going to manually test it.
  3. You add the text files containing the stories to your code base and the new target.
  4. You add the Simon framework to this target.
  5. You then open the startup code and add 1 line to get Simon launched in the background.
  6. You write one or more classes containing the code that executed each step in the stories.
  7. You add one line macros to the classes for each step, mapping the story step to a method.
  8. You lanuch the target on your simulator or device. Simon will automatically startup, read the story files, match the steps to the methods and run each story. At the end Simon will produce a report detailing successful executions of stories or failures.

How do I install it?

First there are three frameworks you need to download. They are:

Download the DMG files for each. Open each one and drag and drop the framework files to a directory. Open XCode again, select your target and go to Build Phases and Link Binary With Libraries". Then click *[+] to add library. Select Add Other… and then the framework directory you just copied. Do the same for the other frameworks.

Adding Simon to your project

Simon is designed to run on a background thread in your app. To save you from having the headache of working this out, there is a simple macro you can use to install and run stories. This macros:

  1. Sets up Simon on a background thread.
  2. Waits for your app to finish loading by watching the UIApplication notifications.
  3. Once your app is up, starts running the stories.

You place this macro into the main.m file of your app. Here’s a simplified example so show you what it looks like:

#import <UIKit/UIKit.h>

#import "SISimon.h"

int main(int argc, char *argv[]) {
   
   NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];

   // Add Simon.
   SIRun();

   int retVal = UIApplicationMain(argc, argv, nil, @"YourAppDelegate");
   
   [pool release];
   return retVal;
}

There are two versions of the macros

  • SIRun() which searches the app for story files (*.stories) and runs them all.
  • SIRunFile(filename-without-extension) which runs stories from the specified file only.

How to I write a story?

Stories are written in plan text files with a “stories” extension Here’s an example:

example.stories

Story: Example story showing the basic syntax.
Given Simon is working
Then I should be able to see abc and 5 in the log
and say goodby at the end.

Thats pretty simple. The Story: … line gives the story a name. The rest of the lines outline the story and what is expected. Simon does impose some rules about the first word on each line. Ultimately it doesn’t matter, but by imposing some simple rules, it helps to better understand the stories. Yes you can have more than one per file. Here’s an outline f the syntax rules:

  • Each story must start with a line that begins with the word story or story:.
  • The structure of the story lines must conform to:
    • As … (optional)
    • Given …
    • And … (zero or more)
    • Then … (zero or more)
    • And … (zero or more)
  • Leading and trail blanks are ignored on each line.
  • Lines that begin with # are treated as comments and ignored.
  • Blank lines are ignored.

How do I relate a story to the executable code?

Firstly Simon regards each line in a story as a seperate item to be executed. So lets look at the Given Simon is working line from the above story. Lets say we have a class called StepImplementations and we want to map the step to a method in it. Here’s what it would look like:

#import <SISimon/SISimon.h>

@interface StepImplementation : NSObject {
}
@end

@implmentation StepImplementation

   SIMapStepToSelector(@"Given Simon is working", simonIsRunning) 
   -(void) simonIsRunning {
      NSLog(@"Woo hoo, Simon is running");
   }

@end

The first thing you wil notice is the inclusion of the SISimon.h header file. A necessary evil unfortunately. The second thing you will notice is the 

SIMapStepToSelector(@"Given Simon is working", simonIsRunning)

line. That’s Simon. This preprocessor directive instructs Simon to map the given step to the selector.

Now it’s not obvious from the above, but that first parameter is a regular expression as specified in the doc for the NSRegularExpression class in Apple’s documentation. This means that we can use it to pick up a wide variety of steps and also (using regular expression groups) pick out arguments to pass to the selector. Oh, and the second argument is the selector to execute. You don’t need to use objective C’s selector(...) with this as Simon needs it as text. Plus it’s less to type in :-)

How do I map arguments in a step?

Lets look at the Then … step

Then I should be able to see abc and 5 in the log

As you can see there are two possible arguments in this step – abc and 5. Simon can map these quite simply. The only rule is that the order of arguments in the method that will be executed matches the order of arguments in the step. Of course Simon also expects that the data types are a match as well. Here’s the mapping:

SIMapStepToSelector(@".* I should be able to see (.*) and (.*) in the log", verifyText:andNumber:)

-(void) verifyText:(NSString *) text andNumber:(int) number {
   NSLog(@"Text: %@", text);
   NSLog(@"Number: %i", number);
}

Simon will automatically scan the data types of the arguments and attempt a matching conversion. Also notice that in the regular expression I set the start to match any characters. This means that, for example, I could use this same expression with both a Then … and a And … step.

Note that each step in a story can matched to only one selector. But that a selector can be mapped to any number of story steps. This gives you the ability to share code across many steps, saving you from repeating yourself. Also know as the DRY principle.

But wait, what if the step implementations are in different classes?

This is where Simon gets a bit smarter than the original Simon.

When steps are in the same class

Simon instantiates each class that contains mapped steps when it needs to run those steps. In addition it also caches and reuses these objects, so transferring data between story step mappings in the same class is as simple as using class properties.

When they are in different classes

But when dealing with step mappings in different classes, Simon has to be a little smarter. Each story has a local storage area your code can use to transfer objects between classes without these classes even being aware of each other. Accessing this storage is done through two macros. Here is an example:

Communications.stories

Story: Should be able to pass objects between step classes.
Given this class stores abc in the story storage using key def
then this class should be able to retrieve abc from storage with key def

CommsASteps.m

@implementation CommsASteps

   SIMapStepToSelector(@"Given this class stores (.*) in the story storage using key (.*)", storesString:withKey:)

   -(void) storesString:(NSString *) aString withKey:(NSString *) key{
      SIStoreInStory(key, aString);
   }
 
@end

CommsAStep reads a key and a value from a step and then stores the value in the story cache so that other objects can access it.

CommsBSteps.m

@implementation CommsBSteps

   SIMapStepToSelector(@"then this class should be able to retrieve (.*) from storage with key (.*)", retrieveString:withKey:)

   -(void) retrieveString:(NSString *) aString withKey:(NSString *) key{
      NSString *value = SIRetrieveFromStory(key);
      GHAssertEqualStrings(value, aString, @"Strings do not match");
   }

@end

CommsBSteps then reads that value back from the cache and verifies it.

Just in case you are wondering

After each story has finished, Simon clears out the story’s caches – releasing the step mapping implementations. So you cannot transfer data between stories. That may come later.

How do I interact with my app?

Initially Simon is just about running code. So it’s effectively a runner for Unit Tests. The next release is planned to include utilities that will provide the means to interact with the GUI of your app. (I’m working as hard on this as I can – there’s a lot to do :-)

I’d like to say thanks to several developers for developing the following tools: