cisimple joins appurify

Hello cisimple users,
I have some big news to announce today.

As you may know, we recently partnered with Appurify to integrate our build system with their on-device testing services.  While working closely with their team, we realized very quickly that in many ways we were working to solve the same problems.  Furthermore, we feel that there is a significant overlap in our visions for the future of mobile testing and CI.

What came of all this?  Well, I’m announcing today that cisimple will be joining the Appurify team.  Appurify has an incredible team of engineers that has developed a lot of surprisingly advanced technology for testing mobile applications.  To this, cisimple will be bringing over our expertise in mobile build systems, testing frameworks and most importantly… simplicity.  I strongly believe that the end result is going to be a major net positive for mobile developers at large.

What does this mean for cisimple?  Unfortunately, given the resource constraints of working in a startup environment you are often forced to make tough choices about where you focus your resources.  As such we will be discontinuing cisimple as a hosted service on 9/1/13.  Until that time the service will remain active and will be free of charge.  In fact, all users who paid in advance for September have already been refunded and no additional charges will be made.  I apologize for the inconvenience this may cause and, as always, please let me know if there is anything we can do to help with the transition.

Finally, I just want to say a big, resounding, thank you.  Developing cisimple has been an incredibly rewarding process for everyone involved.  I have had the distinct pleasure to meet and speak with many of you.  As much as possible, I have tried to drive our development process according to your input.  Your feedback has been invaluable and I hope that in return we have taken some of the pain out of your development process.

P.S. We are considering releasing cisimple as an Open Source project.  If this is something that you think would be interesting and/or valuable, let me know: kevin@cisimple.com

Kevin Rohling
founder/ceo cisimple

Standard

We just made testing mobile applications… simple

Here at cisimple, we’re committed to making mobile development a faster, less painful and more agile process.  Over the last few months, as we’ve developed our product in a private beta, we learned about many of the pain points mobile developers face and have worked hard to incorporate their feedback into what we’re building.

When we started, it was obvious that setting up and configuring a build server was painful so, we developed an incredibly streamlined setup process.  We have since received remarkably positive feedback about that part of our product.  Now, we’re taking the same philosophy and applying it to the process of testing mobile applications.

Comprehensive Platform Coverage

While a limited degree of testing support existed in previous versions, we have expanded on this functionality to allow selection from a wide array of platform options.  When configuring a test, it’s literally as easy as checking check boxes to select which SDK’s and which Simulators to test on.  We currently support all SDK’s back to 4.3 and every type of Simulator, including both iPhones and iPads.

Image

Test Reporting

We updated our build screen to include a report on which tests are passing and failing.  On the new build screen you get a simple layout of each test with pass/fail information, the entire log output as well as a video capture of the entire test execution.

Image

Run In Browser (Kickfolio Integration)

Last but not least, through a partnership with our friends at Kickfolio, we are really excited to announce that we’ve added support for running your mobile applications in a web browser, right from cisimple.  Anytime you see “run in browser” in cisimple, it means you can click the link to see and interact with the designated build, without any devices and without spinning up any simulators.  Yeah, it’s kind of magical…

To see an example, of what this is like, check out our Featured Apps page: https://www.cisimple.com/featured_apps

Since we’ve deeply integrated this functionality right into cisimple, we’re making it available to all of our paid plans at no additional cost.  We hope you enjoy and I really think it’s going to make for a pretty incredible experience!

Image

What’s Next?

We think this is a great step towards simplifying and automating testing for mobile developers but there’s still a lot more to do.  We need to add support for additional testing frameworks for one.  We currently only support UIAutomation and OCUnit, and we know that there are many great frameworks out there that developers are using (and loving) today.  We also know that testing doesn’t stop at Simulators, and we have some really exciting plans in the works to improve our support for other testing tools.  Finally, one of our greatest challenges ahead is how we provide useful, actionable feedback for the great majority of developers that have not implemented automated tests.  All of this, and a lot more, is in the works and coming your way very soon!

Please, if you have any feedback or thoughts, or just want to say hi give us a shout: team@cisimple.com or feel free to reach me directly: kevin@cisimple.com

Thank you!

The folks at cisimple.

Standard

Check out our shiny new API and Pusher integration!

api-docs

Here at cisimple, we’re committed to playing well with others. It’s something our mothers taught us when we were very little and we think those childhood lessons apply equally well to your Continuous Integration data. That’s why we’re super excited to be releasing the first version of the cisimple API!  Here’s a quick overview of what’s included in this release:

  • Retrieve a list of your Jobs
  • Retrieve a list of Builds for a given Job
  • Retrieve data about a single Build
  • Kick off a new Build for a given Job
  • Stop all Builds for a given Job

Be sure to check out the API docs for more details.  Ok, so what’s this about Pusher channels?

Yeah, we’re integrated with Pusher, a bad ass Websocket messaging service!  That means you can easily build apps that receive notifications every time a build starts, passes or fails. In fact, one of our incredibly brilliant users, Romain Pouclet, developed an OS X application that uses our Pusher integration to display Notification Center alerts.  You can download it from the site here and, since it’s Open Source you can check out the code on Romain’s Github account.

cisimple-notifications-screenshot

If you end up building something fun, useful or just interesting with the cisimple API let us know.  We’ll get it featured and spread the word.  One suggestion we’ve been kicking around: Broken Build Attack Robot.

I have to admit, this whole API thing, it was all my mom’s idea… Thanks mom!

Standard

cisimple and HockeyApp: All your builds are belong to us

We’re excited to announce that we’ve has just released support for distributing apps using HockeyApp.  You can learn more about the integration on our DevCenter.

cisimpleplushockey

Why would we do this?
First, we’ve long been fans of their product, and as a mobile dev myself I’ve used them in a number of past lives and previous projects. Speaking from my own experience, I’ve found their OTA uploads to be incredibly fast and they have a great user experience for managing a large number of applications and versions. If you haven’t tried it you really should… go on, this blog post will be right here waiting when you’re done.

Another reason we felt HockeyApp would make a great integration is their support for Android. We’ve been integrated with TestFlight since the very beginning but that left our Android folks feeling a bit left out. We’re glad to finally be closing that gap.

Finally, and most importantly, because you asked for it! There’s nothing more rewarding to us than delivering on feature requests.

What’s it do?
Well, we compile apps and run automated tests. HockeyApp (among other things) does OTA distribution, pushing your app out to beta testers. Think of your build process like a gauntlet: if your commit compiles and passes your automated tests then it’s automatically pushed out to your human testers.

There’s two ways to do this:
1. Fully Automated: cisimple will push a new release to HockeyApp for every code commit
2. Manual: Find the build you want to distribute and click “Upload To HockeyApp”

In each case cisimple does all the compiling and codesigning for you so you can do things that don’t suck and are fun.  Like flirting with your office manager.  Just don’t let HR find out.

screenshot2

What’s Next?
We’re in love with the wealth of services that exist to make mobile developers’ lives easier. Our philosophy is to partner with them and build tight integrations directly into our product. If you’ve got a partnership or integration request, give us a shout: team@cisimple.com

Standard

cisimple gets a makeover and adds OTA support for iOS

Over here at cisimple we’ve been hard at work on new features and we’re excited to announce that we’ve released a complete redesign of our UI.  While the previous experience was functional, we felt there was a lot of room for improvement so we went back to the drawing board and rethought literally every page.  Setting up CI for mobile is now faster and better looking than ever!

In this latest release we’ve also baked OTA support right into the platform.  To be clear, we will continue to support other OTA solutions like TestFlight and plan to add others in the future.  However, some of our users were looking for something lightweight that didn’t require signing up for another service.  Check out our Dev Center article to learn more!

ios_ota

Here are a few of our favorite design updates:

Nav Bar

The new nav bar has a fixed width and a slight glow around the edges.  The Dev Center was added as a top-level menu item and active pages are indicated with a depressed 3D button appearance.  The account drop down is still there but it’s been moved to the right and replaced with the name of the user.

new-nav-bar

Dashboard

The dashboard now has it’s own nav bar that includes the “New Job” button. The “Delete” option has been moved to the job drop down and the grid itself has a brand new visual appearance with sporty green buttons.

new-dashboard

Repositories

I’ve gotta say the new repositories page is one of my favorites!  We’ve separated out your repositories and your organizations using the grouping on the right.  Also, for adding public and non-Github repositories just click “Use Another Repsitory” to show a form for entering your Git URL and Branch.  The repository list itself has also been tidied up a lot and has a much cleaner appearance.

new-repositories

Teams

The new Teams page is another great update.  We’ve separated “My Teams” and “Memberships” into separate tabs and clicking the “Add New Team” button exposes a form for creating new teams.  We also removed text links and replaced them with icons to save room.

new-teams

We hope that you enjoy these updates as much as we enjoyed building them!

Standard

What Makes Continuous Delivery For Mobile Different?

I talk about Continuous Delivery for mobile applications a lot (just ask my girlfriend). Sometimes I talk about it to myself, sometimes to other people, and every once in a while it comes up in casual conversation with taxi drivers. People have a lot of questions about what Continuous Delivery means, which I typically explain as a process engineers use to automate builds, testing and deployment of their software.

Once I get through explaining that bit they often ask what makes mobile different? If this is something people already do, why build a product focused just on mobile? Those are valid questions, the reality is there are a number of Continuous Delivery products on the web that work really well for other platforms like web. Many of them integrate with Github (like cisimple does) and can push code straight to services like Heroku.

Let me explain why mobile is a different animal…

1) Infrastructure

As we all know, iOS is an huge part of the mobile ecosystem and it doesn’t look like this is changing any time soon. If you’re at all familiar with iOS development then you’re almost certainly well acquainted with Xcode and I’ll bet you a warm cup of coffee you’re running all that on a Mac. Know how I know? Of course, you can only build (and compile) iOS applications on Mac hardware! What that means for CD services is that if you’re going to support iOS you better have some Mac servers. Unfortunately, there are all sorts of scaling difficulties inherent in dealing with Macs, which is likely why other CD services don’t do iOS builds.

2) Compilation/Code Signing

Environments designed to compile mobile applications are quite different from web environments. There are often a number of dependencies that need to be in place to make the compile work. Once a compilation has completed and you’re left with an APK (Android) or IPA (iOS) file there’s a code signing step that’s required before an app can be installed on a device. This is an important and non-trivial step that has no analog in a web environment. Any Continuous Delivery solution that isn’t specifically designed to handle this step will not be able to produce builds that run on devices. Furthermore, since code signing certificates/keystores are sensitive pieces of data the signing process needs to be specially designed to protected them.

3) Testing

Testing mobile applications bares some resemblance to web testing. There are both Functional and Unit testing frameworks for iOS and Android. For more information about what frameworks are available check out this earlier blog post. However, running Functional tests on a mobile application requires the use of an Emulator/Simulator or a physical device. In either case, automatically starting/stopping emulators and simulators is difficult and error prone and physical devices become bottle necks in a hosted environment. There just isn’t a corollary to this for web developers

4) Deployment

Deployment for mobile typically means either distribution to testers or publishing your app to an App Store. For web developers getting your app our there is as easy as committing your code, especially if you’re doing some form of Continuous Delivery. For mobile this is a multi step process that requires building, signing and uploading your app to a service like TestFlight or Hockey App.

Standard

A Taxonomy Of (Native) Mobile Testing Frameworks

I’ve almost certainly missed some frameworks here. Let me know what’s missing and we’ll get it added asap.

Despite the fact that there are defacto testing frameworks for both Android and iOS a number of alternatives have cropped up. These 3rd party frameworks address some of the shortcomings of the vendor provided options by adding additional features, addressing ease of use and adding language options. The biggest differentiators between these frameworks are typically 1) Preferred testing language (i.e. JavaScript, Obj-C, Cucumber, Ruby, etc.) 2) Dependencies on things like a web server 3) Required project changes 4) The ability to run tests inside an IDE (XCode/Eclipse).

I think the cause of this fragmentation has been the explosive success of mobile over the last few years. The testing and tools space just hasn’t been able to keep pace. That said, as one of the sponsors of the recent Mobile Testing Summit we saw first hand that there is a growing community dedicated to making testing a first class citizen for mobile. The fragmentation problem is definitely a big one but it’s likely that within the next year or two a clear winner will emerge and that will be good for everybody.

Here we’re attempting to layout a number of these popular testing frameworks useful specifically for native mobile development. Our motivation is to try and bring some clarity to a space that is still highly fragmented. This list is broken up into two categories. The first category is Unit Testing Frameworks. These are frameworks that are meant to test small pieces of code in isolation. The second is Functional Testing Frameworks, which are meant to test at the UI level. Functional tests test the system as a whole, essentially reproducing user interaction.

Unit Testing Frameworks

OCUnit
apple-logo
OCUnit is Apple’s “Out Of The Box” Unit Testing solution. It’s been included with XCode since version 2.1. Probably the biggest advantage is it’s tight integration with XCode. These tests are often quite easy and fast to run as well because there’s no dependency on having a running Simulator.

- (void) testCentigradeFreezingPoint {
	          NSString *kelvinFreezingPoint = @"273";
	          NSNumber *centigradeFreezingPoint =         [transformer transformedValue:kelvinFreezingPoint];
	          STAssertEquals(0, [centigradeFreezingPoint intValue],
	                    @"Centigrade freezing point should be 0, it was %d instead!",
	                     [centigradeFreezingPoint intValue]); 
}

JUnit
android-logo
JUnit is the corresponding “Out Of The Box” solution for Android developers. Similiarly it is tightly integrated with Eclipse but can also be run from the command line using Ant. Test runs are quite slow because of the dependency on a running emulator.

public class SomeTest extends AndroidTestCase {


    public void testSomething() throws Throwable {

       Assert.assertTrue(1 + 1 == 2);

     }


    public void testSomethingElse() throws Throwable {

       Assert.assertTrue(1 + 1 == 3);

    }

}

Kiwi
apple-logo
Kiwi is a BDD framework for iOS with a focus on simplicity and readability. It reminds me a lot of RSpec for you rubyists, which is definitely a plus. Tests are written in Objective-C and run very nicely inside Xcode.

	describe(@"Team", ^{
	    context(@"when newly created", ^{
	        it(@"should have a name", ^{
	            id team = [Team team];
	            [[team.name should] equal:@"Black Hawks"];
	        });

	        it(@"should have 11 players", ^{
	            id team = [Team team];
	            [[[team should] have:11] players];
	        });
	    });
	});

Cedar
apple-logo
Cedar is the brainchild of the folks over at Pivotal Labs. Like Kiwi it’s also a BDD style framework but has a bit more functionality. Some interesting features include Matchers (a cleaner approach to assertions) and built in type mocking. The installation process even installs Xcode code snippets to make writing tests even easier.

	spy_on(someInstance);
	id fake = fake_for(someClass);
	id anotherFake = fake_for(someProtocol);
	id niceFake = nice_fake_for(someClass);
	id anotherNiceFake = nice_fake_for(someProtocol);

NSUnit
apple-logo
NSUnit is a unit testing framework for Objective-C projects. NSUnit builds on top of SenTestingKit and provides a more Objective-C friendly API, particularly for testing against Foundation classes. NSUnit adds some really useful API’s including an Assert class, primitives handling and a fails method on Assert that makes performing failing tests a one-liner.

 [Assert that:name isNot:[AnEmpty string]];
 [Assert that:name isNot:[Equal to:@"Omar"]];
 [Assert that:name is:[Equal to:"McNulty"]];
 [Assert that:someArrayOfInts areAll:[Less than:@20]];

 [Assert fails:^{
    [WebService connectToUrl:nil];
 }];

Functional Testing Frameworks

UIAutomation
apple-logo
Like OCUnit, UIAutomation is deeply integrated with XCode and is the default Functional testing framework for iOS. Tests are written in JavaScript and can be run on either the Simulator or a Device.

	var testName = "Test 1";
	var target = UIATarget.localTarget();
	var app = target.frontMostApp();
	var window = app.mainWindow();

	var textValue = window.staticTexts()["RecipeName"].value();
	if (textValue === recipeName){
	    UIALogger.logPass( testName ); 
	}
	else{
	    UIALogger.logFail( testName ); 
	}

Appium
apple-logo
Appium is a newly introduced framework from the folks at Sauce Labs. Appium is built on top of UIAutomation which is nice because no modifications are needed to your project. Also, tests can be written using any language. It does have a dependency on running a web server which could be a win or loss depending on your setup. Perhaps the most notable feature of Appium is that it is designed to be compatible with all off the shelf Selenium client libraries.

Robotium
android-logo
The name pretty much says it all, “It’s like Selenium, but for Android”. There really aren’t a lot of competing frameworks on the Android platform so Robotium has become quite popular. It’s even supported by a number of hardware testing services like Testdroid.

  public void testPreferenceIsSaved() throws Exception {
                solo.sendKey(Solo.MENU);
                solo.clickOnText("More");
                solo.clickOnText("Preferences");
                solo.clickOnText("Edit File Extensions");
                Assert.assertTrue(solo.searchText("rtf"));
  }

GHUnit
apple-logo
GHUnit was written by Gabriel Handford and has some really interesting features like the ability to test UI elements and parallel and asynchronous tests. It also has a feature that allows you to record a diff of view images so you can see how they’ve changed over time.

- (void)testFoo {
   NSString *a = @"foo";
   GHTestLog(@"I can log to the GHUnit test console: %@", a);
    // Assert a is not NULL, with no custom error description
   GHAssertNotNULL(a, nil);
    // Assert equal objects, add custom error description
   NSString *b = @"bar";
   GHAssertEqualObjects(a, b, @"A custom error message. a should be equal to: %@.", b); 
}

Calabash
apple-logoandroid-logo
Calabash is actively developed and supported by the guys at LessPainful. It’s meant to be a cross platform tool for enabling tests to interact with both native and hybrid applications. In theory actual tests can be written with any Ruby based test framework but Cucumber seems to be the most popular.

Feature: Rating a stand
   Scenario: Find and rate a stand from the list
     Given I am on the foodstand list
     Then I should see a "rating" button
     And I should not see "Dixie Burger & Gumbo Soup"
     When I touch the "rating" button
     Then I should see "Dixie Burger & Gumbo Soup"

KIF
apple-logo
KIF is short for Keep It Functional and was developed by the engineering team over at Square which right off the bat gives it a lot of validation. It’s 3 main design tenants are Minimize Indirection, Easy Configuration and Test Like A User. Also, the fact that tests are written using Objective-C is a big win for most iOS developers.

	+ (id)scenarioToLogIn;
	{
	    KIFTestScenario *scenario = [KIFTestScenario scenarioWithDescription:@"Test that a user can successfully log in."];
	    [scenario addStep:[KIFTestStep stepToReset]];
	    [scenario addStepsFromArray:[KIFTestStep stepsToGoToLoginPage]];
	    [scenario addStep:[KIFTestStep stepToEnterText:@"user@example.com" intoViewWithAccessibilityLabel:@"Login User Name"]];
	    [scenario addStep:[KIFTestStep stepToEnterText:@"thisismypassword" intoViewWithAccessibilityLabel:@"Login Password"]];
	    [scenario addStep:[KIFTestStep stepToTapViewWithAccessibilityLabel:@"Log In"]];

	    // Verify that the login succeeded
	    [scenario addStep:[KIFTestStep stepToWaitForTappableViewWithAccessibilityLabel:@"Welcome"]];

	    return scenario;
	}

Frank
apple-logo
Frank is another Cucumber based testing framework for iOS and claims that setup takes less than 10 minutes. It also has the ability to record video of your UI tests and can run on both the Simulator and Devices. A really useful feature is View Selectors which make it quite easy to interact with UI elements.

	•	view
	•	view marked:'Foo'
	•	view marked:'Foo Bar Baz'
	•	label marked:"I'm a label"
	•	view:'CustomViewClass' tableViewCell view marked:'blah'

Standard