A better, if somewhat costly, way to test apps on mobile devices has been launched by Xamarin.

Xamarin Test Cloud consists of 1000 real mobile devices that you can use to check whether your mobile apps actually work with a wide range of hardware.

In a blog post about the launch, Xamarin CEO Nat Friedman said:

‘The state of mobile testing today is primitive. Last month we ran a survey that found that nearly 80% of mobile developers are relying primarily on manual testing in their attempts to deliver great app experiences. And yet, more than 75% told us that the quality of their mobile apps is either “very important” or “mission critical.”’

The new service aims to provide an alternative, more effective, testing method. You use it by creating test scripts using the Xamarin testing framework. The scripts can be them locally against emulators or devices, after which you can run the same tests on as many devices as you would like in the cloud. The system is integrated with CI systems like Jenkins and TFS, so that your tests run automatically.

The framework used for the scripts is called Calabash, and it can automate and test any iOS or Android app, native or hybrid, from the UI level down. Calabash is based on software acquired by Xamarin when it acquired the startup company LessPainful in 2013. You can write your tests in C# or Ruby with Cucumber. The tests can automate user actions such as tapping, double-tapping, swiping, rotating, panning, long pressing and pinching. You can query the UI elements, app models, and backend servers via public or private endpoints, and use CSS selectors to query interface elements in hybrid apps. Tests can also simulate the conditions your users might encounter in mobile networks from EDGE to 4G, including drop-outs and high latency. Your test can change the GPS location, press physical buttons, activate the camera, and rotate the device.00

Once the test has been developed, you can specify which devices it should be run against from over a thousand models, with 100 new models being added every month. You can filter the devices to be used by manufacturer, operating system, or form factor. Once you’ve run the test, you’re shown a detailed report on the results. This shows you full-resolution screenshots of each step, and you can view data on the CPU and memory usage, how long the test took, and other performance data, along with debugging information. You can also compare reports against previous runs to find regressions and bottlenecks.

As you might guess, the service isn’t cheap. The Basic version which lets you test two apps for a maximum of 200 device-hours per month costs $12,000 per year, with prices going up from there.

A webinar about Xamarin Test Cloud is to be held on Tuesday, September 30th, at 10am PDT.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info