The 8 best Android test automation frameworks

20 March 2020 stoefln Leave a comment Test automation

Are you looking for a suitable tool for automated testing on Android? Is the testing process taking away from the time you would rather spend developing more apps?

An Android test automation framework is a crucial part of the process of app development. Automated testing is necessary to produce high-quality applications in due time.

There are many such frameworks available. However, all the frameworks have their pros and cons. To optimize automated testing for your Android apps, the right choice of frameworks is critical. But a large number of available frameworks make this decision rather tricky.

By the way, we wrote another post on the subject of choosing the right Android UI testing tool, but here we will discuss some testing solutions in more detail.

Moreover, different projects require different tools for proper testing. Not all frameworks are equally effective or equally cheap. Some can cost you quite a bit.

In this article you learn:

  • What is automated testing?
  • What are the benefits of test automation?
  • How to optimize Android test automation?
  • What frameworks are best Android test automation frameworks?

Also, make sure you read till the end for some extra tips and tricks to make your Android automation tests run even more smoothly!

android ui testing tools

What is automated testing?

Before we start, let’s clarify a basic question: What exactly is Android automated testing?

There are two main types of testing: manual and automated.

In manual testing, you have to execute test cases by yourself without the use of tools or scripts.

In automated testing, however, you will use testing frameworks and scripts.

Benefits of automated testing

There are many reasons why automating Android tests is a great thing to do:

  • Automated tests are much faster than manual tests.
  • When test cases are run many times and over longer periods of time, automated testing becomes more practical.
  • More reliable for tests that have to be done repeatedly since the process is done the same way each time.
  • Reliability means that there are less chances of inaccuracy due to human error.

Scenarios of automated testing

Automated testing might turn out to be extremely useful in these cases:

  • In case a task needs to be executed repeatedly.
  • When doing Regression Testing: since the code may need to be changed frequently.
  • In case doing Performance Testing: where we want to test how an app would behave while being used by thousands of users.
  • When doing Load Testing: where we want to test how the app handles different stages of load such as data traffic, location and slower mobile network speeds.

How to optimize automated tests on Android?

There are some general guidelines to optimize automated testing:

  • Be aware of the details regarding the application that needs to be tested. This way, you can decide what tests should be automated.
  • Work out the requirements for these automated tests. While keeping these in mind, research different potential tools (such as the ones we will list here) that fulfill the needs of your project. After finalizing the appropriate tools, proceed with the automated testing.

What are frameworks for Android test automation?

To make this task much easier for you, we have compiled a list of the eight tools that you can use to carry out automated testing on your Android applications.

1. Appium

Appium is probably the most popular automated testing tool that there is. It is an open-source tool. It’s free and supports a wide range of programming languages. These include Java, Java-Script, Objective-C, Ruby, Python, and many more.

Although Appium is a cross-platform tool, we will be discussing its applicability with Android here. It uses either UIAutomater or Instrumentation for Android, and these frameworks are used under Selenium WebDriver, which gives developers that the luxury of being able to use several languages.

Furthermore, Appium allows you to test many types of mobile applications. This includes Native, Hybrid, and Mobile Web. Lastly, Sauce Labs supports Appium for testing with real devices, simulators, and emulators.

Therefore, Appium is an extremely versatile platform for automated Android testing.

Appium features

  • No installation required.
  • A vast and active community.
  • The app does not need to be modified to be tested using Appium.
  • Appium UI Inspector: a tool that comes with Appium to assess an app’s UI components.

2. Robotium

Robotium is an Android UI automation framework. It is an open-source tool that was released back in 2010 and by now has become quite capable and reliable. Its test execution and test readability speed have increased a lot.

Robotium makes its tests more robust by using run-time binding to UI components. It comes as a jar file that you have to compile with the app that you are testing. Not only does it support both Native and Hybrid app testing, but it also supports the basic Android UI tools such as activities, menus, and buttons as well as gestures.

Robotium uses Java as its language of choice. Using Robotium, developers can write functional tests, system tests as well as acceptance test scenarios that cover a host of Android activities. You can also create Automatic GUI (Graphical User Interface) tests.

Similar to Appium, Robotium is supported by Sauce Labs, which allows for testing on real devices.

Read more about the differences between Espresso vs Robotium.

Robotium features

  • Has device control functions such as taking screenshots and changing screen orientation.
  • Can run tests on emulators and real devices.
  • Supports multiple testing at the same time.
  • Robotium recorder: A paid plugin to speed up the testing process.

3. Repeato

Repeato is a test recorder for Android and iOS which allows you to create tests via record and play, so the test author does not have to have coding skills. It works based on computer vision and machine learning to recognize and interact with the elements in your user interface.

Thus, testing becomes feasible for people without coding skills; also, thanks to computer vision, tests are very robust. When needed, tests can be recreated quickly.

Repeato features

  • Creating tests via a test recorder only takes few minutes
  • Blackbox testing (no code required)
  • Testing native widgets (GPU rendered things like maps or 3D content)

Repeato is free for small projects, just download the Android test automation tool and try it out.

4. Calabash

Calabash is another open-source automated testing framework for Android that works for both Native and Hybrid apps. It allows you and even people without coding skills to conduct automated acceptance tests on apps. Xamarin created it.

Calabash supports many languages, including Java, Flex, Ruby, and .NET. It has libraries that enable test scripts to interact with Native and Hybrid applications. You can configure your Calabash tests to run on many different Android devices smoothly. They provide real-time feedback and validation.

Furthermore, Calabash works for Behavior-Driven Development (BDD). This is slightly different and more advanced than Test-Driven Development. Essentially, in BDD, tasks are created in which the intended behavior of the app is specified. The idea behind BDD is that the software is built right from the vision of the owner without the developer having to interpret how it should behave.

Calabash features

  • Configuration facility to make tests compatible with a wide range of devices.
  • Grants access to the Xamarin test cloud.
  • Supports Cucumber (a testing tool for BDD).
  • Cucumber Automation lets you write tests in simple English.
  • Enables automatic UI interactions within an application: pressing buttons, entering text, etc.

5. Selendroid

Selendroid is another UI test automation tool. It is used for testing Android Native as well as Hybrid applications. It tests apps using the Android driver WebView app. Tests have to be written using the Selenium 2 client API.

You do not need to modify your app while using Selendroid. You can also extend it using your own extensions during run-time. It can interact with multiple android devices, including emulators, at the same time. This feature is also owed to Selendroid’s distinct feature that you can plug and unplug devices without interrupting the test.

Using Selendroid, you can also execute parallel test cases on multiple devices. It contains an Inspector tool that can analyze the UI elements of the app that you are testing. Unlike other frameworks like Appium, Selendroid can find UI elements for older versions of Android, too, as early as Android 4.4 KitKat.

Selendroid features

  • Hot plugging.
  • Compatibility with JSON Wire Protocol.
  • Advanced User Interactions API: Support for gestures.
  • Emulators are started up automatically.
  • Supports different locator types.
  • Supports many versions of android from Android 4.0 KitKat onwards.
  • The built-in Inspector simplifies test case development.

Watch a Selendroid mobile web testing demo

6. UI Automator

UI Automator is another framework that allows for automated UI testing of Android applications. It is developed and maintained by Google. It supports only Native Android apps.

UI Automator tests are written in Java. Moreover, it is a black-box testing app. This essentially means that you do not need to know the internal structure of the app being tested and can rely solely on the visible UI elements.

It gives you the ability to switch between installed and system apps, enables cross-app functional testing, and lets you test multiple apps. It comes with two sets of APIs; one allows it to manipulate the app’s UI components, while the other enables it to perform various tasks on the device.

UI Automator features

  • UI Automator Viewer: GUI tool to scan current UI components on the screen.
  • Performs interactions on system apps and user apps.
  • Performs actions such as opening app launcher in test device.
  • Simulate buttons on the screen, such as the home button.

7. Espresso

Espresso is also a useful testing tool for the automated testing of Android UI apps. App developers and testers widely use it. This framework is also from Google. Its tests are quick to execute and reliable.

On the other hand it’s not trivial to create tests. You need a lot of tweaks to really to get them stable. We covered some of the issues of espresso, i.e. about idling resources and thread sleep.

Espresso uses white-box testing, which is responsible for its fast speed. This means that you must be well-aware of the internal structure of the application while using Espresso to test it. Tests have to be written in Java – as with most Google-maintained frameworks.

Furthermore, it’s an open-source framework; hence you can easily extend it to better suit your needs. Another benefit of Espresso is that it allows you to write user interface tests for your application.

You can even create and record a scenario of interacting with the device. Capture snapshots to confirm the behavior of UI elements of the app. Espresso will use the recording and generate UI test cases for your app.

Espresso features

  • Tests can be built-in APK form, which can be installed and used quickly.
  • Does not communicate with a server.
  • Provides very stable tests thanks to the synchronized method of executing tests.
  • Works flawlessly with Android Studio IDE.

8. MonkeyRunner

MonkeyRunner is a popular tool for running automated functional tests on Android applications. The tests are written in Python.

MonkeyRunner gives you multiple device control. You can run tests across multiple devices or emulators at once, even by physically attaching them. It was developed to test applications at functional level, but you can use it to do more than just that.

This tool can also carry out something known as Regression Testing. This method of testing involves running an application and comparing the output to an already known set of correct outputs.

It also comes with an API that allows it to control functions of the device. Since MonkeyRunner uses Python, you can create modules and programs to control Android devices. Furthermore, you can extend MonkeyRunner with plugins and therefore expand its usability.

MonkeyRunner features

  • Multiple-device support.
  • Functional testing.
  • Tests can even be written using a recording tool.
  • Supports Regression Testing.

Tips for Android test automation

Now that you’re aware of the best Android UI testing frameworks for automated testing of Android apps, here are some general tips to make testing even smoother.

Test early and often

It’s a good idea to start testing early on in the development of the project, even testing smaller parts. Not only does testing more frequently help you find more bugs, but the bugs found early in the production process are also cheaper to fix.

Create good test data

Having quality test data can help with data-driven automated testing. This data is usually stored in an external file and is used to fill input fields during an automated test. Having quality test data can make it easier to write automated tests.

Use accessibility IDs

Setting accessibility IDs on your elements makes testing go much smoother.

Design app with testing in mind

Make the app testable while it is being built. If you are a developer, coordinate with the testing team for guidelines.

Use real devices

Although testing on emulators and simulators is not a bad thing as they emulate a real device, it is better to test your app on a real device to make sure that your app is compatible with its software and hardware.


The importance of mobile UI testing, especially in this era where smartphones are so widely used, is undebatable.

Nowadays, users are looking for apps that work seamlessly with their device’s user interface to give them a smooth experience.

This makes it more critical now than ever before for app publishers to ensure that their apps work exactly as they should before deploying them on the Google Play Store.

Tags: , ,

Like this article? there’s more where that came from.

Leave a Reply