Top 5 iOS Testing Frameworks in 2021

iOS test automation frameworks

4 November 2021 stoefln Leave a comment Uncategorized

Are you seeking new ways to improve your mobile testing strategy on iOS?

There are numerous ways to accomplish this, but you only have so much time and money. Even if you think you’re an expert at iOS app testing, there’s always room for improvement. It’s beneficial to know which tools to put into your toolbox.

In this article, you will learn about the most used frameworks for the automation of iOS app testing. An iOS testing framework can cut down time you’d otherwise have to invest to check apps for bugs manually.

Top 5 iOS Frameworks in 2021:

Appium:

Appium is a grown-up open-source test automation framework driven by a big community. Part of the reason could be its programming language variety and ability to test iOS, Android, and mobile web applications.

QAs can use any programming language they want to automate their application tests. Appium uses Selenium WebDriver APIs and language-specific client libraries to allow you to write tests in the language of your choice (Java, Python, JavaScript, PHP, etc). It also provides access to backend APIs and databases, which enables quite a lot of flexibility.

Pros:

  • You can use any WebDriver-compatible language, such as Java, Objective-C, or JavaScript, to write tests in your language and tools of choice
  • The framework as a whole is open source
  • For unit testing, you can use the framework of your choice, such as XCTest
  • Different ways of locating an element are supported (XPath, ID, Text…)

Cons:

  • Setting up Appium including drivers is time-consuming (order of days)
  • Appium’s open-source components are not always compatible with each other’s versions, maintenance of the test automation setup can consume significant resources
  • Appium execution on iOS devices is relatively slow, which can backfire if tests are not executed regularly because of the big amount of time it takes to execute the whole test suite

Testing code example:

driver.findElement(By.id("com.my.app:id/myComment")).click();
driver.findElement(By.id("com.my.app:id/myComment")).sendKeys("Hi there");
driver.findElement(By.name("Post")).click();

XCTest & XCUITest:

Probably the most used iOS testing framework integrates smoothly with XCode’s testing workflow. It’s developed by Apple and therefore well supported and kept in sync with the iOS application framework.

Pros:

  • XCTest is relatively simple to integrate into an app.
  • Unit, UI, and performance testing are possible.
  • Developers can utilize their existing abilities by using Objective-C and Swift to write tests.
  • Xcode Server can execute tests automatically as a “bot” in the CI/CD process.

Cons:

  • Testers need to have developer skills because  XCode is a complex piece of software and not everybody has the knowledge of being able to write code.
  • Object-C and Swift are the only supported programming languages.
  • Not available for Android

Testing code example:

let app = XCUIApplication()
// look up element by id
let toggleButton = app.segmentedControls.buttons["toggleButton"]
// look up element by id
let dayLabel = app.staticTexts["Day: "]
let nightLabel = app.staticTexts["Night: "]
// check elements  
XCTAssertTrue(dayLabel.exists)
XCTAssertFalse(nightLabel.exists)
toggleButton.tap()
XCTAssertTrue(nightLabel.exists)
XCTAssertFalse(dayLabel.exists)

EarlGrey:

EarlGrey is a Google-developed open-source native test framework for iOS UI test automation. Google uses this framework to test their native iOS apps such as Youtube, Google Calendar, Google Photos, Google Translate, etc. It provides increased synchronization features for stable UI testing to QA teams.

Enhanced synchronization options are available with the EarlGrey framework, EarlGrey synchronizes with the user interface, network requests, and multiple queues automatically. You can still manually implement specific timings if necessary.

Tests can be written in Objective C and Swift

Pros:

  • Versatile framework with robust internal component synchronization
  • The entire framework is open source.
  • Integrates with XCode.
  • Well maintained and under active development (5.3k ⭐️ & over 1200 pull requests on github)

Cons:

  • Supporting some of the APIs may necessitate adding some additional code to your app.
  • Setup is straight forward but can be challenging for users without developer experience
  • Not available for Android
  • A lot of unresolved issues (currently 180 open issues on github)

Sample code:

import EarlGrey
import XCTest

@testable import EarlGreyExampleSwift

class EarlGreyExampleTest: XCTestCase {

  func testClickCommentBox() {
    // Select an button by accessibility ID and tap it
    EarlGrey.selectElement(with: grey_accessibilityID("PostComment"))
        .perform(grey_tap())
    // check if comment box went visible
    EarlGrey.selectElement(with: grey_accessibilityID("MyCommentBox"))
      .assert(grey_sufficientlyVisible())
    // Select element by text
    EarlGrey.selectElement(with: grey_text("Setting"))
      .perform(grey_tap())
  }
}

Repeato

Repeato is a low-code test automation framework for iOS and Android. It was designed with test stability and test engineer experience in mind. Tests can be recorded within a short amount of time without the need to code. Afterward, they can be edited and saved to a library. Tests can be run individually or in batches on Mac, Windows, or also headlessly on a CI/CD server.

Repeato uses computer vision and machine learning to observe the UI and execute the interactions just as a normal user.

Pros:

  • Extremely fast creation of test
  • Plug and play, all you need is Repeato Studio, which is set up within 5 minutes
  • Can test iOS, Android, React-Native, Flutter, Ionic, native code
  • Powerful features provided by its “Script Steps” (JavaScript or Command Line)

Cons:

  • Not open-source, freemium model
  • No device farm support (yet)

Detox:

Detox (developed by Wix) is an end-to-end iOS app testing framework that enables grey box testing to decrease the flakiness and unpredictability of black-box testing tools like Appium.

It can test apps, running them on the simulator, making it possible to observe the test runs. Another interesting detail is that  Detox is using Javascript, so you may use it with any of the many great JS test runner tools out there (Jest, Mocha, AVA,…).

Pros:

  • Detox works on iOS and Android
  • The tool’s flakiness reduces through advanced monitoring.
  • Great support with continuous integration tools.
  • Provides Javascript APIs that are straightforward to use.
  • Open Source
  • Well maintained by an active community (8.4k ⭐️ & over 1000 pull requests on github)

Cons:

  • Detox isn’t nearly as well-known or supported as the testing frameworks mentioned above.
  • It does not work on physical iOS devices (yet)

Sample code:

describe('AllAppTests', () => {
  beforeEach(async () => {
    await device.launchApp({ newInstance: true });
  });

  it('should have login screen', async () => {
    await expect(element(by.text('Login'))).toBeVisible();
    await expect(element(by.text('Username'))).toBeVisible();
    await expect(element(by.text('Password'))).toBeVisible();
  });

  it('should show error message after tap', async () => {
    await element(by.text('Login')).tap();
    await expect(element(by.text('You need to provide a username!'))).toBeVisible();
  });

});

Why Is It Important to Automate iOS Tests?

iOS test automation is critical when the goal is to create successful apps at a rapid pace. Using iOS test automation frameworks and tools is the only way to stay ahead of the competition to meet customer expectations in terms of stability and user experience.

You’ll uncover bugs faster and get faster feedback when you automate testing for iOS apps, allowing your team to stay ahead of schedule. It especially saves time when you release updates of your app frequently.

Which iOS Automation Testing Framework Is The Best?

To increase the confidence in your app, we recommend testing on different levels of your software stack. Ideally, you would do unit testing, integration testing, and end-to-end or UI testing. For apps with a backend, we can recommend API testing. But going full bulletproof is not always possible. You might decide to cover only one or two testing layers, in the end, it’s still a tradeoff between UX and costs, and each app developer team needs to find the sweet spot by themselves. Anyway, maybe this overview helps to get you going:

  • For unit testing, XCTest is the best options.
  • For UI testing, XCUITest or Repeato are the best options
  • We didn’t cover API testing tools here, but we still want to mention ApiGee and SoapUI as possible options

Choosing Which Tests to Automate

Have you thought about which use cases are at the core of your app? What are the most important use cases which must not break? This is a great place to start creating a list of flows that could potentially be automated.  We wrote a little about how a strategy for test creation could look like in one of our previous articles. 

Conclusion:

A well-thought-out iOS testing approach is critical to the development of a successful app. There are numerous iOS app testing tools available today, but selecting the appropriate one for your purposes can be difficult.

With that in mind, your product and team will determine the ideal mobile testing tool selection. Your testing tools should fit into your team rather than one that forces your team to work into it.

 

Tags: , ,

Like this article? there’s more where that came from.