Idling resources in Espresso: How they work and problems

28 July 2019 stoefln Leave a comment Espresso

Idling resources are a way to tell Espresso to wait till the background process has finished. In this article we look at how idling resources work. Also we solve some common problems of Espresso idling resources.

Why do we need idling resources?

When Espresso tests are executed by the test runner, the timing of actions is crucial. If the test runner tries to press a button before the button is shown or active, it might trigger an error unintentionally. So the straight forward way to fix this issue is to add a lot of delay (Thread.sleep()) commands into your code. The downside is: We are wasting precious time which can accumulate to minutes and even hours for bigger test batches.

Unfortunately Espresso is not aware of background tasks (such as loading web content, playing an animation, background processing) and therefore can’t know when exactly to trigger the next action.

So Idling Resources are a way to tell Espresso to wait till the background process has finished.

While there are objections about using idling resources, currently it is the standard way to go about it.

How do idling resources work in Espresso?

IdlingResource is a class which has to be created by the test engineer. It contains a logical condition which must be fulfilled for the application to reach the inactive state. After creating an idling resource object, it must be registered with IdlingResourceRegistry to be noticed by Espresso. From Espresso’s point of view, IdlingResource is a representation of the dynamic resource of the Android application under test. It can be any operation performed by the application, for example REST requests, database operations, animations, loading data into the list and more.

How to implement idling resources in Espresso?

  1. We need to configure Espresso according to the documentation. IdlingResource is not included in the base library, it is an extension.
  2. The class that implements the IdlingResource interface must be created.
  3. The IdlingResource interface contains three abstract methods which implementation must be provided for IdlingResource to function properly: getName(), isIdleNow() and registerIdleTransitionCallback(ResourceCallback callback).
  4. The getName() method cannot be null and can return any string. It is used by IdlingResourceRegistry to make sure that each IdlingResource is unique.
  5. Create a local ResourceCallback field in IdlingResource and replace registerIdleTransitionCallback(ResourceCallback callback). This method is used by IdlingResourceRegistry when registering the IdlingResource. IdlingResourceRegistry will send an instance of ResourceCallback to the IdlingResource, which will connect it to the entire application. This ResourceCallback received must be saved in the local field previously created.
  6. The isIdleNow() method is the kernel and the most important part of IdlingResource. It gives IdlingResourceRegistry information about the current state of the condition indicated in IdlingResource. It must contain a logical expression that changes the value from false to true, indicating that the asynchronous operation has ended now.

A line of code is worth more than thousand words

Here is an example. This idling resource will make Espresso wait till the ContentFragment was pushed, and therefore the test can continue.

<span class="token keyword">public</span> <span class="token keyword">class</span> <span class="token class-name">ContentLoadingIdlingResource</span> <span class="token keyword">implements</span> <span class="token class-name">IdlingResource</span> <span class="token punctuation">{</span>
    <span class="token keyword">private</span> <span class="token class-name">ResourceCallback</span> resourceCallback<span class="token punctuation">;</span>
    <span class="token keyword">private</span> <span class="token keyword">boolean</span> isIdle<span class="token punctuation">;</span>

    <span class="token annotation punctuation">@Override</span>
    <span class="token keyword">public</span> <span class="token class-name">String</span> <span class="token function">getName</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
        <span class="token keyword">return</span> <span class="token class-name">ContentLoadingIdlingResource</span><span class="token punctuation">.</span><span class="token keyword">class</span><span class="token punctuation">.</span><span class="token function">getName</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token punctuation">}</span>

    <span class="token annotation punctuation">@Override</span>
    <span class="token keyword">public</span> <span class="token keyword">boolean</span> <span class="token function">isIdleNow</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
        <span class="token keyword">if</span> <span class="token punctuation">(</span>isIdle<span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token boolean">true</span><span class="token punctuation">;</span>
        <span class="token keyword">if</span> <span class="token punctuation">(</span><span class="token function">getCurrentActivity</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token operator">==</span> <span class="token keyword">null</span><span class="token punctuation">)</span> <span class="token keyword">return</span> <span class="token boolean">false</span><span class="token punctuation">;</span>

        <span class="token class-name">ContentFragment</span> f <span class="token operator">=</span> <span class="token punctuation">(</span><span class="token class-name">ContentFragment</span><span class="token punctuation">)</span> <span class="token function">getCurrentActivity</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getFragmentManager</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">findFragmentByTag</span><span class="token punctuation">(</span><span class="token class-name">ContentFragment</span><span class="token punctuation">.</span>TAG<span class="token punctuation">)</span><span class="token punctuation">;</span>
            
        isIdle <span class="token operator">=</span> f <span class="token operator">==</span> <span class="token keyword">null</span><span class="token punctuation">;</span>
        <span class="token keyword">if</span> <span class="token punctuation">(</span>isIdle<span class="token punctuation">)</span> <span class="token punctuation">{</span>
            resourceCallback<span class="token punctuation">.</span><span class="token function">onTransitionToIdle</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
        <span class="token punctuation">}</span>
        <span class="token keyword">return</span> isIdle<span class="token punctuation">;</span>
    <span class="token punctuation">}</span>

    <span class="token keyword">public</span> <span class="token class-name">Activity</span> <span class="token function">getCurrentActivity</span><span class="token punctuation">(</span><span class="token punctuation">)</span> <span class="token punctuation">{</span>
        <span class="token keyword">return</span> <span class="token punctuation">(</span><span class="token punctuation">(</span><span class="token class-name">TestApplication</span><span class="token punctuation">)</span> <span class="token class-name">InstrumentationRegistry</span><span class="token punctuation">.</span>
            <span class="token function">getTargetContext</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getApplicationContext</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">)</span><span class="token punctuation">.</span><span class="token function">getCurrentActivity</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">;</span>
    <span class="token punctuation">}</span>

    <span class="token annotation punctuation">@Override</span>
    <span class="token keyword">public</span> <span class="token keyword">void</span> <span class="token function">registerIdleTransitionCallback</span><span class="token punctuation">(</span>
            <span class="token class-name">ResourceCallback</span> resourceCallback<span class="token punctuation">)</span> <span class="token punctuation">{</span>
        <span class="token keyword">this</span><span class="token punctuation">.</span>resourceCallback <span class="token operator">=</span> resourceCallback<span class="token punctuation">;</span>
    <span class="token punctuation">}</span>
<span class="token punctuation">}</span>
Java

Registering the idling resource

The IdlingResource created must be added to IdlingResourceRegistry via the registerResources() method (static method of the Espresso class). After that, it will be taken into account the next time you use the perform() or check() methods where loopMainThreadUntilIdle() will occur. The isIdleNow() method will be called with a fixed interval until the transition to inactivity is signaled by all registered resources or the delay is lifted.

Once the saved Idling resources have completed and the application reaches inactive status, remember that they are still in the list of IdlingResourceRegistry, which is a singleton shared among all test cases. Therefore, remember to call unregisterResources() (static method of the Espresso class) when they are no longer needed to avoid malfunctions.

The problem with Espresso idling resources

When you write your first Espresso UI tests, everything seems straight forward and intuitive.

  1. Just lookup element A via its ID
  2. Interact with element A
  3. Lookup another element B
  4. Check value of element B

What works great for basic demo apps, usually fails with real world apps. When we first encountered how much additional effort it is to make Espresso work with real world scenarios we were baffled.

Our thoughts in the first place were: “Why does Espresso ask us to take care of such a basic thing? A testing framework should know when a delay is needed for the test to continue, devs should not be asked to implement this.”

We don’t want to complain about the way Espresso is designed, it’s a great tool to have in your toolchain. But we felt that the IdlingResource is like the Achilles heel of Espresso and it’s worth writing about why.

The main problems of Espresso idling resources are:

  1. Complexity of your test setup is increased. Look at the documentation, it’s quite a page. You need another gradle dependency which can (and will!) eventually outdate and break. Using RX? Great there is another dependency you can add. Just figure out the right one, cause there are different versions for RX1 and RX2. Increased complexity- shouldn’t be underestimated.
  2. For each test you need to keep in mind which IdlingResources will be needed and register them. All of the IdlingResources need to be unregistered after you ran your test (tearDown).
  3. The isIdle method of the IdlingResource is called only every 5 seconds. This makes tests slower than they would need to be.
  4. It’s recommended to package your IdlingResources with your production code. You can add ProGuard rules to not ship your test classes with your APK, but of cause this will again take you some time.
  5. Often you need to modify your classes to reveal internal state of your app. While this is not a disaster, it certainly makes things more complicated when working with testing teams, exclusively responsible for writing tests. They actually should not touch your implementation and meetings might be necessary to discuss the approach.

A result of this introduced complexity is that there are devs who think it’s just not worth it and it would be wiser to spend time writing more integration and unit tests.

Tags: , ,

Like this article? there’s more where that came from.

Leave a Reply