When you start to write you first espresso UI tests it all really seems straight forward and intuitive.
- Just lookup element A via its ID
- Interact with element A
- Lookup another element B
- Check value of element B
What works great for basic demo apps, usually fails with real world apps. When we first encountered how much additional effort it is to make espresso work with real world scenarios we were baffled.
My thoughts in the first place were: “Why does espresso ask me to take care of such a basic thing? A testing framework should know when a delay is needed for the test to continue, devs should not be asked to implement this.”
I don’t want to bitch about the way Espresso is designed, it’s a great tool to have in your toolchain. But I felt that the IdlingResource is like the Achilles heel of Espresso and it’s worth writing about why:
- Complexity of your test setup is increased. Look at the documentation, it’s quite a page. You need another gradle dependency which can (and will!) eventually outdate and break. Using RX? Great there is another dependency you can add. Just figure out the right one, cause there are different versions for RX1 and RX2. Increased complexity- shouldn’t be underestimated.
- For each test you need to keep in mind which IdlingResources will be needed and register them. All of the IdlingResources need to be unregistered after you ran your test (tearDown).
- The isIdle method of the IdlingResource is called only every 5 seconds. This makes tests slower than they would need to be.
- It’s recommended to package your IdlingResources with your production code. You can add ProGuard rules to not ship your test classes with your APK, but of cause this will again take you some time.
- Often you need to modify your classes to reveal internal state of your app. While this is not a disaster, it certainly makes things more complicated when working with testing teams, exclusively responsible for writing tests. They actually should not touch your implementation and meetings might be necessary to discuss the approach.
A result of this introduced complexity is that there are devs who think it’s just not worth it and it would be wiser to spend time writing more integration and unit tests.