28 August 2023 Leave a comment Tools & Frameworks
As the mobile app market grows, so does the competition for users’ attention. With millions of options available on the app store, ensuring your app stands out from the crowd is more important than ever. Short release cycles and decent UX are key – enter mobile CI/CD tools, which can help to optimize the mobile development process. In this article, we explore the differences in some of the best mobile CI/CD tools, discussing differences in usability, performance, and pricing.
What is mobile CI/CD?
Mobile CI/CD, standing for Continuous Integration and Continuous Deployment, is an automated approach for developing and releasing mobile apps. It involves automating code integration, testing, and deployment, leading to faster and more reliable releases. This process enhances app quality, minimizes errors, and supports rapid development cycles.
In general, CI/CD stands for Continuous Integration and Continuous Deployment (or Continuous Delivery). It’s a set of practices in software development aimed at improving the efficiency, speed, and quality of delivering software updates and new features. CI/CD involves automating various stages of the software development and release process to streamline the workflow and reduce the likelihood of errors.
Mobile CI (Continuous Integration) is the automated practice of frequently integrating code changes from multiple developers into a shared repository for mobile app development. This process ensures that new code is regularly tested and merged, preventing integration issues.
Mobile CD (Continuous Deployment/Delivery) involves automating the deployment of mobile app changes after passing automated tests. Continuous Delivery typically requires manual approval for deployment, while Continuous Deployment automatically deploys code changes to production environments once tests pass, facilitating quicker and more consistent releases.
Mobile CI/CD tools offer several benefits:
- Faster Development: Automated testing and deployment reduce manual intervention, accelerating the development cycle.
- Reduced Errors: Automated tests catch issues early, preventing bugs from reaching production.
- Consistency: The same process is applied to every code change, reducing the risk of configuration errors.
- Rapid Feedback: Developers receive quick feedback on code changes, helping them make improvements sooner.
- Reliability: Automated deployments reduce the risk of human error during the deployment process.
- Scalability: The automated nature of CI/CD supports rapid scaling and handling frequent code changes.
Mobile CI/CD is often implemented using various tools and services that help automate the building, testing, and deployment processes.
Reviewing the best mobile CI/CD tools
This piece has been created while evaluating some of the most commonly used services on the market as hosts for our mobile app testing solution Repeato. We tested with a focus on mobile apps and evaluated our Flutter app test automation, only on services that could be used to run Android and iOS UI tests.
Bitrise
Bitrise is a CI/CD service dedicated to mobile apps. It helps developers automate the process of building, testing and deploying their apps. Bitrise is easy to use and set up, and it integrates with popular development tools such as GitHub, GitLab, and Azure DevOps.
Pros
- Ease of use – Very easy to configure different workflows via UI (Supports .yaml file configuration as well)
- Supports several Git tools integrations (Github/Bitbucket/GitLab)
- Free for individuals and students
- Supports hardware iOS devices
- A lot of plugins/steps are readily available for integration
- Supports self-hosted runners
- Supportive community and team
Cons
- Documentation should be more extensive to help with debugging
- The latest artifacts cannot be seen on the dashboard
- Slower builds
- No self-hosted runners
CodeMagic
We found that CodeMagic was the best mobile CI/CD tool for Flutter apps. We were able to easily set up a continuous integration and delivery pipeline for our Flutter apps within 1 hour. It is easy to use and has a great user interface. It also has a free plan which is perfect for small projects.
Pros
- Super easy-to-use pipelines
- Supports several Git tools integrations (Github/Bitbucket/GitLab)
- Free/Freemium Version with limited 500 minutes build time
- Emulator and other Flutter apps dependencies come pre-installed
- Built-in chat messenger for support
- Pay-as-you-go option
- Share build Artifacts directly with someone using a link
- Build time was faster compared to Bitrise, as many flutter-specific things were pre-installed on VMs
Cons
- Only M1 machines are available for trial
- The emulator could not be started on M1 Machines
- Only supports flutter apps testing and native IOS & Android Apps
TeamCity
TeamCity is a powerful and user-friendly mobile CI/CD tool from JetBrains. We believe that it’s a great choice for bigger organizations looking for a versatile CI/CD solution. not just for mobile app testing but also for other platforms. It is feature-rich and can be easily customized to meet the needs.
Pros
- Supports several Git tools integrations (Github/Bitbucket/GitLab)
- Supports self-hosted runners
- APIs available, use APIs for building plugins
- Build time is super fast compared to the other services
- Great support
- Can be connected with all other JetBrain tools
- Free for small teams
Cons
- Documentation is poor specifically for mobile apps testing
- Not suitable enough for mobile apps testing lacks documentation and requires a lot of configs
- Needs a lot of configurations even for a small flutter project like ours
- UI is not as user-friendly compared to the other services. For example, reporting is not great, and reading logs could be easier
Gitlab
The GitLab CI/CD service is a part of GitLab that builds and tests software when developers push code to their repository and then deploys any code changes to production.
Pros
- Free up to 400 CI/CD minutes
- Supports self-hosted runners
- Supportive community
- Documentation is good
- Great community
- More affordable pricing
Cons
- Only supports .yaml file configuration
- Requires credit card for shared runners
- Doesn’t support other Git tool integrations
GitHub Actions
Compared to all others, GitHub Actions seem to provide the most versatile collection of predefined steps (called “Actions” of cause). No matter which language you use, which other services you need to integrate into your build flow, there is most probably a predefined Github Action that got you covered. That’s a big deal. However, there are also downsides to using GitHub Actions.
Pros
- Supports self-hosted runners
- Supports almost every language so every project can be integrated into GitHub actions CI/CD
- Free upto 2,000 minutes per month
- A lot of actions are available in the Github marketplace for free (Deploy to Azure, S3 – Notification to Slack, etc…)
- Great community
- Documentation for Github actions is good
- Live logs to see the workflow in real-time
- Multi-container testing using docker-compose
- Affordable pricing
Cons
- Only supports .yaml file configuration
- Due to Github performance issues, some tests of ours timed out – very annoying, because not reliable
- Build time was also slower
- Doesn’t support other Git tools integrations
Key take aways
Incorporating an efficient mobile CI/CD approach is pivotal in addressing modern development challenges. By automating integration, testing, and deployment, mobile CI/CD mitigates the complexities of app development, fostering a seamless and collaborative environment. This results in faster feedback cycles, early bug detection, and heightened overall app quality.
Dedicated mobile CI/CD tools can help getting this process done quickly. While challenges like configuration intricacies may arise, the benefits of reduced human errors, consistent releases, and scalability eclipse these hurdles.
Looking ahead, the evolution of mobile CI/CD solutions will likely continue to optimize development pipelines, supporting the rapid evolution of mobile apps in an ever-changing technological landscape.
Disclaimer: This article has been published in autumn 2023; since then, the software solutions above might have been updated with new features, have been restructured regarding pricing or other changes that might have impact on this evaluation. For more information, please visit the software’s own websites.