CI/CD stands for Continuous Integration and Continuous Delivery. But what does that even mean and why might you want it in your Android (or any other) app?
I like to observe it as two distinct matters, that are complementary but also independent from each other.
Continuous Integration
From an engineering perspective, what’s usually meant by CI/CD is only the CI part. It is a process of automating the integration of new code changes into the codebase. This means validating new code as it arrives, usually via Pull Requests. A successful CI helps with:
- running tests against new code,
- performing static code analysis,
- detecting bugs early,
- ensuring code quality.
Furthermore, it can prevent any unverified code from being merged into the main codebase. And it does it in an automated and transparent fashion. By having such a setup, code reviewers don’t have to interrupt their workflow by checking out a new branch just to conduct manual checks like running tests and code analysis. This way the main codebase is always releasable, which brings us to the second part of CI/CD — Continuous Delivery.
Continuous Delivery (or Deployment)
In the product context, what’s known as CI/CD is in the CD itself — a method of automating app delivery to customers. Typically, this assumes the distribution of new app versions to your clients in an automated and continuous manner. Continuous Delivery set up in your project comes with numerous benefits such as:
- enabling your clients to try and validate the changes first,
- sending new builds to QA teams, faster
- managing multiple flavors (build types) of the same app,
- automating nightly builds,
- abandoning manual processes of building and distributing artifacts,
- managing certificates and signing profiles,
- managing automatic releases on Google Play or App Store.
Which one matters more?
Most projects and startups I’ve worked with start with Continous Delivery. It enables clients and early adopters to be actively included in the software development lifecycle, just by having the latest app version at their fingertips before it’s released to a wider audience. Some will consider it as an unpopular opinion, yet I think that in the early days of projects alike, this way of engagement and transparency brings more value than Continuous Integration. CI is often taken as a premature optimization, particularly at this stage and in smaller teams.
Other projects I’ve worked on do not even have a UI or tangible outputs. These are SDKs (libraries) that other developers can include in their apps. In these environments, CI is more advantageous than CD. There may be no app to roll out, but still the library artifacts must be published somewhere, like on the Maven Central Repository — therefore CD is not to be ignored.
The Pipeline
Although CI and CD are separate processes, when taken together they’re often referred to as a CI/CD pipeline. A pipeline is a series of different steps that must be performed to integrate and deliver a new version of an app.
Engineers often use CI/CD providers to build and run a pipeline. Most notable providers include:
- Codemagic
- Github Actions
- CircleCI
- Bitrise
- Microsoft App Center (formerly HockeyApp).
- Jetbrains TeamCity
- etc.
These providers help automate testing, validation, and deployment and take care of the infrastructure, network, and security. Engineers write a CI/CD pipeline usually in a form of a YAML configuration file where they describe how exactly tests should be run, how the app should be signed, and how to send a Slack message when the build succeeds.
It starts with your Version Control System
The starting and most fundamental point of any CI/CD setup is its integration with a version control system where the source code lives, such as GitHub, Bitbucket, or GitLab. In practice, a CI/CD provider like Codemagic or Bitrise must have access to the code repository, to checkout and build the app from it, i.e. run the pipeline against the codebase.
On the other end, GitHub (or any other provider) triggers the CI/CD provider’s webhook when new code is pushed or a Pull request is opened. This means that the version control system calls an HTTP endpoint and passes a JSON payload as well, containing metadata such as the commit author information, affected git branch, number of files changed, etc. The CI/CD provider consumes that payload and decides whether it should run the pipeline or not. For instance, the pipeline may be configured to run when a Pull Request is merged to the master branch only, which makes the CI/CD provider ignore changes to other branches.
A good CI/CD provider does the webhook wiring when it first gets integrated within a VCS system. For the most part, all of the communication happens behind the scenes and does not require engineers’ attention.
It’s someone else’s computer
In a nutshell, any CI/CD provider grants you a virtual machine in the cloud, which parses that YAML file and executes commands defined in it. This comes with a lot of flexibility — you’re provided with someone else’s computer in the cloud, free to do whatever you want with it. Just like you’d do with your very own computer.
Certain CI/CD providers even allow remote access to the build machine, via SSH or VNC clients. Although this is certainly not needed in everyday CI/CD, it can come in handy in debugging and troubleshooting phases.
Do it once
Contrary to its name, building the CI/CD pipeline doesn’t have to be continuous and usually, it is not — it’s by no means an everyday task. Engineers choose a CI/CD provider according to their needs. They build a pipeline and tailor it to their needs only once. In practice, it’s all about crafting a YAML configuration file that describes what the CI/CD provider should do. Most common setups contain definitions of how the remote machine should checkout the code, build the app, execute tests and deploy it to a wider audience. That YAML configuration file resides in the Git repository alongside the codebase.
With a well-crafted YAML file in place, there’s really not much left there. Unit tests will run as Pull Requests arrive, and builds will be signed and deployed as code gets merged into a specific git branch — if it’s configured to do so. The CI/CD pipeline works under the hood, listens for hooks that trigger it, and most of the time does not require any direct attention.
About YAML
Just like JSON or XML whom you’re probably more familiar with as a mobile developer, YAML is Yet Another Markup Language — hence its original name. Yeah, it’s another data-serialization language to grasp, but the good news is that everything that can be written in XML or JSON, can easily be expressed in YAML as well.
YAML is widely used for writing configuration files in the DevOps world. It’s less verbose than XML, and more readable than JSON, while still allowing developers to precisely describe what they exactly want.
Most importantly, YAML is human-readable and easy to both read and write. Here’s what a typical YAML file looks like:
This YAML file specifically is written to be used with Codemagic, a great and intuitive CI/CD solution for mobile apps. In simple terms, this configuration file instructs Codemagic to:
- Trigger this pipeline when something is pushed to the
masterbranch. - Use a Mac Mini machine.
- Run unit tests.
- Build the app (if the previous step passes).
- Publish the app to the
#generalSlack channel. - Also, post a message there when the build starts.
- Upload the app to Firebase App Distribution and deploy it to the QA-Team.
YAML schemas
Different CI/CD providers have different what’s called YAML schemas. A great YAML schema is self-explanatory, like the one above.
Each CI/CD provider comes up with a different schema that defines what you can do and how you do it with their YAML files. A YAML schema is not the same as the YAML syntax though, which itself is always constant since it’s always the same language — YAML. In the above example, instance_type: mac_mini is how you express what machine you want the pipeline to run on. The equivalent intent in e.g. Github Actions is runs-on: macos-latest.
Luckily, you don’t have to remember schemas or memorize different YAML cheatsheets for your CI/CD provider of choice. Most modern IDEs validate not only the YAML syntax but the schema as well:
Some CI/CD providers don’t support YAML configuration at all. Instead, they provide a web UI that is usually less feature-rich but saves time and resources by kickstarting your pipeline, like this one from Microsoft’s App Center:
What to look for?
There are plenty of options out there when choosing the right CI/CD provider. Each one is different with distinct features.
Here are a few important criteria to consider:
- Setup Complexity or how many resources and efforts are needed to make a pipeline up and running. Certain providers expose robust YAML configuration options while others offer a simple no-code UI.
- Preinstalled software on a given build machine helps save time and money. This means adapting machines to the project in question. Such as bundling JDK and Gradle for Android projects, or Xcode and CocoaPods for iOS projects.
- Pricing depends on how long builds take (build minutes) and what kind of machines it runs on, operating system and hardware. Most CI/CD providers give out from 500 to 2000 monthly build minutes for free. For bigger teams, concurrent builds are an interesting feature that takes part in the price as well.
- Integration with 3rd parties like Slack, Firebase, or Google Play to name a few. For instance, chances are that your coworkers on Slack want to know when there’s a new app version. Although Slack provides an API for this, the CI/CD provider may have a no-brainer wrapper available out of the box, so that engineers don’t have to invest resources to grasp the Slack API.
The aha-moment
There are numerous CI/CD providers out there, as well as zillions of articles on CI/CD. From the average engineering perspective, they don’t make a difference per se, until you reach your very own aha-moment.
Software developers have a harder time grasping concepts that are not necessarily part of everyday development. For instance, I didn’t realize the importance of writing unit tests until the project’s codebase has grown to an extent where introducing regression bugs with every other feature was inevitable.
Without doubt, it took me time to understand why CI/CD is significant as well. Manually building and sending APK files to clients and QA always felt wrong to me.
Another personal and real-world example was a request from a QA team. They asked for a fresh app build from a specific git branch. Surprisingly, it’s a lazy Sunday morning and there is a big deadline in front of us. Without CI/CD setup, these are the usual steps I’d take:
- Get out of bed and get yourself ready.
- Make a cup of coffee.
- Open Android Studio and let Gradle do its thing.
- Checkout the branch (manually).
- Bump the version info, sign, and build the app (manually).
- Upload the APK (manually).
But since we had CI/CD in place back then, all it took me was a tap or two on AppCenter’s web dashboard to deliver a new version–even without getting out of bed! The pipeline took care of all the steps above, and more.
Happy piping!