Mastering Your Build to Release Pipeline
Creating Reliable Software Products with CICD Automation
Intro
In this section, we will provide a step-by-step guide on how to create a robust CICD pipeline to achieve your software development goals. Our main objective is to establish a fully automated process that covers the entire software lifecycle. This process includes developing new features in a component, integrating the code to the component base code, integrating the component with the rest of the product, validating features at the unit and system level, conducting regression testing, reporting the actual system health status, and creating a reliable release.
It is essential to consider scalability when implementing this process, especially when it occurs multiple times a day.
Reminder
In our previous article
“Release High Quality Products: Breaking down and reassembling automation to gain its full potential”
We have mapped the challenges in the field of system automation and validation in various organizations, We also have mentioned the values and the roles we should stick to, in order to overcome these challenges.
Challenges under discussion in this article
- Difficulty in maintaining sync between all product components:
This challenge arises due to the complex nature of modern software systems, which may comprise multiple interconnected components, modules, or services that work together to deliver the intended functionality. When changes are made to one component, it can impact the behavior of other components, leading to potential integration issues and unintended consequences. - Collaboration and communication:
Effective collaboration and communication among different teams involved in system validation and CI/CD processes can be challenging, particularly in large organizations or complex systems. This can include issues such as coordinating testing efforts, sharing results and findings, and ensuring that all stakeholders are informed of the system’s current state. - Timely releases and reliable products without last-minute surprises
The constant pressure to meet deadlines often leads to last-minute surprises, introducing unexpected issues that can jeopardize the product’s dependability. Achieving a harmonious blend of timely quality releases requires efficient workflows, and proactive technical problem-solving tools.
Proposed Solution
In the proposed solution we will divide the process to 4 main parts
- Build
- Validate
- Report
- Release and Deploy
Build
In this part we will build each component after a code change. This process should be implemented per each component in the product.
Each build process (pipeline) should contain the merge of the newly developed code, unit tests to avoid local regressions, feature test to make sure the newly added code works properly, sanity Integration test to make sure no 0-order regression was delivered, creating the component’s image, store it in common place (Cloud or local) with approach for the rest of the CICD systems
Validate
This process will be explained in detail in the “Focus on the Product not on the Feature” article up to come in two weeks. the article will involve different approaches and implementations to ensure that the product’s health is maintained for each component’s change in a CICD echo-system.
A small glance into the future article will help us to establish common language, the full validation flow contains:
Nightly Job: This pipeline runs nightly and includes E2E and system tests.
it is essential that this pipeline will be “Green” before we release the product.
Daily Job: This pipeline runs few times a day and focuses on integration tests for each component of the system. It includes tests that validate the interactions and integrations between different components.
PR Job: This pipeline is triggered with each pull request and includes “Gate Watcher” system tests. These tests are designed to validate the overall health of the system before merging changes into the main branch.
So in the Validate stage, we will use the products that were built in the previous step (Build) to create a fully integrated validation system-level flow.
After completing all the target features for the upcoming release and fixing all the bugs created during the development process, we will validate the product’s health monitor. Once the daily and nightly pipelines have valid results, we can proceed to the report, release and deploy phases.
Report
The report stage serves as a critical link between technical databases and organizational stakeholders, providing them with essential status on the current product’s status and defects within the system, along with their potential causes. further insights of this stage will be available in an upcoming article titled “Utilizing Big Data Methods to Enhance Automation Systems.”
Release and deploy
Upon ensuring the successful execution of all pipelines, completion of all features, and validation of a report that accurately reflects the satisfactory status of the product, it is time to progress to the release and deploy stage.
The Release and Deploy stage involves creating the product release and ensuring it is installed in all supported setups and frameworks. It is essential to validate the installer and release by using them in tests. It is important to note that the clone options typically used in the Validate stage may not accurately represent the actual usage of the system. Therefore, it is highly recommended to trigger the Release and Deploy process periodically (e.g., once a day or week) to maintain its health of the user installation flow and prevent last-minute issues.
Let’s Visualize it
Chart notes
- In the forthcoming article (“Focus on the Product not on the Feature”), I will provide a comprehensive and detailed chart that builds upon the current one, incorporating real-life examples of a specific product.
- In the chart, one important step involves validating the integrated product and then creating the release based on the validated pipeline. The subsequent stage entails installing the release on various platforms and conducting sanity installation tests.
- Another option to consider is to first create the release using stored images and then perform thorough validation on the supported platforms. This approach offers the advantage of achieving extensive coverage across the platforms you support. However, it is important to be aware of the potential drawback that any disruptions in the build and release flow could lead to the loss of all regression tests.
Golden Tips
- It is important to note that the entire Build, Validate, Report and Deploy process should be maintained under one umbrella, even if it is not in the same software project or developed by the same team. However, it is crucial that one point of contact (an Engineer, Manager, or Team) owns this process. Using more coverage and user demonstrative flows in this process will make the Deploy stage easier. Overall, this solution should help the organization stay on track with development and avoid time-consuming and demotivating debugging processes.
- This workflow should fit any type of task management process your organization uses, it can be good for “Sprints” method as well as for “Develop-Test” cycles or any other approach
- The ability to maintain, track and react quickly on defects on a daily basis during the whole development cycle will ease on the customer deploy process, will make it smoother and last 90’s minute surprises.