Key takeaways from " Introduction to Continuous Integration & Continuous Delivery" Course on Udemy

Sunney Sood's photo
Sunney Sood
·

5 min read

Table of contents

What Organisations want:

As Organisations move forward, the need for digital first has become a key priority of all organizations. As per Gartner, the following are the key challenges faced by today's organizations:

  • customer preferences and factors that may impact or change it.

  • regulatory and compliance requirements are constantly changing especially due to more localization demand across geographies.

  • concerns on what will happen if and when the transactions fail i.e. if the payment fails or if payment is made to the wrong account how soon can both be recovered from

  • brand loyalty is reducing and organizations need to constantly focus on digital demands, especially around time to market. This enables organizations to gain a competitive advantage.

  • gain customer trust, especially around digital privacy

Digital transformations and disruptions are still difficult for multiple organization and overcoming these internal hurdles provides a competitive advantage.

Key Challenges Organisations Faced Before DevOps

  • Development and Operations teams doing their own things. With each team having different metrics for measurement, the end goal was often conflicting with each other.

  • Slow release cycles lead to often delayed time to market. Moreover, reaction time over customer feedback was long.

  • Different environments had their own versions of the tools, frameworks and templates.

  • The scope of automation was limited to one particular department such as product, development, testing, operations, production support and customer support.

  • Limited visibility on the end-to-end workflow of taking a requirement and delivering it to production.

How DevOps Solved these problems

  • DevOps brought in a culture change where the entire pipeline from development to release was visualized and looked at from a fresh perspective.

  • If I flip the challenges of the pre-DevOps era, we have a good list of the benefits of DevOps culture.

  • Add to this, we now have a culture shift on continuous improvement and continuous delivery where time to market is cut short into days instead of months.

  • Alongside DevOps, cloud and containerization of applications brought tremendous benefits to the organization in scaling and making their infrastructure elastic.

Ideal DevOps Pipeline

The key step is to determine the current maturity model. The mentioned guide provides a quick overview and assessment of steps that can help determine the current state. Post that, below are some of the key components of the DevOps pipeline

  1. Code Repository:

    • Developers commit their code changes to a version control system (e.g., Git). The repository serves as a central source of truth for the codebase.

  2. Continuous Integration (CI):

    • Triggered by code commits, the CI server (e.g., Jenkins) pulls the latest code, builds the application, and runs automated tests. This step ensures that new changes integrate smoothly with the existing codebase and do not introduce regressions.

  3. Automated Testing:

    • At this stage we can run automated tests including unit tests, integration tests, and functional tests. This ensures the reliability and quality of the code. Test failures trigger alerts and prevent the pipeline from progressing to the next stage. Manual aka human intervention may be needed to override build failure due to automated results.

  4. Artifact Repository:

    • After successful builds and tests, the CI server stores the build artifacts (e.g., executable files, libraries) in an artifact repository (e.g., Nexus, JFrog Artifactory). These artifacts are versioned and can be used for deployment.

  5. Deployment to Staging:

    • The pipeline deploys the application to a staging environment that closely resembles the production environment. This allows for further testing, including user acceptance testing (UAT) which may be manual and performance testing (can be performed automatically and during off-peak hours when business users are not performing UAT).

  6. Automated Security Scans:

    • Security scans, including static code analysis and vulnerability assessments, are performed on the application to identify and address potential security issues early in the development process.

  7. Approval Gates:

    • Manual or automated approval gates can be implemented to ensure that only verified and authorized changes progress to the next stage. This can involve approval from stakeholders or additional automated checks. This can be done via peer reviews, senior developer review, manager reviews and other similar checks.

  8. Continuous Deployment (CD):

    • If all previous stages are successful, the pipeline automatically deploys the application to the production environment. Continuous Deployment is a key aspect of a mature DevOps pipeline, enabling rapid and reliable releases.

  9. Infrastructure as Code (IaC):

    • Infrastructure configurations are managed as code using tools like Terraform or Ansible. This ensures consistency across different environments and allows for automated provisioning and scaling of infrastructure.

  10. Monitoring and Logging:

    • Monitoring tools (e.g., Prometheus, Grafana) are integrated into the pipeline to track application and infrastructure performance. Logs are collected for troubleshooting and analysis.

  11. Continuous Feedback:

    • The pipeline provides feedback to the development and operations teams, including metrics, logs, and notifications in case of issues. This feedback loop is crucial for continuous improvement.

  12. Rollback Mechanism:

    • In case of issues in the production environment, the pipeline should have a rollback mechanism to revert to a previous, stable version of the application.

  13. Documentation and Knowledge Sharing:

    • Documentation is an integral part of the pipeline, providing information on configurations, deployment processes, and troubleshooting. This promotes knowledge sharing and ensures that the pipeline is well-understood by the entire team.

Tools Used for the Ideal DevOps Pipeline

  1. Code Repository:

    • Tool: Git (e.g., GitHub, GitLab, Bitbucket)

  2. Continuous Integration (CI):

    • Tools: Jenkins, GitLab CI, Travis CI, CircleCI

  3. Automated Testing:

    • Unit Testing: JUnit, NUnit, pytest

    • Integration Testing: Selenium, JUnit, TestNG

    • Security Scanning: SonarQube, OWASP Dependency-Check

  4. Artifact Repository:

    • Tools: Nexus, JFrog Artifactory

  5. Deployment to Staging:

    • Containerization: Docker

    • Orchestration: Kubernetes, Docker Compose

  6. Automated Security Scans:

    • Static Code Analysis: SonarQube, Checkmarx, Fortify

    • Vulnerability Scanning: OWASP ZAP, Nessus

  7. Approval Gates:

    • Tools: Jenkins, GitLab CI (manual approval steps)

  8. Continuous Deployment (CD):

    • Deployment Automation: Ansible, Chef, Puppet

    • Orchestration: Kubernetes, Docker Swarm

  9. Infrastructure as Code (IaC):

    • Tools: Terraform, AWS CloudFormation, Azure Resource Manager

  10. Monitoring and Logging:

    • Monitoring: Prometheus, Grafana, ELK Stack (Elasticsearch, Logstash, Kibana)

    • Logging: Log4j, ELK Stack

  11. Continuous Feedback:

    • Communication: Slack, Microsoft Teams

    • Metrics and Reporting: Grafana, Jenkins, GitLab CI

  12. Rollback Mechanism:

    • Version Control: Git (for rolling back code changes)

    • Deployment Rollback: Manual or automated rollback scripts

  13. Documentation and Knowledge Sharing:

    • Documentation: Confluence, GitHub Wiki

    • Collaboration: Slack, Microsoft Teams

Credits:

  • Took assistance of OpenAI's GPT-3.5 language model for research and broader topic understanding.
DevopstoolsDevOps tools