Django CI/CD System

--Rest of infra stack... Compute DeveloperTools CodeDeploy Storage S3GitHub Codeship image/svg+xml EC2EC2AWS RegionRemove link.Link options.Remove link.Link options.Compute Compute --Remove vertex.Remove link.Link options.Remove vertex.Remove link.Link options.Remove link.Link options.Remove link.Link options.

Diagram Overview

This diagram shows a Django CI/CD system. The system uses GitHub and Codeship to quickly deploy new software releases to infrastructure running on AWS.

AWS ELB

AWS Elastic Load Balancer (ELB) is a managed load balancing service. AWS will manage the creation and ongoing management of the service itself. This includes scaling the load balancer as traffic to the backend EC2 instances increases, handling failures should some part of the load balancer fail, and co-ordinating the distributed nature of the system as it works across Availability Zones.

How it Works
In this system, the ELB provides load balancing services to the EC2 instances. The ELB will always strive to know which instances are healthy or not so that it doesn't route traffic to unhealthy instances.

When CodeDeploy initiates a deployment to an instance, it will de-register the instance with ELB so that no traffic is sent to it during the deployment. After the instance has been upgraded it will be once again registered with the ELB.

How it's Configured
When configuring the CodeDeploy service, you can specify a Autoscaling group to use. This Autoscaling group will specify a load balancer. That's all the configuration that is needed. Once the CodeDeploy service knows which Autoscaling group to use, it will take care of the process of removing and adding back in the instances to the load balancer group.

AWS CodeDeploy

AWS CodeDeploy is a managed service that automates code deploys to one or more EC2 instances. From the service's documentation:

AWS CodeDeploy makes it easier for you to rapidly release new features, helps you avoid downtime during application deployment, and handles the complexity of updating your applications, without many of the risks associated with error-prone manual deployments. It is integrated with other AWS services such as Autoscaling and Elastic Load Balancer to help manage the entire deployment process with zero-downtime.

How it Works
In this system, Codeship will initiate the CodeDeploy deployment process. At a high level, the CodeDeploy service will instruct the service agent, installed on each EC2 instance, to download the build artifact from S3, unbundle it, and execute the deployment process. You can configure CodeDeploy to deploy to instances in-place (at a specified rate like one at a time), or in a blue/green type deployment.

You can find a detailed step-by-step description of the CodeDeployment Workflow here.

How it's Configured
You can review the AWS tutorial to learn about how to configure CodeDeploy. It is perhaps the most complex configuration of this entire system.

AWS S3

S3 is used to store the builds to deployed to the EC2 instances. Codeship will upload ready-to-deploy builds to it. And CodeDeploy will instruct each EC2 instance to download the bundle when a new deployment is initiated.

How it's Configured
On the S3 side, all you have to do is have an available bucket/key. Then, when you're configuring Codeship and CodeDeploy you will specify the bucket/key path.

Developer Machine

At the beginning of this system is the developer machine. The developer creates software to be released to a staging or production environment. While there are many different ways to trigger the deployment to production, this system is based on the GitHub Flow approach.

How it Works

To release code to an external environment, the developer does the following:

  • Download a copy of the code repository onto their machine. The repository has a 'master' and 'develop' branch
  • Create a new branch from 'develop' when writing software. Use the branch naming scheme of 'feature/xxx' or 'bug/xxx'
  • To deploy a change to staging, merge the branch into 'develop', and push to Github
  • To deploy a change to production, merge the 'develop' branch into 'master', and push to Github
  • The act of merging into 'develop' or 'master' and pushing to Github will kick off the rest of this workflow.

AWS EC2

EC2 is one of the core services in the AWS family. It provides virtual servers on demand. Whether you need Linux or Windows servers, whether they are light or heavy on resources, EC2 has an option for you.

In this configuration EC2 is acting as our web and application servers. The servers are running both nginx, and a Django uWSGI application. These servers are fronted by the Elastic Load Balancer, as you can see in the diagram.

How it Works
There is a CodeDeploy agent on every EC2 instance in the application tier. This agent is responsible for working with the overall system to coordinate new deployments. When a developer deploys a new release, the CodeDeploy service will coordinate the deployment to the instances in one of several approaches.

How it's Configured
In this configuration, CodeDeploy is set to update one instance at a time. That means that the CodeDeploy service will take one instance offline at a time to work through the lifecycle of events on that instance.

That lifecycle of events is determined by the developer. You can read more about those options here:

https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html#reference-appspec-file-structure-hooks-list

AWS EC2

EC2 is one of the core services in the AWS family. It provides virtual servers on demand. Whether you need Linux or Windows servers, whether they are light or heavy on resources, EC2 has an option for you.

In this configuration EC2 is acting as our web and application servers. The servers are running both nginx, and a Django uWSGI application. These servers are fronted by the Elastic Load Balancer, as you can see in the diagram.

How it Works
There is a CodeDeploy agent on every EC2 instance in the application tier. This agent is responsible for working with the overall system to coordinate new deployments. When a developer deploys a new release, the CodeDeploy service will coordinate the deployment to the instances in one of several approaches.

How it's Configured
In this configuration, CodeDeploy is set to update one instance at a time. That means that the CodeDeploy service will take one instance offline at a time to work through the lifecycle of events on that instance.

That lifecycle of events is determined by the developer. You can read more about those options here:

https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html#reference-appspec-file-structure-hooks-list

GitHub

GitHub is the code repository hosting service. Not only is it an excellent code hosting service, it also has many integrations into third party services. In this design, GitHub is integrated with the Codeship Continuous Integration (CI) service. This makes it extremely easy to automate test and build processes via the simple act of merging branches.

How it Works
The developer will merge their branch into the 'develop' or 'master' branches on their local machine. Then, they push the respective branch to the hosted GitHub repo. Once GitHub receives that merge, it will send a notification to the Codeship service. Codeship will then download the repository and execute its tasks (run tests, create a build artifact, deploy to AWS, etc).

How it's Configured
Codeship has a pre-built integration with GitHub. When setting up Codeship you will be directed on how to connect it to the GitHub repository.

Codeship

Codeship is a Continuous Integration (CI) service. It is used to automate the test and build processes for this system. It has pre-built integrations with GitHub and AWS CodeDeploy that make it easy to build powerful deployment pipelines.

How it Works
GitHub will send a notification to the Codeship service when anything is merged into the 'develop' or 'master' branches. Codeship will then execute a series of steps, depending on how its configured. In this setup, Codeship will perform the following actions:

  • Downloads the respective branch from GitHub
  • Start a setup virtual machine (VM) to run the application tests
  • Run any tests that have been configured
  • Prepare a build artifact (run build modification tasks, prepare a bundle)
  • Upload the build to S3
  • Initiate the AWS CodeDeploy deployment process

How it's Configured
You use Codeship by setting up projects. During the project setup process you will be instructed to perform the following steps:

  • Connect the Codeship project to the GitHub repo, and specify which branch to listen on e.g. 'master' or 'develop'
  • Specify what commands are needed to setup and run tests e.g. 'pip install -r requirements'
  • Specify what build steps should be performed
  • Configure the AWS CodeDeploy settings

Take a look at Codeship's AWS CodeDeploy blog post to learn more.