Setting up a simple CICD pipeline with AWS CodePipeline

Posted bySimón González GuerraPosted inUncategorizedTags:, , , , , ,
Photo by Darya Jum on Unsplash

Goal

Our objective is to build an tool that can build a given application on the cloud, producing an artifact, and deploy it to an EC2 instance on AWS. For this tutorial, the application is a Java program stored in a Github repo and built as a Maven project that produces a jar file. The AWS instance we will deploy to is a preexisting EC2 instance. Our automation will be triggered by our commits, so that every time we commit to the master branch of our repo, it will automatically build the application and deploy it to our instance, following the principles of continuous integration and continuous deployment (CICD).

What you’ll need

  • A Github repo containing a Java Spring web application with standard maven structure. The repo doesn’t need to be public.
  • A working AWS account.
  • An EC2 instance in that AWS account, running Linux.
  • An S3 bucket in that AWS account, to use as artifact repository and to store logs.
  • All resources used in this tutorial are available in the AWS Free Tier, so if the account is less than a year old and you only have one EC2 instance running at any given time, there’s no cost involved.

Architecture overview

The following diagram illustrates the components we will use to solve this problem and how they interact with one another.

As we can see, our components are organized into four groups:

  • Code repository: a version control system that stores our code. In this tutorial, this will be a Github repo.
  • Continuous Integration automation: a tool that runs on the cloud and builds the source code, generating an artifact. In this tutorial, our continuous integration automation will be an AWS Code Pipeline. We will refer to this as Continuous Integration Pipeline or CI Pipeline for short.
  • Artifact repository: a server that stores the generated artifact and allows to retrieve it later. In this tutorial, our artifact repository will be an AWS S3 bucket.
  • Continuous Deployment automation: a tool that runs on the cloud and deploys the artifact generated earlier to the target host and puts it to run. In this tutorial, our continuous deployment automation will be another AWS Code Pipeline. We will refer to this as Continuous Deployment Pipeline or CD Pipeline for short.

A key tool here is AWS CodePipeline. A code pipeline is a predefined process made of stages, which are in turn composed of actions. Each stage executes a step of the process (e.g. downloading code from a repo) and can produce as output any number of files, called artifacts. Artifacts are produced as output in a stage and can the be used as input in the stages that follow. When we create a code pipeline in AWS, AWS automatically creates an S3 bucket that the pipeline will use to store its internal artifacts so they can be shared between its stages. Note this is not the same as our artifact bucket. Whereas the code pipeline’s internal bucket is created automatically and used to share artifacts between stages of the same code pipeline, our artifact bucket is separate from the code pipelines, will be created manually and used to share an artifact between the two pipelines.

Our CI pipeline will be composed of the following stages:

  • Source stage: downloads the code from our Github repo. This stage produces a zip file with the contents of our repo as its output artifact.
  • Build stage: builds the code, generating an artifact. This stage uses as input the output artifact of the source stage. The output of this stage is our application artifact, which is a zip containing the jar file of our application and some metadata.
  • Upload stage: uploads the application artifact generated in the build stage to an artifact repository. In this case, the artifact repository is an S3 bucket.

Our CD pipeline will be composed of the following stages:

  • Source stage: downloads the application artifact from the artifact repository.
  • Deploy stage: copies the application artifact to a target EC2 instance and starts the application on it (previously stopping the process if it was already running).

Repository directory structure

For this tutorial, we will use the following Github repo as an example: aws-cicd-poc. The directory structure of this repo can be seen in the following picture:

As we can see, the repo contains:

  • A folder called basicsprintbootwebapplication, which contains the skeleton code of a Spring Boot application generated using Spring Initializr.
  • A folder called cicd, which contains the following files:
    • buildspec.yml: we will use this file when creating the CI pipeline. More details below.
    • appspec.yml: we will use this file when creating the CD pipeline. More details below.
    • start_server.sh: a script we will use to start the application inside the EC2 instance.
    • stop_server.sh: a script we will use to stop the application inside the EC2 instance.
    • Note: In your repo, you can place these files wherever you want. When you create the pipelines, you’ll tell AWS the paths to find them in your repo.

Buildspec file

The buidlspec is a yaml configuration file whose purpose is to tell the CI pipeline how to map from the files in your repo to the files you want to include in the application artifact. In this tutorial, we are using a buildspec file that contains the following:

version: 0.2

phases:
  build:
    commands:
       - cd basicspringbootwebapplication && mvn package
artifacts:
  files:
    - basicspringbootwebapplication/target/basicspringbootwebapplication-0.0.1-SNAPSHOT.jar
    - basicspringbootwebapplication/cicd/aws/ec2/scripts/start_server.sh
    - basicspringbootwebapplication/cicd/aws/ec2/scripts/stop_server.sh
    - basicspringbootwebapplication/cicd/aws/ec2/appspec.yml
  discard-paths: yes

Key takeaways:

  • The commands section tells CodePipeline what are the commands required to build the project once the source code is downloaded. Our repo is a standard maven project, so we just move to the project directory and call mvn package, which generates a jar file in the target directory.
  • The files section tells CodePipeline which files from the repo we want to include in the output artifact. In our case, we include the jar of our application, the start and stop scripts, and the appspec.yml file. This appspec.yml file will be required for the deployment stage later on.
  • The discard-paths: yes option tells AWS to leave all these files in the root folder of the output artifact. You can choose to create a directory structure in the artifact if you whish, and the buildspec allows you to indicate where to put each file in the directory structure (see this page for the details of how to do this). If you choose to do this, keep in mind that the appspec file must always be in the root of the artifact in order to be able to deploy it using CodeDeploy (more details about this below).
  • Our application artifact will be a zip file with the following content:
    • basicspringbootwebapplication-0.0.1-SNAPSHOT.jar
    • start_server.sh
    • stop_server.sh
    • appspec.yml

CI pipeline

To create the CI pipeline, follow these steps:

  1. In the AWS console, go to Services / Developer Tools / CodePipeline / Create pipeline.
  2. In Pipeline name, enter a suitable name for your pipeline. In this tutorial, we will use set the name basicspringbootwebapplication-ci-codepipeline.
  3. In Service role, select New service role. This is the role that will be used when CodePipeline runs.
  4. In Role name, enter a suitable name. Make sure the name is at most 64 characters long, otherwise AWS may let you move forward with the process but show an obscure error at the end when it tries to create the code pipeline.
  5. Click Next.
  6. In Source provider, select Github.
  7. Click on Connect to Github.
  8. A dialog will pop up where you’ll need to enable AWS to access your Github account. This is required in order for the CI pipeline to be able to download the code from your repo.
  9. In Repository, select which of the repos in your account you want to use.
  10. In Branch, enter the name of the branch you want to build from. This is usually master.
  11. Leave the rest to defaults and click Next.
  12. In Build Provider, select AWS CodeBuild.
  13. Click on Create project. This will open a new window with a dialog where we will create a CodeBuild project to tell AWS how to find our buildspec file in the repo. Once we are done there we will return to the CodePipeline wizard. The following steps describe how to set up the CodeBuild project.
  14. In Project Name, enter a suitable name for the CodeBuild project. In this tutorial, we will set the name basicspringbootwebapplication-CodeBuildProject.
  15. In Environment / Operating System, select Ubuntu.
  16. In Runtime, select Standard.
  17. In Image, select the latest version number available.
  18. In Service role, select New service role. This is the role with which AWS will run when building.
  19. In Role name, enter a suitable name. Make sure the name is at most 64 characters long, otherwise AWS may let you move forward with the process but show an obscure error at the end when it goes to create the CodeBuild project.
  20. In Buildspec, select Use a buildspec file.
  21. In buildspec name, enter the path to the buidlspec.yml file relative to the root if your repo. In our example repo, this is basicspringbootwebapplication/cicd/aws/ec2/buildspec.yml.
  22. In Logs, you choose where to store the build logs. You can use CouldWatch, S3, or both. For this tutorial, we will uncheck CloudWatch and check S3. In Bucket, select the S3 bucket you want to use to store the logs. In Path prefix, enter a suitable filename prefix for your build logs, such as basicspringbootwebapplication.
  23. Click on Continue to CodePipeline to return to the CodePipeline wizard.
  24. Leave the rest to its defaults and click Next.
  25. In Add deploy stage, click Skip deploy stage.
  26. Click on Create pipeline.
  27. Now our code pipeline has been created. Open your pipeline in the pipeline list. It has two stages, Source and Build. We are still missing a stage, the Upload stage, where we take the generated artifact and store it in our artifact repository.
  28. Click on Edit, and then click Add stage at the bottom. Note it’s important to click the last Add stage button, so that we add the stage at the bottom.
  29. In Stage name enter Upload, and click Add stage.
  30. In the Upload stage, click on Add action group to add an action.
  31. In Action name, enter Upload.
  32. In Action provider, enter Amazon S3.
  33. In input artifact, select Build artifact. This tells AWs that the artifact we want to upload is the one generated previously in the build stage.
  34. In Bucket, select the S3 bucket you want to use as artifact repository.
  35. In S3 object key, enter the path where you want to store the artifact in the bucket. Take note of the path you enter here, you will need it later. In this tutorial we will use artifacts/basicspringbootwebapplication/snapshot/basicspringbootwebapplication-SNAPSHOT.zip.
  36. Leave the rest to its defaults and click Done.
  37. Click Save to save your stage addition to the pipeline.
  38. You are all set! Click on Release change to run the CI pipeline manually. If it runs without errors, you should have the build artifact stored in the S3 bucket in the path you chose.

Note: if you open the CodeBuild project in the AWS console, you’ll note that it has the source field set to CodePipeline. This means that the input artifact of the build stage will the output artifact generated in the previous source stage (a zip file containing the content of your Github repo) which is what we want. The CodeBuild project is set up this way because we created it from inside the CodePipeline wizard, which automatically sets the source field to CodePipeline. This is the only way to set the source field with that value. If you create the CodeBuild project from the CodeBuild wizard, the source field won’t have CodePipeline in its available options.

The CI pipeline created this way will run every time you push a commit to the master branch (or whatever branch you specified in the CodeBuild project). You can also run it manually any time you want by clicking on Release change. Every pipeline execution will generate a new application artifact and store it in the S3 bucket, overwriting the previous artifact.

Troubleshooting tips

If a stage of the pipeline fails when executing, CodePipeline will stop the pipeline at that point and will not run the stages that come after it. To find out what happened, click on Details for the stage that failed. This will take you to a page that is specific for the AWS service the stage is based on. Some examples:

  • If the stage that failed is CodeBuild, the details page has a Logs tab that lets you download a log file of the build, where you can check why it failed.
  • If the stage that failed is CodeDeploy, the details page has a View events link where you can see the steps f the deployment and check where it failed.

CD pipeline

When it comes to deploying the application artifact to the infrastructure where it will run, there are several approaches, depending on how the architecture of the system. Some possible approaches are:

  1. Copying the artifact to an existing EC2 instance and run it there. This can be done using a tool like AWS CodeDeploy.
  2. Creating a new EC2 instance for the application to run on. This can be done using a tool like AWS CloudFormation.
  3. Creating a container for the application to run on. This can be done using a tool like Docker, and the creation of the container within a cluster can be orchestrated using tools like Kubernetes and AWS EKS.

In this tutorial, we will use approach 1. The purpose of our CD pipeline is to copy the artifact to the instance, stop the running process (in case it was already running) and start a new process from the new binary.

When we add CodeDeploy as a stage in a code pipeline, the stage will carry out the following steps, in order:

  1. Find the set of instances to deploy to, based on a criteria that we will specify (in this tutorial, there is only one instance, but this is not required by CodeDeploy).
  2. Copy the artifact to the instance(s).
  3. Extract the artifact inside the instance(s) and copy the files to their target locations in the file system, based on a mapping that we will specify.
  4. Stop the application if it is running, by calling a script that we will provide.
  5. Start the application from the new binary, by calling a script that we will provide.

The following sections go over the steps we will need in order to set up the CD pipeline.

In order to deploy an application with CodeDeploy, we must specify three things:

  1. The application artifact to deploy (also known in AWS as a revision). This artifact must contain a metadata file called appspec.yml which AWS will read to know how to work with the artifact (more details about this below). We will specify the revision by providing the S3 bucket where our revision is stored.
  2. The deployment configuration we will use. This means indicating to AWS to how many instances your application revisions should be simultaneously deployed and to describe the success and failure conditions for the deployment.
  3. The set of instances to deploy to. This is specified in what AWS calls a deployment group, which is a set of rules that defines the set of instances to deploy to.

We will specify these three elements by creating a CodeDeploy application. In CodeDeploy, an application is a name that functions as a container to ensure that the correct combination of revision, deployment configuration, and deployment group are referenced during a deployment.

Appspec file

In order for an application to be deployed using CodeDeploy, the application artifact must contain a file named appspec.yml in the root of the artifact. Note you can lay out the rest of the files in the artifact however you want, but the appspec must always be in the root. The function of the appspec file is to tell CodeDeploy how to map from the files in your artifact to the files that must be deployed in the instance. At at minimum, this involves indicating the path to our executable and how to start and stop the application.

In this tutorial, our appspec looks like this:

version: 0.0
os: linux
files:
  - source: basicspringbootwebapplication-0.0.1-SNAPSHOT.jar
    destination: /basicspringbootwebapplication/
hooks:
  ApplicationStart:
    - location: start_server.sh
      timeout: 300
  ApplicationStop:
    - location: stop_server.sh
      timeout: 300

Key takeaways:

  • The files element contains a list of files from the artifact that we want to copy to the instance. In our case, it’s just one, the jar file. The source field indicates the path to the file in the artifact relative to its root. The destination field indicates the parent folder you want the file to be copied under (note it’s not the path, it’s the parent folder).
  • The ApplicationStart hook tells CodeDeploy where to find the script to start the application relative to the root of the artifact.
  • The ApplicationStop hook tells CodeDeploy where to find the script to stop the application relative to the root of the artifact.
  • In this case, we are not copying the scripts to any other location within the instance, we will just run them from the directory where the artifact was extracted, which is enough for our purposes. If we wanted to copy the scripts and run them from another location, we would need to include it in the files element.
  • The application jar fle does need to be copied, so that we have a clear location to reference it from the start script (more details about this below).

CodeDeploy agent

Remember we are going to use AWS CodeDeploy to deploy our application to an EC2 instance. In order to be able to do this, CodeDeploy requires that the instance is running the CodeDeploy agent. The agent is a service process that runs in the instance and acts as the interface between the instance and CodeDeploy. Installing the CodeDeploy agent only needs to be done once per instance. You can find instructions on how to install the agent on this page.

Tagging the instance

CodeDeploy determines the set of instances it will deploy to based on a condition specified in a deployment group. The set of all instances that meet this condition is the set CodeDeploy will deploy to. The condition can be expressed in a number of ways. In this tutorial we will select based on the existence of a tag in the instance, so the deployment set is the set of instances that have that tag (which in this tutorial is composed of only one instance). In order for this to work, we need to add the tag to the instance we want to deploy to. This only needs to be done once per instance.

  1. In the AWS console, go to Services / EC2.
  2. Select Instances in the side panel.
  3. Look for the instance you want to deploy to and click on it.
  4. Go to the Tags tab and click on Manage tags.
  5. Add the following tag:
    1. Key: basicspringbootwebapplication
    2. Value: (leave empty)

You can use whatever string you like as key as long as it’s the same string you input later on when you create the deployment group (more details about this below).

Start script

This is what our example start script looks like:

#! /bin/bash

SERVICE_HOST=localhost
SERVICE_PORT=8080
STARTUP_WAIT_TIME=10
APPLICATION_JAR_PATH=/basicspringbootwebapplication/basicspringbootwebapplication-0.0.1-SNAPSHOT.jar
LOG_FILE_PATH=/basicspringbootwebapplication/general.log
APPLICATION_MAIN_CLASS_NAME=com.sgonzalez.basicspringbootwebapplication.App

echo "Starting application..."

nohup java -jar $APPLICATION_JAR_PATH $APPLICATION_MAIN_CLASS_NAME > $LOG_FILE_PATH 2>&1 &
sleep $STARTUP_WAIT_TIME
nc -z $SERVICE_HOST $SERVICE_PORT

if [ ! $? -eq 0 ]; then
  echo "Could not start application"
  exit 1
fi

echo "Application started OK"

Key takeaways:

  • An & at the end makes the application run in the background.
  • Starting the process with nohup lets the java process run in the background even after the user is logged out.
  • Ten seconds after starting the process we check if it’running by attempting a connection to local port 8080 with nc -z and if this fails we fail the script. This is important so that if the application fails to start for whatever reason, CodeDeploy will fail the stage and CodePipeline will let us know that there was an error.

Stop script and how to do graceful shutdown with the Spring Boot Actuator module

Every time we deploy a new version of the application to our instance, CodeDeploy will call our stop script which is expected to do a graceful shutdown of the process. In this tutorial, we will use the Spring Boot Actuator module to do the graceful shutdown. Actuator is a module that runs inside the application and exposes a URL (/actuator/shutdown) that shuts down the process when we send an HTTP POST to it. There are two approaches regarding how to call this URL:

  1. HTTP (unsecured request). Any HTTP POST request to the URL is accepted. This is a security concern since an attacker that can get a request through to this URL can kill your application. This is simple and useful for development purposes, but it’s not suitable for a production system.
  2. HTTPS (secured request). The request is only acted upon if it carries correct authentication information. This is the best approach for a production system. It requires some extra set up steps:
    1. The application must have credentials defined for the URL, which should be stored in some credential manager service like AWS Secrets Manager or Hashicorp Vault.
    2. The stops script must retrieve these credentials from said credential manager service and send them in the request to the shutdown URL.

For the sake of simplicity, in this tutorial we will use approach 1. In order to do this, we include the following in the application.properties file:

# In order to access the actuator endpoints using HTTP, we need to both enable and expose them. Here, we expose all endpoints.
management.endpoints.web.exposure.include=*

# Explicitly enable the shutdown endpoint
management.endpoint.shutdown.enabled=true
endpoints.shutdown.enabled=true
endpoints.shutdown.sensitive=false

In the stop script, we call the Spring /actuator/shutdown endpoint over HTTP. The following is an example of a viable stop script:

#! /bin/bash

SERVICE_HOST=localhost
SERVICE_PORT=8080

pid=`ps aux | grep -i <my_app_name> | grep -v grep | awk '{ print $2 }'`
if [ ! $pid ]; then
  echo "Process not runing, nothing to do"
  exit 0
fi

echo "Sending shutdown request to service..."
curl -X POST http://$SERVICE_HOST:$SERVICE_PORT/actuator/shutdown
curl_result=$?
echo -ne "\n"

if [ ! $curl_result -eq 0 ]; then
  echo "Could not do the graceful shutdown of the service. This may mean that we couldn't send the shutdown request. We will kill the process anyway"
  echo "Curl exit code: $curl_result"
fi

kill $pid
kill_result=$?
if [ ! $kill_result -eq 0 ]; then
  echo "Could not kill process. You should kill it manually"
  echo "Kill exit code: $kill_result"
  exit 1
fi

echo "Service stopped"

Key takeaways:

  • The configuration above enables and exposes the shutdown URL.
  • Note there’s nothing in the configuration that sets the shutdown URL as unsecured. It is unsecured because we intentionally left out the Spring Security module in the pom file, so all URLs are unsecured by default. If we add Spring security later, the shutdown URL will be secured by default and the above command will fail with 401. In that case we’ll need to either explicitly configure the shutdown endpoint as unsecured, or have the stop script pass the credentials.

Enabling versioning on the artifact bucket

In order to use an S3 bucket as source for a deployment, the bucket must be versioned. To enable this, navigate to the bucket we are using as artifact repository in the AWS console, open the properties tab, go to Versioning and select “Enable versioning”.

Installing appropriate version of the Java JRE on the instance

If your application’s pom.xml is configured to build with the 1.8 version of the JDK or above, you’ll need to install the 1.8 version of the JRE or above on the instance in order to be able to run it.

Follow these steps to install the Java 1.8 JRE on an Amazon Linux 1 EC2 instance:

$ sudo yum install java-1.8.0-openjdk
$ sudo alternatives --config java # Follow the instructions prompted and set java 1.8 as the alternative to use

Note: the purpose of the alternatives command is to configure the system so that the command java will run the java 1.8 binary. The command above is setting /usr/bin/java as a symbolic link to /etc/alternatives/java, which in turn will point to /usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java which is the actual binary we want to call. We are using Linux alternatives system here.
Without using the alternatives system, you can get the same effect by just adding the directory where the JDK 1.8 is installed (/usr/lib/jvm/jre-1.8.0-openjdk.x86_64/bin/java) to the beginning of the PATH, so that it comes before /usr/bin. In these instructions we have used the alternatives system instead. Note though the alternatives system requires configuring each command individually, whereas using PATH overrides all JDK commands to point to whatever directory is found first in the environment variable.

Creating the CD pipeline

Follow these steps to create the CD pipeline:

  1. In the AWS console, go to Services / IAM.
  2. On the side panel, click on Roles.
  3. Click on Create role.
  4. In Select type of trusted entity, select AWS service.
  5. In Choose a use case, select CodeDeploy.
  6. There are several variants of use cases involving CodeDeploy. In Select your use case, click on CodeDeploy.
  7. Click on Next: Permissions. The AWSCodeDeployRole managed policy will already be attached to the role.
  8. Cick on Next: Tags.
  9. Click on Next: Review.
  10. In Role name, enter a suitable name for the role. In this tutorial we will use codedeploy-role. Take note of this role name, we will use it later when we create the deployment group.
  11. Click on Create role.
  12. In the AWS console, go to Services / Developer Tools / CodeDeploy.
  13. On the side panel, click on Applications, then on Create Application.
  14. In Application name, enter a suitable name for the set of conditions we will use to specify the instances o deploy to. In this tutorial, we will use basicspringbootwebapplication.
  15. In Compute platform, select EC2/On premises.
  16. Click on Create application.
  17. This will take you back to the application list. Click on the application you just created.
  18. Click on Create deployment group.
  19. In Deployment group name, enter a suitable name for the deployment group. In this tutorial, we will use basicspringbootwebapplication-deployment-group.
  20. In Enter a service role, enter the name of the service role you created earlier.
  21. In Deployment type, select In-place.
  22. In Environment configuration, select Amazon EC2 instances.
  23. Now we need to enter the actual condition based on which instances will be selected to deploy to. The condition is a set of tags. The set of instances that the application will be deployed to is the set of instances that have all the tags specified, with the same values (if given). In this tutorial, we will match based on only one tag, regardless of value. Add the following tag:
    1. Key: basicspringbootwebapplication
    2. Value: (leave empty)
  24. In Matching instances, check that your instance shows up. This is the instance CodeDeploy will deploy to.
  25. In Install AWS CodeDeploy Agent, select Never. We can skip this step because we already installed the CodeDeploy agent previously (if you haven’t done so, do it now using the instructions above).
  26. In Deployment configuration, select CodeDeployDefault.OneAtATime.
  27. In Load balancer, uncheck Enable load balancing.
  28. Click on Create deployment group.
  29. Now that we have an Application and a deployment group created, we will create the cd pipeline using them.
  30. In the AWS console, go to Services / Developer tools / CodePipeline.
  31. Click on Create pipeline.
  32. In Pipeline name, enter a suitable name for your pipeline. In this tutorial, we will use set the name basicspringbootwebapplication-cd-codepipeline.
  33. In Service role, select New service role. This is the role that will be used when CodePipeline runs.
  34. In Role name, enter a suitable name. Make sure the name is at most 64 characters long, otherwise AWS may let you move forward with the process but show an obscure error at the end when it tries to create the code pipeline.
  35. Click Next.
  36. In Source provider, select Amazon S3.
  37. In Bucket, enter the name of the S3 bucket you configured as artifact repository in the Upload stage of the CI pipeline.
  38. In S3 object key, enter the path to the application artifact within the artifact repository as you configured it in the Upload stage of the CI pipeline.
  39. In Change detection options, select Amazon CloudWatch events. This makes sure that whenever we update the artifact in the artifact repository, the CD pipeline will run automatically to deploy the new artifact.
  40. Click Next.
  41. Click Skip build stage. Our CD pipeline doesn’t need a build stage, because we have already set up the build separately as part of the CI pipeline.
  42. In Deploy provider, select AWS CodeDeploy.
  43. In Application name, enter the name of the CodeDeploy Application you created earlier.
  44. In Deployment group, select the deployment group that you created earlier.
  45. Click Next.
  46. Click Create pipeline.
  47. You are all set! Click on Release change to run the CD pipeline manually. If it runs without errors, the EC2 instance should be running your application.

If the pipeline fails, see the troubleshooting tips above for some pointers on how to investigate.

Sources

Leave a comment