By Wojciech Kocjan, Solution Artichect at Bitnami
Bitnami Stacksmith is a tool that automates and optimizes the packaging of your applications for deployment to cloud and container platforms. It also continuously monitors your applications for updates and patches, and maintains them so they remain up to date and secure.
At Bitnami, we understand that enterprise companies rarely conduct release management manually - it is a complex process used to coordinate developers, components, and steps that include application development, code check in, compiling and testing, versioning and releasing, and deploying. Typically, this process is automated using one or more Continuous Integration (CI) or Continuous Deployment (CD) tools to streamline the process.
Stacksmith performs a few critical pieces of the release management process. In certain use cases, like when trying to repackage your traditional applications so you can replatform them from your data center to the cloud, using Stacksmith as a ‘stand-alone’ solution can be the right approach. With that being said, Stacksmith provides significant value in use cases that involve ongoing application development, deployment and maintenance, which is why we felt it was critical that Stacksmith be able to fit easily into any existing CI / CD or orchestration tooling you already utilize.
This article goes through the process of creating a fully automated setup where Bitnami Stacksmith is part of the continuous integration (CI) cycle and generates up-to-date cloud deployable assets. Here, I’ll use a Java web application that uses a database, but the process is applicable for different setups and use of the various
stack templates that Stacksmith provides.
My setup will consist of the following components:
The following image shows how the process of application development, compilation and packaging will look like:
Why introduce Stacksmith to the CI / CD pipeline?
Atlassian Bamboo is a powerful continuous integration server that lets you automate release management in your CI / CD pipeline. It provides the basic tooling to define projects and how they’re built, but it’s up to its users to define the exact stages and tasks that should be run.
In many cases, when companies set up Bamboo, they focus on the development part of the lifecycle - compiling the application, then running unit tests to ensure it is working properly. Bamboo comes with multiple tasks for compiling different types of applications, running tests and understanding its results. In these instances, the continuous deployment part is given less focus - in many cases, it simply involves copying the built artifacts to a staging and/or production server, and restarting it.
Bitnami Stacksmith can improve the CD part of the pipeline by enabling packaging of the application and creating deployable assets for various public clouds as well as Kubernetes. It creates outputs specific to the target platform - such as AMIs and CloudFormation templates for AWS, VM images and ARM templates for Azure, and Docker images and helm charts for Kubernetes. Stacksmith outputs are also optimized for each platform - leveraging load balancers, databases and other services available to those specific public clouds.
Stacksmith also lets you include your IT and security policies in the process - including image OS and configuration hardening, network policies, agents and any other policies, and applies them as it builds the deployable assets. This ensures that every image is built according to Security’s specification. In addition, Stacksmith can analyze whether those have changed, and either notify you or automatically repackage your application with the latest policies.
Stacksmith packages the application and provides environment-specific deployment mechanisms that launch the application along with all of its dependencies - setting up networking, databases, load balancers and other resources. The deployments can be used to automatically or manually launch the application. It can also be used to manage existing deployments - providing a smooth way to upgrade as well as downgrade the application in development, QA or production environments.
By having assets that allow new deployments of the application to be easily launched, it’s possible to perform full deployment of the application in the cloud, run automated tests of the application and its entire environment for each build, and validate the exact setup it will be run in.
The following illustrates how performing continuous packaging fits into the CI / CD pipeline:
Bitnami Stacksmith provides additional value by constantly monitoring the components that went into the deployable assets. It monitors trusted sites for new releases and security updates for your application's components, provides alerts when available, and offers a simple repackaging process.
Integrating Jira, Bitbucket, Bamboo and Stacksmith
First, I’ll set up Atlassian Jira, Bitbucket, Bamboo and Bitnami Stacksmith and show how the suite of tools work together to provide a CI / CD pipeline that includes the creation of deployable assets. Next I will show how this pipeline performs end to end testing by launching the application in the cloud and verifying that it is working correctly. Finally I’ll update the deployment provided the application or deployable asset tests were successful.
The components and setup
The first piece of software is
Atlassian Jira. It’s an issue and project tracker that is used to track projects - specifically to track the progress of application development as well as all the issues related to it.
Second application is
Atlassian Bitbucket - it’s used for storing the source code of the application. It’s configured to be linked to the Bitbucket and Jira servers, following the process in
Atlassian documentation.
Third element is
Atlassian Bamboo - it is the continuous integration (CI) server that allows you to define build plans and run them to create cloud deployable assets from the application’s source code. In order to have all the data available across all Atlassian apps, Bamboo is linked with
Jira as well as
Bitbucket. And since Stacksmith will be used to do the actual packaging and creation of the deployable assets, Bamboo will also be linked to Stacksmith.
The final and centrally important element is
Bitnami Stacksmith - the tool that will actually create the assets that will be deployed to the cloud. All that is needed to set up Stacksmith is to sign in (if you don’t already have a subscription you can sign up for a
free trial), and go through a very easy process of onboarding your
AWS or
Azure account.
Defining the CI pipeline - from source code to deployable assets
Now that all of the applications are set up, I can configure them so that the process of going from source code of the application to deployable assets that I can run in the cloud is fully automated.
Building application from source code with Atlassian Bamboo
I’ll start with Atlassian Bamboo. I’ve created a very basic build plan that packages the application - simply checking out the source code from Bitbucket server and using
Maven to build the application from source code.
Bamboo will now monitor the source code in the Atlassian Bitbucket server, and will perform a new build every time the application source code changes. Thanks to the links that have been set up in this integration, all of this data is also accessible from Jira.
Creating deployable assets using Bitnami Stacksmith
Next I need to set up Stacksmith, so it can be used to build the deployable assets. The first step is to set up the application definition in Stacksmith - this involves selecting the specific stack template you will use. The stack template informs Stacksmith about the type of application you are packaging and its requirements. In this case, I will select the "Java Tomcat Application with DB (MySQL)" stack template, which informs Stacksmith to handle Tomcat set up and create the database, leveraging cloud-native resources if possible (such as
RDS on AWS or
Azure Database for MySQL on Azure). This step only has to be done once, and can be done via
Stacksmith Web UI, its APIs, or the Stacksmith CLI.
The next step is to integrate Atlassian Bamboo with Stacksmith. This can be done with the stacksmith-cli tool that provides a convenient way to trigger rebuilds of an application in Stacksmith, retrieve the build log in real time, and get results back. All of the Stacksmith APIs, as well as the CLI tool, are available to all Stacksmith users - including those using the free trial.
The stacksmith-cli is used inside a CI tool such as Atlassian Bamboo. To configure integration with Stacksmith, a simple YAML file with configuration for the CLI tool is created - Stackerfile.yml. Here is a sample contents of the file:
appId: bitnami/apps/7cbf4630-8c6d-0136-d7fb-66a676282717
appVersion: 1.0.0
files:
userUploads:
- target/myapp-1.0.0.war
userScripts:
boot: boot.sh
The
appId specifies the application identifier that’s provided by Stacksmith. The
appVersion field specifies the version to pass to Stacksmith. The
files section specifies the files to upload - the application file(s) as well as the scripts that Stacksmith should use.
The
target/myapp-1.0.0.war file was built by Maven (as described earlier in this article) in Bamboo. The
boot.sh file is a
boot script and includes application-specific logic and specifies how to provide the application, with details required for connecting to database.
The
Stackerfile.yml can either be written manually or generated programmatically - such as using the Maven
filtering functionality to create the file with appropriate placeholders.
The last thing that is needed is to create a helper script called
build-with-stacksmith.sh that calls the
*stacksmith-cli* to build the application. Here’s the contents of the script:
#!/bin/bash
set -eu -o pipefail
stacksmith auth login --access-token "${STACKSMITH_ACCESS_TOKEN}"
stacksmith build
The
Stackerfile.yml,
boot.sh and
build-with-stacksmith.sh files are committed to the repository with the source code of the application, which makes it possible to call them directly from Bamboo.
Setting up Bamboo to trigger a build in Stacksmith is easy. It requires setting up the
stacksmith-cli binary on the Bamboo server. Then all that is needed is to create an additional step in the build plan - running the
build-with-stacksmith.sh script from the source code repository.
The
STACKSMITH_ACCESS_TOKEN environment variable is passed to the script and contains an API token that can be retrieved using the
stacksmith-cli.
This new step can be tested by manually running a new build, which calls the CLI tool and triggers a new build in Stacksmith. The link and live information about the build is also shown in Bamboo at build time, and is accessible after the build finishes.
Bamboo is already monitoring the source code repository in Bitbucket for changes - any new change to in the application code will now trigger a rebuild, performing a full automation from source code to deployable assets.
Continuous deployment to the cloud
At this point our application is packaged and Stacksmith has created the charts or templates that will be used to launch the new deployment or update an existing deployment -
Helm charts for Kubernetes,
CloudFormation templates for AWS and
ARM templates for Azure. This enables you to create a test deployment and run additional tests against the application as well as upgrade the application version in the development, QA or production environments. The Stacksmith API or the stacksmith-cli tool can be used to retrieve the outputs from the packaging process.
Having an simple way to deploy the application makes it easier to run tests of the entire cloud deployment. All that’s needed is a relatively small script that can be run from Atlassian Bamboo - to simply retrieve the template URL, launch a test deployment and run tests against the application.
Below is a sample shell script that runs tests using AWS. Deploying to Azure or using helm charts is very similar, except they use different client tools.
#!/bin/bash
# unique name for the deployment
stack_name=teststack-$(date +%Y%m%d%H%M%S)-$$
# template URL is retrieved from Stacksmith and passed as argument
template_url=$1
# launch the template in AWS or fail immediately
aws cloudformation create-stack --stack-name "${stack_name}" \
--template-url "${template_url}" || exit 1
# (wait for cloudformation stack to be launched)
# retrieve URL of the application from AWS
url=$(aws cloudformation describe-stacks --stack-name "${stack_name}" | jq -r \
'.Stacks[0].Outputs[] | select(.OutputKey == ("PublicDnsName")) | .OutputValue')
# run test and store whether it succeeded
./application-tests.sh "${url}"
code=$?
# delete the deployment
aws cloudformation delete-stack --stack-name "${stack_name}"
exit ${code}
The
application-tests.sh script should perform application specific tests, getting the URL that the application is available. It should indicate if the tests have failed using non-zero exit code. The tests can include anything - such as interacting with the application via APIs and/or web automation tests.
Similar to the b
uild-with-stacksmith.sh file, this script can be committed to the application source code repository and set up to be run as a task in Bamboo.
Deploying an updated version of the application in various environments is very similar - the only thing that is needed is to update the existing deployment - using
aws cloudformation update-stack command instead of
create-stack. For example:
#!/bin/bash
set -eu -o pipefail
# unique name for the deployment
stack_name=teststack-$(date +%Y%m%d%H%M%S)-$$
# template URL is retrieved from Stacksmith and passed as argument
template_url=$1
# launch the template in AWS or fail immediately
aws cloudformation update-stack --stack-name "deployment-staging" \
--template-url "${template_url}"
# (wait for cloudformation stack to be updated)
The script above will update the existing deployment in AWS and wait to ensure it has succeeded. It should also be committed to the application source code repository and invoked as a task from Bamboo. It should be invoked after the tests of the deployment have passed.
The packaged application can also be manually launched from the Stacksmith Web UI. Launching the application into any cloud simply requires choosing the right build and clicking the
Launch button:
This starts the process of deploying the packaged application, redirecting the user to the cloud provider’s console and finishing the process - for AWS this is done using the
AWS CloudFormation Console. For Azure, this starts the process of deploying an ARM template in the
Azure Portal.
Monitoring application for security issues and updates
Stacksmith provides functionality to ease the maintenance of your applications. It continuously monitors the components that make up the deployable assets (such as system packages), checking for updates and known security vulnerabilities. It alerts the user about security issues as well as whether updates are available that resolve the security issues. It also provides convenient manual options or fully-automated and continuous ways to repackage the application.
This ensures a smooth process for upgrading your application, making it easy to repackage a fixed version of the application and update existing deployments to fix any security issues.
Summary
As this article has demonstrated, it is easy to utilize our set of APIs and CLIs to integrate
Bitnami Stacksmith with
Jira,
Bitbucket and
Bamboo. This combination enables you to set up a fully automated pipeline that starts with source code and ends with the creation of assets you can deploy to the cloud.
The Stacksmith APIs and CLIs can also be used to integrate with other popular CI / CD tools. It is easy to apply the same steps we discussed here to an environment that utilizes
Jenkins,
GitLab and many other solutions.
Package, deploy, and maintain your own application today. Sign up for the Stacksmith 30 day free trial and find out how easy it is!
Have questions about how Stacksmith can fit with the CI / CD tools you currently use? Or want a personalized demo? Contact us at enterprise@bitnami.com.
