A major requirement in developing a non-trivial software product is the establishment of a formal workflow for Continuous integration and Continuous delivery. Whether it’s an individual or group project, the CI/CD workflow lays the groundwork for getting code from the developer’s box to the live environment.
But it goes beyond that. CI/CD allows code to be deployed frequently while maintaining quality and reliability. Typically, this is accomplished with the help of automated unit and integration tests at specific points in the development pipeline. Tests that will help to ensure that new code does not break existing code.
Continuous Integration is so important that a failure to implement it formally results in haphazard development and loss of code quality which only creates more stress for the engineering team. You can read more about CI/CD in this link.
Dependency Management
Due to the nature of modern-day software development, entire systems are built by relying on functionality from different components. As these components are allowed to evolve independently, there will need to be a way to ensure they stay compatible. We don’t want updates on one component to cause bugs to manifest in another component. Hence, in order to write software, we need to rely on dependency management.
Dependency Management allows us to package dependencies in our project and gracefully upgrade those dependencies if we need to. In the Java ecosystem, there are two primary dependency management tools: Maven and Gradle. Maven uses XML configurations to define the project build, including all plugins and dependencies. Gradle on the other hand uses a Groove / Kotlin-based Domain Specific Language (DSL) to define the build tasks and dependencies.
Let’s talk a bit about versioning.
Versioning
Versioning is a process of assigning a unique number to a specific version of a software artifact. Such that at every point in time, the software version number gives an idea of how the artifact has evolved.
Versioning is an important part of Dependency Management because it allows us to keep track of the latest developments in our dependencies.
A few questions might arise: how do we upgrade the dependency versions of our Projects? What format do we employ for versioning? How do we ensure that upgrades to dependencies are tracked and don’t break existing functionality?
Currently, Semantic Versioning is very widely used. Here’s what it looks like:
Given a version number: For example 1.5.2, there are 3 parts: MAJOR(1), MINOR(5), PATCH(2).
MAJOR: updated when you make incompatible API changes; that is, changes that can break existing systems using that dependency.
MINOR: updated when you add functionality in a backward-compatible manner.
PATCH: updated when you make backward-compatible bug fixes.
Now we can delve into branching models in git.
Branching Models
A branching model is a set of rules developers follow when working on a shared codebase. They help structure the approach to branching and merging code, which helps teams work more efficiently. I strictly adhere to a branching model that allows me to compute a version for my project based on the Git history. There are a few branching models in software development:
- GitFlow
- GitHub Flow
- Trunk-based development
I’ll shed more light on GitFlow since it’s the most popular. Gitflow uses different branch types:
- Master or Main: This is the stable branch that contains the last version of code released into production and should ALWAYS be production-ready.
- Feature: feature branches are meant for implementing features. Usually, developers will branch off develop and write all code pertaining to that feature on the feature branch before merging it to develop
- Develop: Developers merge to develop branch when features are ready to be integrated into the master, the development branch serves as a branch for integrating different features planned for an upcoming release
- Release: branches off develop and used to prepare a production release. When the release branch is tested, it is typically merged into develop and master.
- Hotfix: Is used to fix bugs that arise on the master branch. Hotfix branches off master and is merged back into master and develop on completion and testing.
The GitFlow model works best for larger teams though can be more tedious to manage.
A model I employ in my projects is somewhat a combination of GitFlow and Trunk-based development.
- Create two permanent branches: main and develop. main for production-ready code, develop for feature integration.
- When working on a feature, branch off develop, and create a branch prefixed with ‘feature/’. For example to work on user login, create a branch off development called feature/user-login.
- When the feature is ready, push the latest feature code to the remote repo and create a pull request to develop.
- Run code review, source code analysis, automated unit, and integration tests on the feature branch to ensure it is suitable to be merged to develop.
- Merge code to develop and run automated unit and integration tests again to ensure feature merge has not broken other features.
- Merge code to the main branch. Run tagging, and versioning, and prepare the release of the artifact. The merge to main can be automated or done manually by a Senior engineer who might have reviewed the changes.
- Publish.
- Merge back to develop.
The primary difference between this process flow and the more popular Gitflow is I don’t create a release branch. Instead, after testing develop, I merge to main and run my release management on main. The develop branch exists for testing and integration purposes. This will ensure that only fully tested code is merged to main. The process of merging develop to main can be automatic (triggered by CI) or manual should you want a Senior Dev to review the code before it reaches main.
In a sense, this process is similar to Trunk Based development where main and develop both serve as Trunk. After running release management, merge main back to develop. Github avoids cyclical builds so the merge back to develop will not trigger CI.
Let’s implement the CI Pipeline
First, we install GitVersion locally:
$ brew install gitversionI use GitVersion to handle my versioning.
GitVersion uses the git commit history to compute a version for the software project. The commit messages are parsed for certain patterns:
- When you add a new feature, the commit message should include: +semver: feature
- When you add a bug fix, the commit message should include: +semver: patch
- When a commit introduces a breaking change, the message should include: +semver: major
To set up the project for semantic versioning support. I use the bash script:
#! /bin/bash
git flow init
gitversion init
git commit -m "Setup Versioning"The above script will show a wizard in the command prompt requesting to fill in information about the project setup. The wizard is quite simple and easy to follow. Once you’re done, you should have a GitVersion.yml file with the following structure in the project root:
mode: ContinuousDeployment
branches: {}
ignore:
sha: []
merge-message-formats: {}In the root folder of your project, you should have 3 workflow files declared in the .github/workflows path. Something like this:
Feature/Hotfix branch Integration. (/.github/workflows/non-mainline-branch-update.yml)
name: Feature/Hotfix Build
on:
push:
branches:
- 'feature/*'
- 'hotfix/*'
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Set up JDK 17
- uses: actions/checkout@v3
uses: actions/setup-java@v3
with:
java-version: 17
distribution: zulu
- name: Cache SonarCloud packages
uses: actions/cache@v3
with:
path: ~/.sonar/cache
key: ${{ runner.os }}-sonar
restore-keys: ${{ runner.os }}-sonar
- name: Cache Maven packages
uses: actions/cache@v3
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2-
- name: Build and analyze
env:
run: mvn -B verify -s settings.xml -f pom.xml
The above script is triggered on push events for branches matching the following patterns: feature/*, hotfix/*. The step names describe what is being done in each step of the build.
- Setup the JDK environment,
- Restore cache for Sonar. You can skip the Sonar caching step if it doesn’t apply to you. I use Sonar Cloud to scan my artifacts, hence I need to include the Sonar Caching step to speed up the build.
- Restore cache for Maven. This helps avoid re-downloading dependencies and thus speeds up the build. The cache key is related to the hash of pom files in the project, so changes to the poms will compute a new hash and trigger the re-downloading of dependencies.
- The last step will run the build and analysis
Develop branch integration (/.github/workflows/develop-integration.yml)
name: Develop Branch Integration
on:
pull_request:
branches: [ develop ]
types: [ closed ]
jobs:
build:
runs-on: ubuntu-latest
if: github.event.pull_request.merged == true
steps:
- uses: actions/checkout@v3
- name: Setup Java 17 env
uses: actions/setup-java@v1
with:
java-version: 17
- name: Cache Maven packages
uses: actions/cache@v3
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2-
- name: Build and analyze
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
run: mvn -B verify -s settings.xml -f pom.xml
- name: Set Commit Message
id: commit
run: |
${{ startsWith(github.head_ref, 'feature/') }} && echo
::set-output name=message::"+semver: feature" \
|| echo ::set-output name=message::"+semver: patch"
- name: Commit Build Message
env:
COMMIT_MSG: ${{ steps.commit.outputs.message }}
run: |
git config user.email ${{ secrets.GIT_EMAIL }}
git config user.name ${{ secrets.GIT_USERNAME }}
git add .
git commit -m "$COMMIT_MSG" --allow-empty || true
- name: Push changes
uses: ad-m/github-push-action@master
with:
branch: develop
github_token: ${{ secrets.GITHUB_TOKEN }}
merge-main:
name: Merge to Main
needs: [ build ]
runs-on: ubuntu-latest
if: github.event.pull_request.merged == true
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Fetching
run: |
git fetch --all
- name: Merge to Main
uses: devmasx/[email protected]
with:
type: now
target_branch: 'main'
env:
GITHUB_TOKEN: ${{ secrets.GIT_ACCESS_TOKEN }}
The steps are conditional on merging to develop. Hence the snippet: ” if: github.event.pull_request.merged == true". This build starts off almost like the previous build, but this time has a ‘Set Commit Message‘ step where I set a message in output which I intend to use in the next build step. This message is going to be used in the “Commit Build Message” step and will form part of the message in the commit.
On the master build, the commit message will be utilized by GitVersion to compute the version of the software. If I merge a feature/* branch, I commit with the message “+semver: feature” else I use “+semver: patch”. For breaking changes introduced to the artifact, I would manually commit a message “+semver: major” on my local. After the tests run successfully, I commit the changes to develop branch and automatically trigger a Merge to Main.
The above script will not work without declaring the secrets in your Git repository settings. Go to Actions under Secrets and Variables.
When you push code to a branch prefixed with feature/*, hotfix/* the above pipeline is triggered. You can check the status of the job under the Actions tab.
Main Branch Integration (/.github/workflows/main-integration.yml)
name: Main Branch CI
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the main branch
on:
push:
branches: [ main ]
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Fetching All
run: |
git fetch --prune --unshallow
# Install .NET Core as it is required by GitVersion action
- name: Setup .NET Core
uses: actions/setup-dotnet@v3
with:
dotnet-version: |
3.1.x
5.0.x
# Install Git Version
- name: Installing GitVersion
uses: gittools/actions/gitversion/[email protected]
with:
versionSpec: '5.3.x'
# Use Git Version to compute version of the project
- name: Use GitVersion
id: gitversion
uses: gittools/actions/gitversion/[email protected]
# Setup Java environment
- name: Setup Java 17 env
uses: actions/setup-java@v1
with:
java-version: 17
# Cache and restore Maven dependencies
- name: Cache Maven packages
uses: actions/cache@v3
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2-
# For a maven artifact, set version to what was computed by GitVersion in earlier step
- name: Evaluate New Artifact Version
run: |
NEW_VERSION=${{ steps.gitversion.outputs.semVer }}
echo "Artifact Semantic Version: $NEW_VERSION"
mvn versions:set -DnewVersion=${NEW_VERSION}-SNAPSHOT -s settings.xml
# Deploy artifact to repository. Could be ossrh, archiva etc.
- name: Build and Deploy with Maven
env:
ARTIFACT_REPO_USERNAME: ${{ secrets.ARTIFACT_REPO_USERNAME }}
ARTIFACT_REPO_PASSWORD: ${{ secrets.ARTIFACT_REPO_PASSWORD }}
run: |
export MAVEN_OPTS="--add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED --add-opens=java.desktop/java.awt.font=ALL-UNNAMED"
mvn clean deploy -s settings.xml -f pom.xml
# Optional step where I like to write the version number to a file in the project root.
- name: Upgrading Version
run: |
RELEASE_TAG=${{ steps.gitversion.outputs.semVer }}
echo $RELEASE_TAG > version.ver
git config user.email ${{ secrets.GIT_EMAIL }}
git config user.name ${{ secrets.GIT_USERNAME }}
git add .
git commit -m "Upgraded Version >> $RELEASE_TAG" || true
- name: Push changes
uses: ad-m/github-push-action@master
with:
branch: main
github_token: ${{ secrets.GITHUB_TOKEN }}
merge-develop:
name: Merge to Develop
needs: [build]
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Fetching
run: |
git fetch --all
- name: Merge to Develop
uses: devmasx/[email protected]
with:
type: now
target_branch: develop
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Following the comments in the main build should be straightforward. I compute the version of the artifact using GitVersion, then I deploy the artifact after calling “mvn versions:set”. The new version is Committed to main and then merged back to develop. The deployment step will vary depending on the nature of the artifact. For example, it could be a service being deployed on Heroku. In this case, I will have a step like this:
- name: Push changes to Heroku
uses: akhileshns/[email protected]
with:
heroku_api_key: ${{secrets.HEROKU_API_KEY}}
heroku_app_name: ${{secrets.HEROKU_APP_NAME}}
heroku_email: ${{secrets.HEROKU_EMAIL}}
We can do anything really. Deploy to Heroku, AWS, GCP, or push to docker. Whatever works based on our process flow.
Here’s a sequence diagram to recap what our CI looks like:
Conclusion
In this article, we have discussed how to automate the CI process of Maven artifacts using GitHub Actions. By using GitHub Actions, you can easily configure and run Maven commands, such as “mvn test” and “mvn deploy”, whenever changes are pushed to your repository. This helps to catch errors early and ensure that the code is always in a releasable state.
Please note that this is an example and it might be necessary to adjust the steps and commands to match your specific use case 😉