Implementing Your Own Botkube Plugin - Keptn Use Case
May 24, 2023
10 minutes
read
Huseyin Babal
Cloud Engineer
Botkube
Botkube has plugin system that allows you to extend the functionalities of Botkube core to satisfy your requirements. Here is a real-life example using Keptn.
Botkube's plugin system allows users to extend the functionalities of Botkube core to enhance the capabilities of cloud native tools already in their tool belt. Whether you're using some of our pre-built plugins like Helm or Prometheus, or want to use tools like Flux, Thanos, or any others on the CNCF landscape, the plugin system will help you and your team with collaborative troubleshooting instantly through your favorite messaging platform.
So let's dive in!
In this article, we will implement a Botkube plugin to integrate it with Keptn, a cloud-native application lifecycle management tool for kubernetes. Use this as a starting point for your own creation!
Background
Before starting Keptn plugin implementation, let's take a look what it is and what kind of use-case it solves for us.
Assume that you have CD flow that ships your features to live environment whenever you merge that specific feature into main branch.
You might want to do a couple of checks before deployment, and Keptn is good for that kind of verifications. You can do a load test with k6, an open source load testing tool, by using in Keptn to do load test and verify results. However, to make it simpler to understand how Keptn works and how to integrate it with Botkube, we will be deploying a simple task with Keptn and consume Keptn events with Botkube to receive notifications in communication platform like Slack.
Keptn Installation
You can easily deploy keptn via Helm Chart with the following command.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To execute a task in Keptn environment, we can use a Keptn integration which is 'Job Executor Service'. This will help us to execute task
based on our configuration. You can deploy executor service as shown below.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Keptn produces lots of events during the lifecycle management, but notice we are only interested in the event type sh.keptn.event.remote-task.triggered. Since, we
deploy job executor service in same Kubernetes cluster, it will automatically find Keptn gateway with the flag remoteControlPlane.autoDetect.enabled="true"
Accessing Keptn Gateway
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
You can visit http://localhost:8080 to access Keptn Gateway by providing user and password which we extracted before.
Keptn CLI
While you can handle everything in Keptn Gateway Dashboard, it would be beneficial to use Keptn CLI to handle things faster. You can install
Keptn CLI with following command.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In order to access Keptn Gateway, you need to authenticate Keptn CLI with following command.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
You can copy this command from Keptn Gateway Dashboard under User icon in top right corner.
Now we are ready to create resources in Keptn.
Create Keptn Project
In Keptn, everything starts with Project where you basically define your Keptn resources. Keptn projects works with Git Upstream that you can also create one as shown below.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$GITHUB_USERNAME and $GITHUB_TOKEN is used by Keptn to maintain its state in remote repository which is defined by empty $GITHUB_REPO. --shipyard parameter
is used to defined project stages each deployment should go through until production stage. You can see an example shipyard file as follows.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We have only one stage production and it contains a task which is named as remote-task. Now that we defined our project, let's continue with creating a service as follows.
Create Keptn Service with Resource
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Above command simply creates a hello service under botkube project. Next step is to add resources to this service, for example a job definition as you can also see below.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We added job config to hello service for production stage under botkube project. Also, you can see the content of jobconfig.yaml below.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
As you can also understand, it simply provisions a task by using alpine image to print Hello World. Once the task is started, it will produce an event sh.keptn.event.remote-task.triggered. This event
is the one our job executor service interested in.
Trigger Keptn Sequence
We have all the resources ready and now we can trigger a sequence to deploy remote task to see the response with following command.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
You will see the result of the command, but to see everything in detail, you can navigate to Keptn Gateway and see Sequences as follows.
Now that we understand how to trigger a sequence in Keptn, let's take a look how to access its API to consume events to integrate with Botkube.
Accessing API
You can navigate to http://localhost:8080/api to see Swagger Documentation of Keptn APIs. It contains multiple modules, and to receive events, we can select mongodb-datastore from the dropdown.
Notice we already authorized Keptn CLI in previous sections, and we can reuse the token inside keptn auth ... command to use in Swagger Authorize to call endpoints in Swagger UI or construct following curl command.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Now we are ready to implement actual business logic of the Keptn plugin to receive events. You will see partial
examples of the codebase in this article, but don't worry, you can see the full codebase here.
Plugin Configuration Structure
In order to integrate Botkube with Keptn, Botkube should now be the entrypoint of Keptn to consume events. Also, it needs a token for successful authentication with Keptn. Additionally, we can add filtering parameters like project and service to consume events for certain project and service. With those information, following Go struct can be used for plugin configuration.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To have a smooth plugin integration, we can have configuration defaults, and once end user provides their own configuration, we should merge those configs
as follows.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We will implement an SDK to connect Keptn API to consume events. This will be a facade to reduce complexity and coupling.
Keptn already has a Go SDK, and in our client we will initialize an instance of that client and call events endpoint as shown below.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
To summaries above example, Events method accepts context and filter request to get events based on the filtering parameters.
That method can return slice of Events or an error. Now that we have a client, let's implement the core of our plugin to call this
Keptn client implementation.
Keptn Plugin Core
In order to write a source plugin, we need to implement following contracts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Stream is a gRPC endpoint that is consumed by a gRPC client in Botkube core. So, Botkube will stream events from this endpoint.
Metadata is used for providing information about Keptn plugin. This will be also used by CI system to generate plugin metadata.
Metadata can contain plugin configuration details.
In Stream method, we need to consume Keptn events as follows.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Above implementation will poll Keptn endpoint each for 5 seconds and will return a Message to Botkube so that it will dispatch this message to configured platform.
Notice, we use FromTime parameter to fetch only last 5 seconds windowed events. As you also noticed, we use a special struct api.Message for structured message that you can see advanced details about that here.
Building Plugin
In order to build plugin, you can use following command.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This will build the plugin, and generate a metadata file for plugin index. PLUGIN_DOWNLOAD_URL_BASE_PATH environment variable is used
for defining the base path of plugin download url. Now we have plugin binaries locally, then we can serve them with following command.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Once you navigate to http://localhost:8080/dist, you can see the keptn plugin binary. This means, Botkube core can consume this endpoint to donwload and register as Botkube plugin.
Running Botkube
In order to use the Keptn plugin, we can use following repository config in Botkube configuration file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The above config is just for making Keptn plugin visible to Botkube core, now we can add a Keptn source plugin configuration as follows.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
All the Keptn configurations are defined under config section. Once Botkube application starts, it would stream Keptn plugin and that plugin
consume the endpoint http://localhost:8080/api with query parameters project=botkube and with auth headers X-Token=<keptn_token>. However, we need to add this Keptn source to communications configuration of Botkube as follows.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Whenever an event is fired in Keptn, this event will be stored in mongodb-datasource and this will be consumed by
Keptn plugin. Finally, with above configuration, it will be sent to platforms which are defined in communications section.