In a Kubernetes cluster, it is often necessary to safely evacuate nodes for maintenance or troubleshooting purposes. One of the tools that can help with this process is kubectl drain. However, understanding how to use the command effectively and safely can be a challenge. In this blog post, we will demystify kubectl drain and explore its various options and best practices for safely evacuating Kubernetes nodes.

Understanding the kubectl drain command

What is kubectl drain?

• kubectl drain is a command-line tool that helps in safely evacuating Kubernetes nodes.
• It gracefully terminates all the pods running on a node and reschedules them to other available nodes in the cluster.

Why use kubectl drain?

• kubectl drain ensures that pods are not abruptly terminated, avoiding any potential data loss or disruption to running applications.
• It allows for planned maintenance or troubleshooting of nodes without impacting the availability of applications.

Draining Kubernetes Pods with the Kubectl Command

Syntax and basic usage

• The basic syntax of kubectl drain is kubectl drain .
• This command will gracefully evict all the pods from the specified node and reschedule them to other available nodes.

Options and flags

• --ignore-daemonsets: This flag allows kubectl drain to ignore DaemonSet-managed pods, which are typically meant to run on every node.
• --force: This flag forces the drain operation, even if there are pods that are not managed by a ReplicationController, ReplicaSet, Job, or StatefulSet.
• --delete-local-data: This flag deletes any local data associated with the pods being evicted.

Best practices for using kubectl drain

Communicate with the DevOps or SRE team

• Before draining a node, communicate with the team to ensure that the planned maintenance or troubleshooting does not impact critical applications.
  • Using a Kubernetes collaboration tool, such as Botkube, can help with this communication. Simply send a message to the team in the chat channel that is connected to the cluster and sending alerts about the pod that needs to be emptied. Use Slack or Teams ability to tag teammates to ensure the right people see your message.
  • Use the same communication channel to run the command to empty the pod with Botkube's kubectl plugin.

Considerations for DaemonSets

• When using kubectl drain, it is important to consider DaemonSets, as they are meant to run on every node.
• Use the --ignore-daemonsets flag to exclude DaemonSet-managed pods from being evicted.

Graceful termination of pods

• By default, kubectl drain waits for a pod's termination grace period to expire before evicting it.
• Ensure that the termination grace period is set appropriately for your pods to allow them enough time to gracefully shut down.

Conclusion

In this blog post, we have explored the command to drain pods and its various options and best practices for safely evacuating Kubernetes nodes. By understanding how to use kubectl drain effectively, you can ensure that your maintenance or troubleshooting tasks do not disrupt the availability of your applications. ‍

‍
Further into kubectl commands

At Botkube, we have created a toolset that not only assists with other areas of K8s, our tool specifically has executor plugins that allow for kubectl command automations. Our chat platform integration allows users to set kubectl aliases to run the commands directly from their preferred chat productivity platform such as Sl.ack or Teams.

If you found our above article helpful, we invite you to check out our kubectl cheat sheet where we go further into kubectl commands. It talks about the benefits of running these commands during troubleshooting and how Botkube can help platform engineers run helpful scripts quickly.

‍