Managing resources in Kubernetes can be both complex and time-consuming. In an era where automation is key, finding dynamic solutions is crucial. In this blog post, I will guide you through creating a custom Kubernetes operator using the Operator SDK, detailing the process I followed to develop the clientid-operator. Why custom operators?
Custom operators extend the Kubernetes API, allowing you to automate complex tasks like resource provisioning and configuration management. For instance, we created a custom operator because the Crossplane provider we were using could not sync the IDs for managed identities to other Kubernetes resources. This operator continuously monitors and manages these IDs, ensuring seamless integration and automation.
Other scenarios where custom operators are beneficial include:
Automatically scaling applications based on custom metrics.
Managing complex application dependencies and configurations.
Ensuring consistent and automated deployment of multi-component applications.
By leveraging custom operators, you can significantly streamline and enhance your Kubernetes operations.
We will use the clientid-operator as an example to demonstrate how to create a custom Kubernetes operator. Follow these steps to create and deploy your own operator: Prerequisites
Setup
Initialise your operator project using the Operator SDK in your terminal:
operator-sdk init --domain example.com --repo github.com/yourusername/your-operatorAdd custom logic
We will leverage the existing APIs from Crossplane, namely RoleAssignments and ManagedIdentity. This lets us skip creating our own API. Define the core method and helper functions required for resource updates in a file called controller.go. Lets start with our package, imports and defining our struct.
The struct we define here will be used by the Reconciler method, once we implement that. You may also notice the alias for our Crossplane API imports - those are necessary when both APIs are from the same repository. Moving forward, we will reference those APIs using the alias.
Lets continue with implementing the functions that will handle the actual updating of our resources in Kubernetes.
These functions audit and update Kubernetes service accounts and role assignments to ensure they align with the specified ManagedIdentity. They check each relevant resource across all namespaces and update them as necessary to match the designated client and principal IDs.
The last helper function we need to implement is used to extract the application name from the managed identity name. This function assumes a naming convention where the application name is always the third part of the managed identity name, split by hyphens. For example, in id-service-appname-dv-azunea-001, appname is the application name.
This function is critical for identifying which application the managed identity belongs to. It is possible to develop a much more versatile operator not dependent on a specific naming syntax, but it's fine for a simple operator such as the one we are creating in this blog post. Now, all of this comes together in the Reconcile method. This method fetches the user assigned identity, extracts the necessary client and principal IDs, and updates the corresponding service accounts and role assignments in our cluster by utilising the helper functions we created.
In addition to watching for changes in the resources already owned by the controller, it will check for changes every two minutes as defined in the method here:
return ctrl.Result{RequeueAfter: 2 * time.Minute}, nilThats our controller logic implemented. However, there is one crucial step left. We need to integrate our controller with a Kubernetes controller manager. We can do that by creating a function like this:
Thats our custom operator ready for building and deployment!
Build and push
To simplify the deployment process of the operator in our Kubernetes clusters, we will create a Docker image. When you initialised your Operator SDK project, a default Dockerfile was generated for you. We can use this out of the box, meaning everything is ready for us to build and push the Docker image to our container registry.
You may use whatever container registry you want for this.
docker buildx build --platform linux/amd64,linux/arm64 -t yourusername/clientid-operator:latest --push .Deploy your operator
The custom operator is ready for deployment. You can deploy your operator to your cluster with the following Kubernetes manifests:
Thats it!
You now have your very own custom operator created using the Operator SDK running in your cluster.
To explore the full implementation, check out the clientid-operator repository on GitHub.
Opmerkingen