The Anatomy of Kubernetes and OpenShift Resource Waste

All companies that transition to Red Hat OpenShift quickly come to the same point. The initial excitement is palpable. Developers quickly embrace self-service deployments and operators appreciate built-in observability and declarative routing. Members of the platform engineering team quickly move company services to new machines. However, 6-18 months later, management has to face an unpleasant truth: cloud expenses keep increasing while actual hardware utilization remains in the range of 10-20% of the available capacity.

Cluster administrators are often perplexed by the fact that their nodes find themselves out of usable CPU or memory, while metrics show the opposite. The phenomenon under discussion is characterized by the following paradox: while the cluster is exhausted, it is idle.

Kubernetes determines the arrangement of all pods based on reservations rather than usage. When a container declares its resource needs, the kube-scheduler ensures that the required amount of resource is reserved, regardless if the application utilizes it or not. The scheduling system won't care whether this amount is indeed used by the application; thus, the cluster will be out of this capacity for any other workloads. In such cases, developers will overestimate their resource needs, and the platform will be obliged to spin up additional hardware to meet the specified reservations. In essence, the company will end up paying for resources that it won’t even use.

To remedy this situation it is necessary to stop relying on the reactive provisioning of nodes and turn to sound resource management practices.

The Economics of Container Sizing: Requests, Limits, and Quality of Service

To grasp how resources are allocated, one must delve into how computing resources are limited by the various behaviors instigated by the Linux kernel. OpenShift uses the standard UNIX control groups or cgroups to manage resources. While creating a pod, engineers will declare two parameters that specify CPU and memory resources. These parameters are called requests and limits. Mixing up the two forms of resources is mostly responsible for overprovisioning.

A request is the minimum resource needed, and it defines where the pod is going to be located and guarantees its minimum room for operations. The limit defines the maximum threshold above which the container may stop the flow of the workload. How the engineering team utilizes requests and limits for their pod dictates the Quality of Service classification provided by the platform and has a direct influence on the node density and resilience of the cluster.

In case requests and limits have been put to the same high levels, this means that a pod is classified as Guaranteed. The allocated pods get complete isolation; the host will not kill them in case of resource crunch unless there are absolutely no other choices left. Importantly, this situation hinders bin packing costs for regular workloads since there is excessive capacity wasted and the scheduler treats all services as important ones with the capacity allocated for no load on every worker.

If requests fall under limits on the other hand, the pod is called Burstable in OpenShift. This is where the high density economics begins. Under this tier, a burstable pod gets all that is needed for the boot and normal processing of a given workload and can use the excess node capacity in case of urgent traffic issues.

BestEffort represents the most basic level of service in terms of pod usage. It does not have any restrictions or applications. It simply utilizes whatever free resources are available on a node. However, it gets first priority termination at the moment at which the memory usage reaches a critical level. BestEffort pods are usually not accepted in corporate usage, even though they have advantages when it comes to testing and batch processing.

The main risk of working with these types of pod levels is related to CPU throttling and memory eviction issues. CPU is compressed in nature. If there is a situation of the pod exceeding the limit for using the CPU resource, the operating system uses the fair share mechanism to limit its work. The performance of the application drops, and the execution time increases, but the process itself survives.

As for memory usage, the situation is completely different. Memory is incompressible. Thus, when a certain application tries to allocate more physical memory than it is allowed, the operating system triggers the Out of Memory kill immediately.

Because developers are worried about sudden memory dropouts, they ask for twice or even four times more memory than needed. This precautionary step has a multiplying effect over hundreds of microservices causing numerous phantom allocations. In order to solve this issue, the application teams require systematic learning about the interaction between JVM heap and container runtimes, aspects concerning Linux cgroups, and profiling methods. Providing full openshift online training for engineering teams makes it possible to eliminate the knowledge disconnect between local development practices and Kubernetes resource management.

Setting up the Guardrails: OpenShift Resource Quotas and Limit Ranges

If left unchecked, self-service quickly runs amok. If a developer can freely set up a namespace and get unlimited compute on their own with no admin ever being involved, every team can be expected to overuse as much compute resources as possible. In OpenShift, this process is controlled with the help of two built-in policy components: Limit Ranges and Resource Quotas.

Limit Range serves as the first level of protection both at the container and pod level within a particular project. Without a Limit Range, a developer might create a pod without even mentioning request or, what is even more absurd, use a pod manifest to claim a full node with 64 cores just for a basic web application.

The LimitRange enables system administrators to define certain limits for individual computing components. The redhat openshift course for Linux requires an understanding of setting up the native policies ensuring that cluster engineers can set default limits before the overprovisioning process begins.

Above all, Limit Range applies default requests and limits. Upon submitting a manifest without resource definitions, a developer will automatically get project defaults applied by the operating system instead of allowing the pod to use other defaults with no allocation of resources whatsoever.

While Limit Ranges apply micro-limits to each container, Resource Quota manages macro-utilization across the entire project. Resource Quota limits the total amount of computing resources, storage, and other platform objects assigned to the team in the project’s namespace. It acts as a budget within the project. Under the active quota, the system adds all requests and limits from each project deployment together. If creating a new replica would lead to excessive CPU requests, then the OpenShift API server will not approve the project deployment in any case.

This rejection mechanism gives rise to proper friction within engineering organizations. If there is a failure due to quota exhaustion, the developer cannot simply choose to disregard cluster capacity. Instead, he/she must either perform a formal request for a quota increase to platform administrators or check existing deployments to identify inactive pods, oversized replicas, or outdated services.

There are high-level quota strategies that limit the number of objects on top of the computation limits. Developers may leave behind a number of done build pods, orphaned secrets, forgotten persistent volume claims, and stale service definitions. If not taken care of, these resources pile up in the etcd data store, which may lead to responsiveness issues.

Creating a Design Plan for Tiered Multi-Tenancy

Using OpenShift in big businesses might suggest that it is a good idea to set up different clusters depending on the organization’s departments, however, this leads to excessive expenses of control plane resources. Concepts of OpenShift architecture that one can learn at a good advanced openshift course online highlight that tiered multi-tenancy on the shared infrastructure is way more cost-efficient than scattered isolated cluster installations.

An efficient multi-tenancy design classifies business workloads into tiers according to their isolation needs that match risk levels in the organization:

The tasks that take place in non-production environments (such as sandboxes, development zones, and QA) should be performed on shared clusters. Here, the priority is always on maximizing costs, not on performing strict isolation. The typical practice is to run development jobs in common worker pools of servers, which leads to high levels of hardware utilization.

Governance for staging and production tasks ought to be different. In small enterprises, it might be possible to have them share the same cluster, but larger firms tend to run production processes on special systems to comply with security, compliance, and performance standards. Nevertheless, it is still important to use multi-tenancy in production environments. The ability of different important business applications to run on the same worker nodes is possible due to the use of OpenShift isolation projects and necessary network management policies.

The very basis of multi-tenancy is a special automated template. Unlike ordinary systems with manual configuration of namespaces, in multi-tenant environments everything is done automatically. When a project is initiated, the system creates all relevant components together with necessary LimitRanges and ResourceQuotas.

To save on hardware spending, it is essential to implement various labels and taints that allow creating special compute pools. Intensive applications should not use standard servers.

In establishing different machine pools and associating them with node selectors at the namespace level, it becomes possible for admins to avoid situations where costly, high-performing physical machines may be misused by unimportant applications. For instance, a machine learning team may use nodes equipped with GPUs and limited by quotas, while components of the web based microservices architecture may operate on cheap and simple nodes.

Dynamic sizing: Horizontal and Vertical Pod Autoscaling

The problem with static sizing is its inefficacy in an unpredictable environment. For example, the online shop will face a busy lunchtime and a calm midnight, while the company's internal payroll application will be remaining inactive for weeks and then become overloaded for several days in a row due to the payroll account reconciliation process. Therefore, the costs of implementing static system sizes can be very high due to the need of being operable at the peaks of loading in between the prolonged periods of inactivity.

The term "dynamic optimization" refers to two interrelated concepts: the horizontal pod autoscaler and the vertical pod autoscaler.

The Horizontal Pod Autoscaler changes the number of pod copies according to the data received from different sources. In case network traffic increases which leads to greater usage of CPU or other metrics, the autoscaler can increase the total number of copies. And when the situation gets back to normal, the autoscaler reduces the number of copies too and rehabilitates the node capacity.

The true benefits of horizontal autoscaling come from its combination with the basic method of sizing. Instead of utilizing four pods all the time, application teams use merely two small-scaled pods that will only change their size once they reach the threshold rate of utilization given beforehand. Thus, the minimum cost of utilization is achieved while at the same time a possibility of increasing the number of copies during traffic spikes remains intact.

However, the process of scaling horizontally can only be used provided that the application architecture allows scaling across processes. In many cases, namely with some monolithic applications, state-dependent processes, or old platforms, horizontal scaling is not available because of problems with session management and data synchronization.

To perform its functions, the vertical autoscaler relies on real data of a particular container's consumption recorded over time and uses this information to optimize its CPU and memory requests and limits. Thus, the autoscaler eliminates the difficulties associated with the overprovisioning of workloads due to the attempts of tracking the workload's resource needs.

For example, while an application developer asked for eight gigabytes of memory and four CPU cores in an attempt to avoid any resource shortages in the future, the vertical autoscaler has established that the workloads employ only one gigabyte of memory and 200 millicores over the course of many weeks.

The vertical autoscaler is characterized by different modes of operation:

In recommendation mode, it only analyzes the workload and generates the report recommending optimization. This is the most relevant mode for corporate governance, as it offers the evidence to be presented to the application owners regarding the waste of their defensive requests.

The active update mode implies that the system intervenes and deletes the pods that have been using excessive resources.

It is importantly stated that horizontal and vertical autoscalings must not be based on similar metrics for the identical workloads. When both start autoscaling an application based on raw CPU use, they are in competition with each other: as horizontal autoscaler is increasing the number of pods, the vertical one is increasing the current pods, creating unnecessary loops. It is known that horizontal autoscaling is used by microservices being under variable requests and some sorts of vertical-scaling are useful for optimizing basic container definition during the deploying process.

Node efficiency and automated machinery handling

Optimizing container requests and number of pods is only a part of the entire task. Even if a company has reduced cheap resources at the reasonable amount of hundreds of processors cores and terabytes of memory, it may not bring a financial benefit if the organization still uses the limited amount of around a hundred physical or virtual nodes.

The cluster autoscaler observes the kube-scheduler. If there’s a sudden traffic increase that requires horizontal scalers to create new pods, while there are not enough nodes to perform new requests from these pods, the pods will be put in a pending state. In turn, the cluster autoscaler handles these pending jobs by ordering the entire infrastructure provider (which can be AWS, Azure, Google Cloud or a dynamic on-premise virtualization system) to create new nodes and to add them to the cluster.

What is more significant in terms of saving costs, the cluster autoscaler operates in the other way round. It keeps an eye on the current usage of the worker nodes, and once a node has pods, the required resources of which may easily fit into the other nodes, the autoscaler will consider that node as the underused one. Then, it will mark this node and will no longer allow new deployments to that node, moving the currently running pods somewhere else.

To attain aggressive scaling down at the node level, purposeful engineering at the cluster level is required.

Pod Disruption Budgets need to be designed carefully. A Pod Disruption Budget specifies the minimum percentage of replicas that have to be in good health during node maintenance operations. If a team mistakenly creates budgets that require all the pods to be active at all times, the autoscalers won't be able to remove that pod to drain a node with low utilization. The budgets need to allow for some form of disruption during maintenance and reallocation operations.

The choice of storage has direct effects on node elasticity. If a pod is bound to a non-replicated local storage volume on a particular physical machine, the autoscaler cannot move it to a different node. The machine holding that pod cannot be terminated, so it would not matter how small the workload is. Scalable and effective solutions will make use of spread network storage systems, for example, the OpenShift Data Foundation.

Cloud provider limitations can be overcome through the usage of various instance families. To set up machine sets for cluster autoscaling, platform teams need to utilize various instance families which cover different availability zones. In cases when the cloud provider encounters difficulties in the provision of capacity in its region, the usage of only one instance type may fail. The right combination of allowable instance types ensures that the cluster always receives the affordable capacity that it needs.

6. Waste Elimination Approach. Pruning The Growth Of

The OpenShift platform does not only require active microservices to operate which consume computing resources. Gradually, there are accumulations of digital garbage that diminishes performance and increases storage costs of companies. The program of aggressive cost-cutting should address three major elements of hidden waste: obsolete builds, inactive namespaces, and orphaned storage space.

The pipelines of continuous integration and continuous deployment are hefty storage users. Whenever teams take advantage of OpenShift's Source-to-Image capabilities or OpenShift Pipelines, each build generates intermediate build pods, logs, and layers of container images.

If there are no automated lifecycle policies, the integrated container registry will expand constantly, using a costly cloud object storage or block storage system. Platform operators should set up the built-in image pruners, which will continually look through the integrated registry and remove all the unreferenced images, failed build artifacts, and outdated layers over a certain limit.

In the same way, the development environment suffers from permanence. If a developer creates a project to check a bug fix, the developer spins up three databases and some service dependencies and simply abandons the namespace after merging. If no action is taken, the pods will work forever and will not stop eating cluster reservations for months.

Tech-savvy organizations practice automated temporary character in their non-production environments:

The Time-to-Live operators are to be used for the non-production namespaces. As the system creates a development project, it also assigns expiration metadata tags automatically. In case project developers do not extend the expiration of their projects internally, the entire project will be erased automatically within fourteen or thirty days.

The automated mode of hibernation makes off-time economics possible. In standard business environments, the teams invest about eight to ten hours a day in their work week. Keeping their development clusters running at full capacity during the evenings and weekends, companies spend money on unused computing time for one hundred and twenty-eight hours for each week.

With the help of the cron-based scaling controllers, all the non-production deployments can be scaled down and up automatically.

In conclusion, Persistent Volume Claims (PVCs) incur ongoing costs without any oversight. Deleting StatefulSets or namespaces leads to the associated volumes that are retained, but no longer used, to be still allocated by the cloud provider and incur monthly costs. Therefore, the platform admin must run automated auditing scripts to find and remove orphaned volumes from the entire cluster.

Technical mechanisms such as quotas and auto-scaling systems do provide the tools needed for efficiency, but cultural changes are what enable long-term adoption. A company’s investment in hands-on openshift training online helps both DevOps engineers and application developers learn how to implement lifecycle management practices, autoscaling technology, and effective FinOps in their organizations.