Most teams managing multiple Kubernetes clusters eventually hit a wall with static configuration management. When you run an application across ten clusters using standard continuous delivery tools, you usually end up hardcoding placement rules, managing duplicate manifests, and juggling a dozen different configuration contexts. If a single target cluster goes offline, workloads do not automatically migrate.
A Karmada multi-cluster setup solves this exact scheduling problem natively. It replaces static configuration with a dynamic control plane that treats multiple physical clusters like individual nodes. It shifts the burden of multi-cluster placement from the engineer writing the deployment pipeline to a dedicated Kubernetes scheduling engine.
Why kubectl and standard GitOps fall short
Tools like Argo CD and Flux are excellent for declarative delivery. They pull manifests from a Git repository and apply them directly to a target cluster. However, they do not maintain global state awareness across separate runtime targets.
If an application needs to span three separate clusters based on available CPU or node count, a GitOps controller cannot do that math. It only knows what is written in the repository. If a target cluster fails, a standard GitOps pipeline will not automatically redistribute those pods to a healthy environment. You have to manually update the repository to change the deployment target, wait for the sync loop, and monitor the rollout.
If your multi-cluster strategy relies entirely on active-passive disaster recovery, static GitOps routing is usually sufficient. But if you want to run active-active workloads geographically, route traffic based on compute availability, or automatically rebalance workloads when a cluster fills up, you need a true orchestrator.
Karmada takes this orchestration approach. It introduces a federated control plane. You submit standard Kubernetes manifests once. Karmada evaluates the available capacity across all member clusters and decides where those workloads should go based on real-time metrics.
The virtual control plane architecture
The Karmada control plane looks and acts exactly like a standard Kubernetes cluster. It has an API server, a controller manager, and a scheduler. You interact with it using standard tools and commands.
The key difference is that this control plane is entirely virtual. It does not run application pods. It only orchestrates them. It stores your deployments, config maps, and scheduling policies in its own data store, then computes how to distribute them to the physical member clusters doing the actual work.
You can connect physical member clusters to this control plane in two distinct ways, depending on your network topology:
- Push mode: The Karmada control plane connects directly to the member cluster API server. The Karmada control plane needs direct network access to the member cluster. This works well within a single cloud provider using private network peering or transit gateways.
- Pull mode: A component runs inside the member cluster and reaches out to the central control plane to pull down instructions. This is necessary when member clusters sit behind strict firewalls, network address translation, or in isolated edge environments where inbound API traffic is blocked.
The propagation lifecycle and scheduling math
Karmada separates the workload definition from the placement rules. This is a fundamental shift from how tools like Helm manage placement through templated values files.
First, you submit a standard Kubernetes manifest to the Karmada API. Next, you submit a policy object. A namespace-scoped resource uses a PropagationPolicy, while a cluster-scoped resource uses a ClusterPropagationPolicy.
This custom resource defines your scheduling constraints. You can map one policy to multiple workloads using label selectors, which prevents configuration sprawl. When configuring a propagation policy, you must choose a replica scheduling type.
The scheduling engine evaluates your configuration using two primary modes:
- Duplicated scheduling: If you ask for 5 replicas and target 3 clusters, Karmada will place 5 replicas in every cluster. You get 15 pods total. This is common for high-availability microservices running in multiple availability zones.
- Divided scheduling: If you ask for 15 replicas and target 3 clusters, Karmada splits them up. If all clusters have equal capacity, it places 5 in each. If one cluster is nearly full, Karmada might place 10 pods in the largest cluster, 4 in the second, and 1 in the smallest based on cluster resource modeling.
Once the policy is evaluated, the scheduler creates a ResourceBinding object. This object records the mathematical decision of where the workload should go.
The execution controller then steps in. It watches for these bindings and creates a Work object. This object acts as a carrier envelope containing the exact resource manifest that needs to run on the target cluster. Finally, Karmada dispatches the object to the target cluster execution space, and the actual pods spin up.
Managing configuration asymmetry via overrides
Running the exact same manifest in every cluster is rarely practical. Different environments often require different configurations.
A cluster running in one cloud provider might need a specific ingress class annotation for a particular cloud load balancer. A cluster in a different provider might need a distinct storage class for its persistent volumes. An edge cluster might need to pull images from a local proxy registry instead of the public internet.
If you handle this with Helm, you end up with complex nested values files for every environment. Karmada handles this cleanly using an OverridePolicy.
An override policy intercepts the resource before it is dispatched to the member cluster and mutates it on the fly. You can write a policy that alters the storage class designation exclusively when deploying to a specific target cluster environment. The base deployment manifest remains untouched in the virtual control plane, but the generated work object delivered to that target cluster contains the modified storage configuration.
This separation of concerns keeps your core application definitions clean while delegating environment-specific quirks to centralized policies.
Combining GitOps with federation pipelines
Engineers often assume they have to choose between GitOps delivery tools and Karmada. In practice, the best architectural pattern combines them.
Instead of pointing your continuous delivery tool at ten different production clusters, you point it exclusively at the virtual API server. The delivery tool watches your Git repository and synchronizes your standard deployments, propagation policies, and override policies to the central control plane.
The federation layer takes over from there. It interprets those policies, calculates cluster capacity, and pushes the workloads to the physical clusters.
This pattern gives you the auditability and declarative drift detection of GitOps, combined with the dynamic resource scheduling and automated failover capabilities of a federated orchestrator. It simplifies the pipeline configuration drastically because the deployment pipeline only needs to track a single management target.
Cluster failures, evictions, and split-brain
Multi-cluster management gets difficult when network partitions happen. Karmada handles cluster failures using a taint and eviction model heavily inspired by standard core node management.
Member clusters continuously report their status to the control plane. When a cluster drops offline or experiences severe API latency, the controller detects the timeout and adds a NoExecute taint to the cluster object. The system then evaluates your configured eviction policy to decide how to handle the orphaned workloads.
Karmada supports two primary eviction behaviors:
- Graceful eviction: The system waits for the application to become healthy on a new cluster before removing the old records. This prevents service interruptions for stateless applications, ensuring you do not drop traffic while waiting for new pods to pass their readiness probes.
- Direct eviction: The system immediately evicts the application. This is critical for stateful workloads like Apache Flink or distributed databases. If a stateful application runs in two places at once during a network partition, it can cause split-brain scenarios and corrupt data. Direct eviction ensures the old instance is fully terminated before a new one spins up, maintaining processing consistency.
A solid understanding of resource boundaries and right-sizing limits is necessary before federating workloads to ensure target clusters have the capacity to accept evicted tasks during a failure.
The operational cost of a second control plane
Every architectural choice has a cost. Karmada introduces a clear operational burden.
You are running a second control plane. This means managing another data store, securing another API server, handling separate access control permissions, and monitoring a new set of controllers. It adds a layer of abstraction that makes debugging harder. If a pod fails to start, operators must check the member cluster events, the central work objects, and the scheduler logs to find the root cause.
Failover storms are another operational risk. If a large cluster fails, the federation layer might need to reschedule thousands of workloads simultaneously. This sudden burst of API calls can overwhelm both the central control plane and the API servers of the remaining healthy clusters.
To manage this pressure, operators must configure rate limiting using the resource-eviction-rate parameter on the controller manager. The default value is 0.5, meaning it evicts roughly one workload every 2 seconds. If you set this rate to 2, approximately 2 resources will be evicted per second. This throttles the eviction process, intentionally trading a longer recovery time for overall control plane stability. Engineers must tune this parameter based on the specific load limits of their multi-cluster infrastructure.
