Kubernetes v1.36 introduces a noteworthy alpha metric designed to give cluster operators better visibility into route synchronization with cloud providers. This new metric, route_controller_route_sync_total, is added to the Cloud Controller Manager (CCM) route controller implementation. It directly supports the experimental watch-based reconciliation feature gate introduced in v1.35, allowing for efficient A/B testing between the legacy polling method and the new event-driven approach. In this listicle, we break down the five most important things you need to know to take advantage of this enhancement.
1. A Fresh Alpha Metric: route_controller_route_sync_total
The new alpha counter, route_controller_route_sync_total, lives in the k8s.io/cloud-provider package. Every time the CCM synchronizes routes with the underlying cloud provider, this metric increments. It provides a simple but powerful way to track exactly how often route reconciliation occurs. By exposing this counter, Kubernetes gives operators a direct tool to measure the impact of the new watch-based reconciliation feature, helping them decide whether to enable it in their clusters.
2. Watch-Based Route Reconciliation: What Changed in v1.35
Before v1.35, the route controller operated on a fixed-interval loop, constantly querying the cloud provider for route updates regardless of whether any node had changed. This generated unnecessary API calls, consuming rate limits and quota. The CloudControllerManagerWatchBasedRoutesReconciliation feature gate (alpha in v1.35) introduced a smarter approach: the controller now watches for actual node changes via Kubernetes events and only triggers route reconciliation when nodes are added, removed, or updated. This reduces the number of API calls to the infrastructure provider, alleviating pressure on rate-limited endpoints and allowing operators to use their available quota more efficiently.
3. How to A/B Test the New Feature with the Metric
The route_controller_route_sync_total metric makes it straightforward to compare the old and new reconciliation methods. Run two clusters (or two CCM instances) – one with the feature gate disabled (default) and one with it enabled. Then monitor the metric values over time. If your cluster experiences infrequent node changes, you should observe a substantial drop in the sync rate when the watch-based approach is active. This direct comparison helps validate that the feature is working as intended and provides data to support enabling it in production.
4. Expected Behavior: Polling vs. Watch-Based Reconciliation
To illustrate the difference, consider a stable cluster with no node modifications. With the feature gate disabled (fixed-interval loop), the counter increments every few seconds even if nothing changed:
- After 10 minutes:
route_controller_route_sync_total 60 - After 20 minutes:
route_controller_route_sync_total 120
- After 10 minutes with no changes:
route_controller_route_sync_total 1 - After 20 minutes still no changes:
route_controller_route_sync_total 1(unchanged) - When a new node joins:
route_controller_route_sync_total 2
5. Getting Involved: Feedback and Further Reading
If you want to test the new metric or the watch-based reconciliation feature, you can share your experiences and suggestions through the Kubernetes community. Reach out on the #sig-cloud-provider Slack channel, discuss the design in KEP-5237 on GitHub, or visit the SIG Cloud Provider community page for additional communication channels. For deep technical details, refer directly to KEP-5237, which documents the proposal and implementation.
In summary, the addition of route_controller_route_sync_total in Kubernetes v1.36 marks a significant step toward more efficient and observable route synchronization. Whether you are an operator looking to reduce API load or a developer evaluating the new watch-based reconciliation, this metric provides the insight you need. By following the A/B testing approach outlined above, you can confidently adopt the feature and optimize your cloud provider interactions.