feat: enable use of CEL expressions for CR valueFrom#2868
feat: enable use of CEL expressions for CR valueFrom#2868alexandernorth wants to merge 13 commits intokubernetes:mainfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: alexandernorth The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
needs-triage
|
This issue is currently awaiting triage. If kube-state-metrics contributors determine this is a relevant issue, they will accept it by applying the The DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
the
cncf-cla: yes
k8s-ci-robot
added
the
size/XXL
alexandernorth
changed the title
|
Is there any chance you could provide a sample CEL expression on picking up an array? So this part: |
Hi @amy as discussed in the slack channel, I added documentation for using CEL Expressions in docs/metrics/extend/customresourcestate-metrics.md#cel-expressions. Hope this helps! |
alexandernorth
force-pushed
the
feature/custom-resource-cel-expressions
branch
from
c1cfeb2 to
516d53a
Compare
amazing! Hope this gets merged 🤞 |
There was a problem hiding this comment.
Pull request overview
Adds CEL (Common Expression Language) support to CustomResourceState Gauge valueFrom, enabling expression-based value extraction/transformation and dynamic labels (via WithLabels) while preserving the existing path-based behavior for backward compatibility.
Changes:
- Introduces a
ValueFromconfig type that supports eitherpathValueFrom(legacy path extraction) orcelExpr(CEL extraction). - Refactors Gauge value extraction behind a
valueExtractorstrategy with separate path-based and CEL-based implementations. - Adds a CEL helper library (
WithLabels) plus extensive unit tests and documentation updates.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/customresourcestate/registry_factory.go | Wires Gauge compilation to a value-extractor strategy and updates StateSet to use the new ValueFrom type. |
| pkg/customresourcestate/registry_factory_test.go | Updates/reshapes tests; adds an error-case test for invalid CEL+labelFromKey config. |
| pkg/customresourcestate/path_value_extractor.go | Extracts prior path-based Gauge value extraction logic into a dedicated strategy. |
| pkg/customresourcestate/path_value_extractor_test.go | Re-homes and expands path-based extraction tests. |
| pkg/customresourcestate/interfaces.go | Introduces the valueExtractor interface. |
| pkg/customresourcestate/config_metrics_types.go | Adds the ValueFrom struct and (JSON/YAML) unmarshalling to preserve legacy config shapes. |
| pkg/customresourcestate/config_metrics_types_test.go | Tests ValueFrom unmarshalling for JSON/YAML, including CEL form. |
| pkg/customresourcestate/cel_value_extractor.go | Implements CEL-based extraction and label merging behavior. |
| pkg/customresourcestate/cel_value_extractor_test.go | Adds coverage for WithLabels, conversions, real-CR-like examples, and compilation failures. |
| pkg/cel/withlabels.go | Defines the WithLabels CEL value type. |
| pkg/cel/library/library.go | Registers WithLabels as a CEL function in a KSM CEL library. |
| pkg/cel/README.md | Documents the CEL extension library and WithLabels. |
| go.mod | Promotes cel-go to a direct dependency. |
| docs/metrics/extend/customresourcestate-metrics.md | Adds a CEL quick start and detailed CEL configuration documentation for CustomResourceState Gauges. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -196,7 +196,8 @@ func newCompiledMetric(m Metric) (compiledMetric, error) { | |||
| if err != nil { | |||
| return nil, fmt.Errorf("each.stateSet: %w", err) | |||
| } | |||
| // Ensure fields are mutually exclusive | ||
| if len(valueFromStruct.PathValueFrom) > 0 && valueFromStruct.CelExpr != "" { |
| // unmarshallValueFrom unmarshalls ValueFrom from either a string slice or struct. | ||
| func (vf *ValueFrom) unmarshallValueFrom(unmarshal func(interface{}) error) error { | ||
| var stringSlice []string |
| type compiledGauge struct { | ||
| compiledCommon | ||
| labelFromKey string | ||
| ValueFrom valuePath | ||
| extractor valueExtractor | ||
| NilIsZero bool | ||
| } |
| labels := make(map[string]string) | ||
| addPathLabels(v, s.labelFromPath, labels) | ||
| // Apply AdditionalLabels last so they are not overwritten | ||
| for k, v := range value.Labels { | ||
| labels[k] = v | ||
| } | ||
| value.Labels = labels |
What this PR does / why we need it:
Adds CEL (Common Expression Language) support to Custom Resource State metrics for transforming and extracting values from custom resources.
CEL expressions enable more complex calculations, conditional logic, array/object iteration, and dynamic label generation directly in metric configurations, with a low cost overhead. This is more flexible than the existing path-based approach for complex data structures (multi-level, iteration, etc).
Examples:
Static number:
Iterate over arrays adding dynamic labels:
Path-based valueFrom continues to work unchanged for backwards compatibility.
The documentation is also updated to include examples and details on the implementation customresourcestate-metrics.md
How does this change affect the cardinality of KSM: (increases, decreases or does not change cardinality)
None.
Which issue(s) this PR fixes: (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged)I did not find a specific comment, but it does solve a TODO regarding multi-length valueFrom paths: #1958 (comment)