(helm/v2-alpha plugin) Allow customizing pod spec fields (e.g. `priorityClassName`)

[camilamacedo86]

Problem

The Helm chart generated by the helm/v2-alpha plugin exposes some pod configuration through the values.manager.<value> pattern. However, certain pod spec fields such as priorityClassName are not currently configurable.

This makes it harder for users to customize scheduling or other pod settings without modifying the generated chart templates.

Proposal

We could consider one of the following approaches:

• Add support for specific fields such as priorityClassName.
• Introduce a more flexible option such as extraPodSpec that allows users to extend the pod spec with additional fields not explicitly exposed in the chart values (similar to patterns used in other Helm charts).
• Explore dynamically exposing options only when they exist in the kustomize manifests to avoid divergence.

The goal would be to keep the Helm chart aligned with the kustomize manifests, while still allowing users to customize pod settings when needed.

Additional Context

Suggestion originally raised by @liam Mackie in Slack: https://kubernetes.slack.com/archives/CAR30FCJZ/p1773631814394009

[cmallikarjunah]

Hi @camilamacedo86 @liam-mackie
I was thinking how the extra (but frequently used) pod spec could be exposed in helm charts given that it should align with kustomize manifest. This is how I understand it could work in the simplest way (Pls do correct me if my understanding is not correct. I'm just starting to deep dive into the kubebuilder code):

1. To configure fields like priorityClassName in a deployment, additional resource need to be created: PriorityClass resource. User applies customizations (like addition of any extra Pod Spec and new resources as required) via Kustomize patches in the config/ directory in the Kubebuilder scaffolded project.
2. The user attempts to generate the helm charts using:
   `kubebuilder edit --plugins=helm/v2-alpha
3. The kustomize output dist/install.yaml file is generated based on the project's config/ including the custom patches.
4. The kubebuilder kustomize output to helm chart converter code should now support the new fields i.e; identify these new fields and reflect in helm charts accordingly.

[camilamacedo86]

I think we can map all that the Pod, Deployment can have that would make sense to expose in the helm values, and then only add to the helm values if found in the yaml. That seems a good strategy.

[cmallikarjunah]

@liam-mackie Are you still working on this? If not I would like to pick it up and try it out. Thanks!

[liam-mackie]

@liam-mackie Are you still working on this? If not I would like to pick it up and try it out. Thanks!

@cmallikarjunah - I had a bit done, but I've been at KubeCon EU all week, and underestimated my availability (as always). Happy for you to take a look!

The main thing I'm hoping for is to give consumers of the resultant helm chart the ability to modify the spec in a way that the maintainers of the built controller did not anticipate. As an example, I have a controller, and I publish the helm chart. It's simple, so I don't include a priority class. When someone wants to use my controller, they let me know that they need to have a priority class assigned, otherwise, Gatekeeper will not allow the pod to be scheduled.

As a maintainer of the controller, I don't want to have to do work, and update my kustomize, for each individual users use case - especially if it's got nothing to do with what the controller actually needs to run.

[camilamacedo86]

Hi @liam-mackie

Thank you for your input.
We have here more than one category of things, IHMO, they are:

a) Specs that should be exposed by default always
b) Specs that never should be exposed since they make no sense (end users should never change those)
c) Specs that should be exposed only when we find them in the Pod/Deployment.

It would be nice if we could map and categorise all so we can address those.

[cmallikarjunah]

@camilamacedo86

Here is the list of spec categorized as per the above meaning:
a) Specs that should be exposed by default always and are currently exposed:

• replicas
• containers.image
• containers.args
• imagePullSecrets
• containers.SecurityContext
• securityContext
• readOnlyRootFilesystem
• containers.resources(Resource limits and requests)
• containers.imagePullPolicy

b)The following specs most probably should never be exposed:
livenessProbe, readinessProbe, selector, template, containers.command, containers.name, restartPolicy

c) Specs that should be exposed only when we find them in the Pod/Deployment

• initContainers, ephemeralContainers
• hostname, dnsConfig, dnsPolicy, hostIPC, hostNetwork
• restartPolicy
• strategy (for a deployment)
• affinity, nodeSelector, tolerations, priorityClassName, nodeName, topologySpreadConstraints, schedulerName:
  I think these fields can be exposed only if found in pod/deployment. Currently affinity, nodeSelector and tolerations are already exposed, but I'm sure there is a reason for that.

[cmallikarjunah]

@liam-mackie To be able to give consumers of the resultant helm chart the ability to modify the spec, the generated helm templates can include the extra pod spec within conditional blocks that check if the corresponding value is present in values.yaml. So then the consumer just needs to define the spec values in values.yaml without the maintainer having to do anything.

For the purpose of this issue, we need to add the logic to generate the conditional blocks for extra pod spec irrespective of the kustomize output for helm plugin - hope I have understood it right.

[liam-mackie]

For the purpose of this issue, we need to add the logic to generate the conditional blocks for extra pod spec irrespective of the kustomize output for helm plugin - hope I have understood it right.

That sounds perfect @cmallikarjunah - I'm happy to contribute this, as I've got a lot of previous work in this area, otherwise, I can help with reviews if you're keen on writing code?

[cmallikarjunah]

@liam-mackie Yes, I can try implementing this—it’ll help me get more familiar with the project. I will work on it over the next couple of days and open a draft PR for feedback. No problem if you’d prefer to take this on due to timelines and other requirements.

[cmallikarjunah]

Going ahead assigning to myself.
/ assign