A new way report issues with integrations and see them before HA upgrades

[flygekorren]

Describe the feature

If you upgrade your HA installation to latest version and a integration breaks you could have a button to report it within HomeAssistant. We could then have a system that decides if there is a major issue with more than X users or something like that and if this gets "true" then the integration will be marked as "Defective" on the selected version on your integrations page, if you are not on the latest version already

Use cases

You can se beforehand if there are issues with your particular integrations directly from your integrations page when there is an update of Home assistant available

You wont have to search the forums or be afraid to upgrade

The Fortums will propbably be cleaner or less flooded with people that upgraded and want help, instead they can now have the ability to see the issues BEFORE they upgrade and wait until the state of the defective integrations changes on the integrations page

You minimize the risk and headaces when things go wrong. Some users setups are pretty delicate and in some cases control power, solar systems and other things that can go seriously wrong if stuff breaks.

It also makes the whole reporting system more centralized and could probably be integrated directly to github in some way. Everything more streamlined and easier!

Current workarounds

No response

Anything else?

No response

[mik-laj]

I’m skeptical about this one. While it sounds helpful, it’s a potential minefield:

• We already have safety nets: Between rigorous CI/CD testing for core integrations and the "Backup before Update" feature, we already have a solid way to handle bad updates. If it breaks, just roll back.
• The "Custom Component" mess: Custom integrations come with no guarantees. If we add an official "report" button, users will expect the HA team to fix things they don't even own. It’s a support nightmare.

[flygekorren]

I respect that, I understand that if done wrong this could get tough to manage, but I also think this is an important issue and this would be a step to make it safer. Also there maybe is a way doing this to that it wouldnt so big of a mess?

While the rollback feature solves the problem if it happens it will still not prevent the problem from happening. This is where I see a potential to get better. In worst case I will have to find out myself that my modbus integration failed after a upgrade and it did not shut down my inverter from charging my 20kWh lifepo4 batterybank and potentially harm the batteries.

Of course when you deal with power and other things that could get dangerous you should have multiple backup plans IF home assistant doesnt do its job. But still it would be nice to make the whole update concept sturdier.

[mik-laj]

I believe there is a very clear line here. The Home Assistant community provides updates without any guarantee of functionality. It is the user who is ultimately responsible for ensuring that the code works within their specific, often unique, environment.

On the development side, we do everything we can by using rigorous CI/CD testing to ensure updates are as flawless as possible. However, managing tickets and bug reports is purely based on the goodwill of the integration maintainers. Some are highly active, while others rely on the community to troubleshoot "teething" issues.

While you suggest this system would be driven by user reports, it ultimately forces a "vetting" burden onto the maintainers. If a few users incorrectly report an issue, the maintainer is the one who has to spend time investigating, proving it’s not a bug, and managing the "defective" status to clear their integration's reputation.

This adds a massive layer of administrative overhead to people who are already donating their time for free.
You should always assume that if you update, you must verify it yourself. The fact that an update works for others doesn't guarantee it will work in your specific configuration.

As mentioned, HA is a community-driven project shared in the spirit of a hobby. While it is powerful, it shouldn't be treated as a mission-critical, industrial-grade system out of the box. Users must be aware of this and implement hardware-level fail-safes for high-risk systems like energy storage. Ultimately, we provide the tools to make management easier, but we do not take responsibility for the actions you take within your own system.

[mik-laj]

If I were in your position, I would focus on mitigation to make the update process safer for your setup:

• Aim for Platinum: I’d check if the integrations I rely on have a "Platinum" quality scale rating. If not, I’d contribute to improving them. Higher quality ratings mean better test coverage and fewer "teething" issues.
• Fixing Community Issues: I’d look at existing bug reports for my integrations. Even if a bug doesn't affect me now, it might after an update. Fixing it early is the best way to ensure long-term stability.
• The "Wait" Rule: Avoid .0 releases. Wait for .1 or .2 patches so the initial bugs can be cleared out first.
• Staging: Run a secondary HA instance on a separate device to test your Modbus configs before touching the production system.
• Health Checks: Set up an automation to alert you immediately if critical integrations (like Modbus) stay "unavailable" for more than 2 minutes after a restart.
• Hardware Fail-safes: Ensure the inverter and BMS have their own hard-coded limits. HA should be the "brain," but the safety layer must be independent of the software.
• Stability over Features: If your current version manages the batteries perfectly, prioritize that stability over monthly updates.

[flygekorren]

I understand your concerns and agree that this shouldn't be another burden or something big that has to be extra maintained. But I'm still pretty sure that there is room for improvement without any major sidequests. And wether we like it or not, there will always be installations out there that have potentially dangeropus outcomes if anything goes wrong, HA is just too smart for people out there to not try to schedule a heater when they are away or anything like that. This would be one way to make it safer or at least make people more aware. With that said I agree with all your recommendations above fully.

You write: "While you suggest this system would be driven by user reports, it ultimately forces a "vetting" burden onto the maintainers."
What if we can use what already there? We already have all information on github. Isn't there some way to fetch the state of more recent issues and get that input into a green orange red flashing light in the integrations page that shows before an upgrade? It could measure recent issues x amount of entries or something like that? We already have door open with the link to "known issues" from each integration page in HA.

All im saying is that IF it could be done in a relatively easy way then I think many would benefit from it. Im of course one of those getting tired of stuff breaking lately. It would be a lot easier to get this error indication before an upgrade instead of searching beforehand.