Skip to content

[Tests] Reorganize final exception handling#71557

Merged
mergify[bot] merged 2 commits intoproject-chip:masterfrom
MarekPikula:reorg-test-final-exc
Apr 14, 2026
Merged

[Tests] Reorganize final exception handling#71557
mergify[bot] merged 2 commits intoproject-chip:masterfrom
MarekPikula:reorg-test-final-exc

Conversation

@MarekPikula
Copy link
Copy Markdown
Contributor

@MarekPikula MarekPikula commented Apr 14, 2026

Summary

Small improvements to the final exception handling in YAML tests:

  • Avoid printing stacktrace multiple times in logs.
  • Don't pollute the stacktrace when reraising a single exception.
  • Add name of resource that failed cleaning to the log.

Related issues

Submitted along:

Testing

Manual tests with run_test_suite.py and injected exceptions in various places to check for the behavior.

Readability checklist

The checklist below will help the reviewer finish PR review in time and keep the
code readable:

  • PR title is
    descriptive
  • Apply the
    “When in Rome…”
    rule (coding style)
  • PR size is short
  • Try to avoid "squashing" and "force-update" in commit history
  • CI time didn't increase

See: Pull Request Guidelines

- Avoid printing stacktrace multiple times.
- Don't pollute the stacktrace when reraising a single exception.
- Add name of resource that failed cleaning to the log.

Signed-off-by: Marek Pikuła <m.pikula@partner.samsung.com>
Copilot AI review requested due to automatic review settings April 14, 2026 10:11
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the exception handling in the test suite runner to aggregate errors from both execution and cleanup phases using BaseExceptionGroup. While the goal is to improve error reporting, the implementation introduces several issues: the use of the match statement and BaseExceptionGroup breaks compatibility with Python versions older than 3.10 and 3.11 respectively. Furthermore, catching BaseException incorrectly captures SystemExit, which is then inappropriately wrapped in a RuntimeError, and the use of raise ... from error adds unnecessary layers to the stack trace. Finally, removing exc_info=True from cleanup logs makes debugging resource termination failures more difficult.

Comment thread scripts/tests/run_test_suite.py
Comment thread scripts/tests/run_test_suite.py
Comment thread scripts/tests/run_test_suite.py
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refines exception handling in run_test_suite.py to reduce duplicated stack traces and improve cleanup logging for YAML test runs.

Changes:

  • Consolidates execution exception capture into a single errors list and defers final raising logic until after cleanup.
  • Improves cleanup logging by including the resource name being terminated.
  • Adds special-case handling for single errors (KeyboardInterrupt / ResultError) and uses BaseExceptionGroup when multiple errors occur.

Comment thread scripts/tests/run_test_suite.py
Comment thread scripts/tests/run_test_suite.py Outdated
Signed-off-by: Marek Pikuła <m.pikula@partner.samsung.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 14, 2026

PR #71557: Size comparison from 04ec448 to e998699

Full report (34 builds for bl602, bl616, bl702, bl702l, cc13x4_26x4, cc32xx, efr32, esp32, nrfconnect, nxp, psoc6, qpg, realtek, stm32, telink)
platform target config section 04ec448 e998699 change % change
bl602 lighting-app bl602+mfd+littlefs+rpc FLASH 1090500 1090500 0 0.0
RAM 144858 144858 0 0.0
bl616 lighting-app bl616+thread FLASH 1101988 1101988 0 0.0
RAM 104280 104280 0 0.0
bl616+wifi+shell FLASH 1588868 1588868 0 0.0
RAM 98176 98176 0 0.0
bl702 lighting-app bl702+eth FLASH 1053666 1053666 0 0.0
RAM 108461 108461 0 0.0
bl702l contact-sensor-app bl702l+mfd+littlefs FLASH 892364 892364 0 0.0
RAM 105852 105852 0 0.0
cc13x4_26x4 lighting-app LP_EM_CC1354P10_6 FLASH 775904 775904 0 0.0
RAM 103404 103404 0 0.0
lock-ftd LP_EM_CC1354P10_6 FLASH 788108 788108 0 0.0
RAM 108588 108588 0 0.0
pump-app LP_EM_CC1354P10_6 FLASH 734432 734432 0 0.0
RAM 97396 97396 0 0.0
pump-controller-app LP_EM_CC1354P10_6 FLASH 717860 717860 0 0.0
RAM 97556 97556 0 0.0
cc32xx air-purifier CC3235SF_LAUNCHXL FLASH 560842 560842 0 0.0
RAM 204608 204608 0 0.0
lock CC3235SF_LAUNCHXL FLASH 592730 592730 0 0.0
RAM 204816 204816 0 0.0
efr32 lock-app BRD4187C FLASH 992512 992512 0 0.0
RAM 131268 131268 0 0.0
BRD4338a FLASH 796281 796281 0 0.0
RAM 243372 243372 0 0.0
window-app BRD4187C FLASH 1098000 1098000 0 0.0
RAM 130308 130308 0 0.0
esp32 all-clusters-app c3devkit DRAM 98556 98556 0 0.0
FLASH 1600484 1600484 0 0.0
IRAM 93514 93514 0 0.0
nrfconnect all-clusters-app nrf52840dk_nrf52840 FLASH 863484 863484 0 0.0
RAM 162186 162186 0 0.0
nxp contact mcxw71+release FLASH 739032 739032 0 0.0
RAM 67016 67016 0 0.0
psoc6 all-clusters cy8ckit_062s2_43012 FLASH 1713300 1713300 0 0.0
RAM 214124 214124 0 0.0
all-clusters-minimal cy8ckit_062s2_43012 FLASH 1610708 1610708 0 0.0
RAM 210996 210996 0 0.0
light cy8ckit_062s2_43012 FLASH 1466844 1466844 0 0.0
RAM 197076 197076 0 0.0
lock cy8ckit_062s2_43012 FLASH 1499564 1499564 0 0.0
RAM 224820 224820 0 0.0
qpg lighting-app qpg6200+debug FLASH 842716 842716 0 0.0
RAM 127868 127868 0 0.0
lock-app qpg6200+debug FLASH 781332 781332 0 0.0
RAM 118816 118816 0 0.0
realtek light-switch-app rtl8777g FLASH 680920 680920 0 0.0
RAM 101616 101616 0 0.0
lighting-app rtl8777g FLASH 724816 724816 0 0.0
RAM 101956 101956 0 0.0
stm32 light STM32WB5MM-DK FLASH 475468 475468 0 0.0
RAM 141396 141396 0 0.0
telink bridge-app tl7218x FLASH 730822 730822 0 0.0
RAM 95924 95924 0 0.0
light-app-ota-compress-lzma-shell-factory-data tl3218x FLASH 850332 850332 0 0.0
RAM 44344 44344 0 0.0
tl7218x FLASH 841738 841738 0 0.0
RAM 99728 99728 0 0.0
light-switch-app-ota-compress-lzma-factory-data tl7218x_retention FLASH 730292 730292 0 0.0
RAM 56004 56004 0 0.0
light-switch-app-ota-compress-lzma-shell-factory-data tlsr9528a FLASH 793518 793518 0 0.0
RAM 75180 75180 0 0.0
light-switch-app-ota-factory-data tl3218x_retention FLASH 730236 730236 0 0.0
RAM 33488 33488 0 0.0
lighting-app-ota-factory-data tlsr9118bdk40d FLASH 613132 613132 0 0.0
RAM 118400 118400 0 0.0
lighting-app-ota-rpc-factory-data-4mb tlsr9518adk80d FLASH 839872 839876 4 0.0
RAM 97436 97436 0 0.0

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 14, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 54.38%. Comparing base (7363906) to head (e998699).
⚠️ Report is 25 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #71557      +/-   ##
==========================================
+ Coverage   54.34%   54.38%   +0.04%     
==========================================
  Files        1577     1580       +3     
  Lines      108287   108385      +98     
  Branches    13399    13388      -11     
==========================================
+ Hits        58844    58950     +106     
+ Misses      49443    49435       -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@andy31415 andy31415 added the sdk-maintainer-approved PR marked by `matter-sdk-maintainers` as suitable for MERGE - meets guideline & sufficient reviews. label Apr 14, 2026
@mergify mergify Bot merged commit 9c12d49 into project-chip:master Apr 14, 2026
83 checks passed
@MarekPikula MarekPikula deleted the reorg-test-final-exc branch April 14, 2026 17:02
raul-marquez-csa pushed a commit to raul-marquez-csa/connectedhomeip that referenced this pull request Apr 30, 2026
* Reorganize final exception handling

- Avoid printing stacktrace multiple times.
- Don't pollute the stacktrace when reraising a single exception.
- Add name of resource that failed cleaning to the log.

Signed-off-by: Marek Pikuła <m.pikula@partner.samsung.com>

* Fix reraising single exception

Signed-off-by: Marek Pikuła <m.pikula@partner.samsung.com>

---------

Signed-off-by: Marek Pikuła <m.pikula@partner.samsung.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review - pending scripts sdk-maintainer-approved PR marked by `matter-sdk-maintainers` as suitable for MERGE - meets guideline & sufficient reviews.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants