Limit global test pipeline options to integration tests

[aIbrahiim]

Fixes: #36127

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refines how test pipeline options are handled within the Apache Beam Python SDK. By ensuring that global test pipeline options are only applied when running integration tests, the change prevents unintended configuration leakage during standard unit test execution, addressing the issue identified in #36127.

Highlights

• Pipeline Option Scoping: Updated the test pipeline to restrict the application of global test pipeline options exclusively to integration tests.
• Documentation Cleanup: Removed trailing whitespace in the TestPipeline class docstring.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[Abacn]

Would you mind sharing some context? Looks like the test has been been flaky though not permared on HEAD. How this is related to the flaky test?

[aIbrahiim]

Would you mind sharing some context? Looks like the test has been been flaky though not permared on HEAD. How this is related to the flaky test?

Yeah sure happy to add more context and sorry forgot to add that on the PR description as well, so the stack trace looked like the ApproximateUnique snippet test was just timing out / flaking, but I dont think the transform itself was the real problem so what was happening is those snippet tests use TestPipeline under the hood, and TestPipeline was still inheriting the global --test-pipeline-options that pytest sets up for the cloud job. so sometimes you would get this tiny unit snippet test accidentally running with runner settings meant for heavier integration-style runs, and then you hit weird gRPC / deadline stuff under load which matches what was found so the fix is basically is only use those global pytest pipeline options for tests that are actually integration tests (is_integration_test=True) and by that way snippet tests stay on the normal local path, and we dont get this accidental coupling to remote runner config @Abacn

[Abacn]

I don't see it's being setup for cloud etc

beam/.github/workflows/beam_PreCommit_Python_Examples.yml

Line 97 in 4e24a9c

gradle-command: :sdks:python:test-suites:tox:py${{steps.set_py_ver_clean.outputs.py_ver_clean}}:preCommitPy${{steps.set_py_ver_clean.outputs.py_ver_clean}}

Do you mean TestPipelines.pytest_test_pipeline_options could get mutated by one test and affecting other tests? Nor this is immutable as it's a str

[Abacn]

DEADLINE_EXCEEDED is due to the tests run on PrismRunner and involves grpc, so when prism runner is stuck to bring up, etc there is always a vague DEADLINE_EXCEEDED error there

[aIbrahiim]

I don't see it's being setup for cloud etc

beam/.github/workflows/beam_PreCommit_Python_Examples.yml

Line 97 in 4e24a9c

gradle-command: :sdks:python:test-suites:tox:py${{steps.set_py_ver_clean.outputs.py_ver_clean}}:preCommitPy${{steps.set_py_ver_clean.outputs.py_ver_clean}}

Do you mean TestPipelines.pytest_test_pipeline_options could get mutated by one test and affecting other tests? Nor this is immutable as it's a str

yeah that YAML line doesnt pass cloud runner flags and the “cloud” bit is indirect as preCommitPy… runs both the normal tox env and the py*-cloud one (wired up in the tox Gradle common.gradle), so its not obvious from the workflow file alone and yeah, not saying the string gets mutated between tests. pytest_test_pipeline_options is basically “whatever pytest got for --test-pipeline-options at session start.” so my change is just about not using that global fallback for regular unit-style TestPipeline unless it’s actually an integration test (is_integration_test)

[github-actions]

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

[Abacn]

After checking logs / code I don't think this is relevant

The PreCommit runs four independent pytest command, initiated from run_pytest.sh

Running parallel tests with: pytest -m "not no_xdist"  --pyargs  apache_beam/examples/
Running sequential tests with: pytest -m "no_xdist"  --pyargs  apache_beam/examples/
Running parallel tests with: pytest -m "not no_xdist"  --pyargs  apache_beam/examples/
Running sequential tests with: pytest -m "no_xdist"  --pyargs  apache_beam/examples/

There are duplicate commands because one without extensions (from normal tox target), one with cloud dependency extensions (from *-cloud tox target). TestPipelines.pytest_test_pipeline_options stays empty all time.

I don't think we want to change the behavior of flag here. It may break internal / downstream tests

[aIbrahiim]

After checking logs / code I don't think this is relevant

The PreCommit runs four independent pytest command, initiated from run_pytest.sh

Running parallel tests with: pytest -m "not no_xdist"  --pyargs  apache_beam/examples/
Running sequential tests with: pytest -m "no_xdist"  --pyargs  apache_beam/examples/
Running parallel tests with: pytest -m "not no_xdist"  --pyargs  apache_beam/examples/
Running sequential tests with: pytest -m "no_xdist"  --pyargs  apache_beam/examples/

There are duplicate commands because one without extensions (from normal tox target), one with cloud dependency extensions (from *-cloud tox target). TestPipelines.pytest_test_pipeline_options stays empty all time.

I don't think we want to change the behavior of flag here. It may break internal / downstream tests

Yeah thanks and makes sense as my as I did investigation in assuming global options were leaking and logs you showed say they’re not in this precommit path, so I agree the TestPipeline change isnt the right fix here