Add --build-constraint(s) flag support to constraints pipeline

[nailo2c]

closes: #54394

Hi folks, I know this is a big PR. I'll try my best to explain it concise and clear.

Summary

Add build constraints support to Airflow's constraints pipeline.

Airflow now generates and publishes build-constraints-{python}.txt next to the existing runtime constraints, and install flows pass them to uv pip install --build-constraints or pip install --build-constraint when available.

How it works

flowchart LR
    A["Existing inputs:<br/>uv.lock + workspace pyproject.toml"]
    B["Existing runtime constraints"]
    C["Existing constraints branch"]
    D["Existing install commands (uv pip install / pip install)"]

    X["NEW build constraints generation"]
    Y["NEW build-constraints-3.x.txt"]
    Z["NEW --build-constraints flags"]

    A --> B --> C --> D
    A --> X --> Y --> C
    C --> Z --> D

    classDef existing fill:#ffffff,stroke:#777777,color:#111111;
    classDef added fill:#e7f5ff,stroke:#1971c2,stroke-width:2px,color:#111111;

    class A,B,C,D existing;
    class X,Y,Z added;

Loading

Key design decisions:

• Build constraints are a single file per Python version, shared across all 3 constraint modes.

• uv sync paths are excluded because uv sync does not support --build-constraints.

• Fallback no-constraints paths never include build constraints, matching runtime constraints behavior.

• Auto-inferred missing build constraints are skipped for backward compatibility with old constraint branches

• Explicit AIRFLOW_BUILD_CONSTRAINTS_LOCATION values fail fast if the file or URL is missing.

• The generator targets build requirements relevant to Airflow's default wheel-preferred install path, rather than forcing source-build coverage for every locked package.

  Source-scan scope validation

  A separate full-scan script scans all ~749 packages with sdist URLs (no wheel filtering, no cache). Running both against the same uv.lock:

  Production (cold):         24.4s  (110 sdists) → 24 unique build deps
  Production (warm cache):    0.4s  (0 downloads)
  Full scan:                 35.9s  (749 sdists) → 61 unique build deps
  
  Only in full scan (production skipped): 37
  poetry, poetry-core, pdm-backend, mypy, jupyterlab, twine, ...
  Only in production (should be 0): 0
  

  All 37 extra deps come from packages with universal wheels. Under the default wheel-preferred install path, those packages are not expected to enter build isolation, so the production scanner covers the build deps relevant to normal Airflow installs while keeping generation fast.

Review guide

Suggested review order follows the data flow:

Area	Main files	What to review
Generator	scripts/in_container/run_generate_constraints.py	Collect workspace build requirements and upstream build requirements from uv.lock, then resolve pinned build dependency versions with uv pip compile.
Install helpers	scripts/docker/common.sh	Resolve/download build constraints and emit the correct uv/pip install flag.
Install consumers	scripts/docker/*.sh scripts/in_container/install_airflow_and_providers.py scripts/in_container/install_development_dependencies.py	Pass build constraints to relevant install commands and omit them from fallback paths.
Breeze plumbing	dev/breeze/src/airflow_breeze/...	Expose --airflow-build-constraints-location and pass it through env vars / Docker build args.
Publishing	.github/workflows/generate-constraints.yml scripts/ci/constraints/*.sh	Upload, diff, and commit build-constraints-*.txt, including first-time untracked files.
Docs	installing-from-pypi.rst MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md 09_release_management_tasks.rst	Document generated files and user-facing uv / pip usage.
Tests	scripts/tests/in_container/test_build_constraints.py	Unit coverage for resolver behavior, helper behavior, command assembly, and fallback invariants.

Generated/synced files:

• Dockerfile, Dockerfile.ci were regenerated from the command prek update-inlined-dockerfile-scripts --all-files.
• Breeze command help docs/images were regenerated after adding the new CLI option.

Verification

• 72 focused unit tests cover generator, resolver, command assembly, fallback behavior, and regression coverage for transient workspace artifacts, nested sdist pyproject.toml files, and uv conflict diagnostic wording.
• Manual Breeze generation produced build-constraints-*.txt for all three constraints modes.

Generation: workspace scan, upstream scan, and constraint resolution

$ uv run --project scripts python -c "
import sys; sys.path.insert(0, 'scripts/in_container')
from run_generate_constraints import _collect_workspace_build_reqs
from pathlib import Path
reqs = _collect_workspace_build_reqs(Path('.'))
print(f'{len(reqs)} deps: {sorted(reqs)}')
"
10 deps: ['flit-core', 'gitdb', 'gitpython', 'hatchling', 'packaging', 'pathspec', 'pluggy', 'smmap', 'tomli', 'trove-classifiers']

$ uv run --project scripts python -c "
import sys; sys.path.insert(0, 'scripts/in_container')
from run_generate_constraints import _parse_uv_lock
from pathlib import Path
pkgs = _parse_uv_lock(Path('uv.lock'))
targets = [p for p in pkgs if not p.has_universal_wheel and p.sdist_url]
print(f'{len(pkgs)} total, {len(targets)} targets')
"
893 total, 110 targets

$ uv run --project scripts python -c "
import sys; sys.path.insert(0, 'scripts/in_container')
from run_generate_constraints import _stream_build_reqs_from_sdist, _parse_uv_lock
from pathlib import Path
pkgs = _parse_uv_lock(Path('uv.lock'))
pkg = next(p for p in pkgs if p.name == 'pydantic-core')
reqs = _stream_build_reqs_from_sdist(pkg.sdist_url)
print(f'pydantic-core: {reqs}')
"
pydantic-core: ['maturin>=1.9.4,<2']

Integration: 3 constraint modes all produce build-constraints-*.txt

$ breeze ci-image build --python 3.12 --upgrade-to-newer-dependencies --answer yes
$ breeze release-management generate-constraints \
    --airflow-constraints-mode constraints --run-in-parallel --answer yes
$ breeze release-management generate-constraints \
    --airflow-constraints-mode constraints-source-providers --run-in-parallel --answer yes
$ breeze release-management generate-constraints \
    --airflow-constraints-mode constraints-no-providers --run-in-parallel --answer yes

$ ls files/constraints-3.12/
build-constraints-3.12.txt              constraints-3.12.txt                    diff-constraints-3.12.md
build-deps-cache.json                   constraints-no-providers-3.12.txt       diff-constraints-no-providers-3.12.md
                                        constraints-source-providers-3.12.txt   diff-constraints-source-providers-3.12.md
                                        original-constraints-3.12.txt           original-constraints-no-providers-3.12.txt
                                                                                original-constraints-source-providers-3.12.txt

$ cat files/constraints-3.12/build-constraints-3.12.txt
# (header omitted)
beniget==0.4.2.post1
flit-core==3.12.0
hatchling==1.29.0
maturin==1.13.1
meson-python==0.18.0
setuptools==82.0.1
wheel==0.46.3
... (31 pinned build deps total)

Negative & positive tests: build constraints actually block/allow installation

# Create a test package requiring setuptools>=70
$ mkdir -p /tmp/test-pkg && cat > /tmp/test-pkg/pyproject.toml << 'EOF'
[build-system]
requires = ["setuptools>=70"]
build-backend = "setuptools.build_meta"
[project]
name = "test-build-constraint"
version = "0.1.0"
EOF
$ echo "from setuptools import setup; setup()" > /tmp/test-pkg/setup.py
$ cd /tmp/test-pkg && uv build --sdist && cd -

# Negative: conflicting build constraints → correctly fails
$ echo "setuptools==60.0.0" > /tmp/bad-build-constraints.txt
$ uv pip install /tmp/test-pkg/dist/test_build_constraint-0.1.0.tar.gz \
    --build-constraints /tmp/bad-build-constraints.txt
  × Failed to build `test-build-constraint`
  ├─▶ Failed to resolve requirements from `build-system.requires`
  ╰─▶ Because you require setuptools>=70 and setuptools==60.0.0,
      we can conclude that your requirements are unsatisfiable.

# Positive: matching build constraints → succeeds
$ echo "setuptools==82.0.1" > /tmp/good-build-constraints.txt
$ uv pip install /tmp/test-pkg/dist/test_build_constraint-0.1.0.tar.gz \
    --build-constraints /tmp/good-build-constraints.txt
Installed 1 package in 59ms
 + test-build-constraint==0.1.0

CI Results

We can find the build constraints related log in this PR's CI run.

1. Generate constraints - Generate constraints for 3.10 on linux/amd64 - Source constraints

2. Generate constraints - Generate constraints for 3.10 on linux/amd64 - Upload constraint artifacts

3. Build PROD images - Build PROD Regular image 3.10 - Build PROD images w/ source providers 3.10

---

Was generative AI tooling used to co-author this PR?

• [v] Yes (please specify the tool below)
  Claude Opus 4.6 + Codex 5.4

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.