From 55d5fb1a763d3c475e302555104f8c18b5c925c0 Mon Sep 17 00:00:00 2001 From: Lalatendu Mohanty Date: Mon, 1 Jun 2026 10:59:40 -0400 Subject: [PATCH] Adding proposal for test-mode retrospectively As the proposal docs are are useful to understand the architecture behind the big features we are doing. So having a doc which explain the motivation and architecture behind the feature would be useful for future contributors. Co-Authored-By: Claude Signed-off-by: Lalatendu Mohanty --- docs/proposals/test-mode.md | 123 ++++++++++++++++++++++++++++++++++++ 1 file changed, 123 insertions(+) create mode 100644 docs/proposals/test-mode.md diff --git a/docs/proposals/test-mode.md b/docs/proposals/test-mode.md new file mode 100644 index 00000000..a1dca845 --- /dev/null +++ b/docs/proposals/test-mode.md @@ -0,0 +1,123 @@ +# Bootstrap `--test-mode`: resilient source builds with pre-built fallback + +- Author: Lalatendu Mohanty, Doug Hellmann +- Created: 2026-06-01 (retrospective; feature merged 2025-12-19) +- Status: Implemented +- GitHub issue: [#713](https://github.com/python-wheel-build/fromager/issues/713) +- Pull request: [#865](https://github.com/python-wheel-build/fromager/pull/865) + +## What + +Serial `fromager bootstrap --test-mode` continues after failures instead of +aborting. On source-build failure, fromager downloads a pre-built wheel for +the same package so traversal can proceed; dependents are still discovered and +built. The run ends with `test-mode-failures-.json` and exit code 1 +if any failure was recorded. + +Primary outputs of a degraded run are `graph.json`, `build-order.json`, and the +failure report—not a full set of source-built wheels. + +## Why + +Large source bootstraps often fail one package at a time. Without fallback, +dependents fail too, hiding which packages have real build problems. Operators +work around this by manually marking failures as `pre_built` and re-running +bootstrap repeatedly ([#713](https://github.com/python-wheel-build/fromager/issues/713)). + +Test mode automates that loop: continue traversal, substitute pre-built wheels +on source failure, surface many build gaps in one run, and emit a structured +failure list for automation. + +## Goals + +- Continue serial `bootstrap` after failures; fallback via + `BootstrapRequirementResolver.resolve(..., pre_built=True)`. +- Classify failures: `resolution`, `bootstrap` (fatal, error log) vs `hook`, + `dependency_extraction` (non-fatal, warning log). All are recorded; any + failure yields exit code 1. +- Preserve graph traversal and record `source_url_type` in `build-order.json` + (`"prebuilt"` marks fallback wheels). + +## Non-goals + +- `bootstrap-parallel` (serial only); incompatible with `--sdist-only`. +- Replacing fail-fast default bootstrap or treating fallback wheels as the + shipped product. +- Recording source-build failures when fallback succeeds ([#1166](#limitations)). + +## How + +`commands/bootstrap.py` enables the flag; logic lives in `Bootstrapper` +([iterative-bootstrap.rst](iterative-bootstrap.rst)). + +### Architecture + +**Run flow** — serial `bootstrap` with `--test-mode` never aborts the whole +command on a single package failure; it finishes the run and reports at the end. + +```mermaid +flowchart LR + A[Resolve top-level requirements] --> B[Bootstrap dependency tree] + B --> C[finalize] + C --> D[graph.json + build-order.json + failures JSON] +``` + +**On error** — `_handle_phase_error()` in `bootstrapper.py` decides whether to +skip the package, substitute a pre-built wheel, or continue with a warning. + +```mermaid +flowchart TD + E([Phase error]) --> F{Where?} + + F -->|Resolution| G[Record failure, skip package] + + F -->|Source build| H{Pre-built wheel available?} + H -->|Yes| I[Use wheel, continue traversal] + H -->|No| G + + F -->|Hook or dep extraction| J[Record warning, continue] +``` + +**Phase errors** (`_handle_phase_error()`): + +| Context | Behavior | +| --- | --- | +| `RESOLVE` | Record `resolution`; skip | +| `PREPARE_SOURCE` / `PREPARE_BUILD` / `BUILD` | `_handle_test_mode_failure()`: resolve+download pre-built wheel; on success advance to `PROCESS_INSTALL_DEPS` with `SourceType.PREBUILT`; else record `bootstrap` | +| Post-bootstrap hooks | Record `hook` (warning); continue | +| Install dep extraction | Record `dependency_extraction` (warning); empty deps | + +Settings `pre_built` packages skip the fallback path. Failure records include +package, version, exception type/message, and `failure_type`; see +`FailureRecord` in `bootstrapper.py`. + +**Artifacts:** `build-order.json` `source_url_type` reflects runtime fallback; +graph node `pre_built` reflects settings only (not updated after fallback). + +| | `--test-mode` | `--multiple-versions` | +| --- | --- | --- | +| Purpose | Source-build gap analysis | All matching versions | +| On build failure | Pre-built fallback, keep traversing | Remove version from graph | +| Output | `test-mode-failures-*.json` | Logs only | + +## Usage + +```bash +fromager bootstrap --test-mode -r requirements.txt +``` + +Review `test-mode-failures-*.json`, `build-order.json` (`source_url_type: +"prebuilt"`), and `graph.json`. Tests: `tests/test_bootstrap_test_mode.py`, +`e2e/test_mode_*.sh`. + +## Limitations + +1. Source-build failure not recorded when fallback succeeds ([#1166](https://github.com/python-wheel-build/fromager/issues/1166)). +2. Audit fallback via `build-order.json`, not graph `pre_built`. +3. Serial bootstrap only. +4. Pre-built version may differ from requested source version (warning logged). + +## Key source files + +`commands/bootstrap.py`, `bootstrapper.py`, `bootstrap_requirement_resolver.py`, +`dependency_graph.py`