Adding proposal for test-mode retrospectively

docs/proposals/test-mode.md

@@ -0,0 +1,123 @@
+# Bootstrap `--test-mode`: resilient source builds with pre-built fallback
+
+- Author: Lalatendu Mohanty, Doug Hellmann
+- Created: 2026-06-01 (retrospective; feature merged 2025-12-19)
+- Status: Implemented
+- GitHub issue: [#713](https://github.com/python-wheel-build/fromager/issues/713)
+- Pull request: [#865](https://github.com/python-wheel-build/fromager/pull/865)
+
+## What
+
+Serial `fromager bootstrap --test-mode` continues after failures instead of
+aborting. On source-build failure, fromager downloads a pre-built wheel for
+the same package so traversal can proceed; dependents are still discovered and
+built. The run ends with `test-mode-failures-<timestamp>.json` and exit code 1
+if any failure was recorded.
+
+Primary outputs of a degraded run are `graph.json`, `build-order.json`, and the
+failure report—not a full set of source-built wheels.
+
+## Why
+
+Large source bootstraps often fail one package at a time. Without fallback,
+dependents fail too, hiding which packages have real build problems. Operators
+work around this by manually marking failures as `pre_built` and re-running
+bootstrap repeatedly ([#713](https://github.com/python-wheel-build/fromager/issues/713)).
+
+Test mode automates that loop: continue traversal, substitute pre-built wheels
+on source failure, surface many build gaps in one run, and emit a structured
+failure list for automation.
+
+## Goals
+
+- Continue serial `bootstrap` after failures; fallback via
+  `BootstrapRequirementResolver.resolve(..., pre_built=True)`.
+- Classify failures: `resolution`, `bootstrap` (fatal, error log) vs `hook`,
+  `dependency_extraction` (non-fatal, warning log). All are recorded; any
+  failure yields exit code 1.
+- Preserve graph traversal and record `source_url_type` in `build-order.json`
+  (`"prebuilt"` marks fallback wheels).
+
+## Non-goals
+
+- `bootstrap-parallel` (serial only); incompatible with `--sdist-only`.
+- Replacing fail-fast default bootstrap or treating fallback wheels as the
+  shipped product.
+- Recording source-build failures when fallback succeeds ([#1166](#limitations)).
+
+## How
+
+`commands/bootstrap.py` enables the flag; logic lives in `Bootstrapper`
+([iterative-bootstrap.rst](iterative-bootstrap.rst)).
+
+### Architecture
+
+**Run flow** — serial `bootstrap` with `--test-mode` never aborts the whole
+command on a single package failure; it finishes the run and reports at the end.
+
+```mermaid
+flowchart LR
+  A[Resolve top-level requirements] --> B[Bootstrap dependency tree]
+  B --> C[finalize]
+  C --> D[graph.json + build-order.json + failures JSON]
+```
+
+**On error** — `_handle_phase_error()` in `bootstrapper.py` decides whether to
+skip the package, substitute a pre-built wheel, or continue with a warning.
+
+```mermaid
+flowchart TD
+  E([Phase error]) --> F{Where?}
+
+  F -->|Resolution| G[Record failure, skip package]
+
+  F -->|Source build| H{Pre-built wheel available?}
+  H -->|Yes| I[Use wheel, continue traversal]
+  H -->|No| G
+
+  F -->|Hook or dep extraction| J[Record warning, continue]
+```
+
+**Phase errors** (`_handle_phase_error()`):
+
+| Context | Behavior |
+| --- | --- |
+| `RESOLVE` | Record `resolution`; skip |
+| `PREPARE_SOURCE` / `PREPARE_BUILD` / `BUILD` | `_handle_test_mode_failure()`: resolve+download pre-built wheel; on success advance to `PROCESS_INSTALL_DEPS` with `SourceType.PREBUILT`; else record `bootstrap` |
+| Post-bootstrap hooks | Record `hook` (warning); continue |
+| Install dep extraction | Record `dependency_extraction` (warning); empty deps |
+
+Settings `pre_built` packages skip the fallback path. Failure records include
+package, version, exception type/message, and `failure_type`; see
+`FailureRecord` in `bootstrapper.py`.
+
+**Artifacts:** `build-order.json` `source_url_type` reflects runtime fallback;
+graph node `pre_built` reflects settings only (not updated after fallback).
+
+| | `--test-mode` | `--multiple-versions` |
+| --- | --- | --- |
+| Purpose | Source-build gap analysis | All matching versions |
+| On build failure | Pre-built fallback, keep traversing | Remove version from graph |
+| Output | `test-mode-failures-*.json` | Logs only |
+
+## Usage
+
+```bash
+fromager bootstrap --test-mode -r requirements.txt
+```
+
+Review `test-mode-failures-*.json`, `build-order.json` (`source_url_type:
+"prebuilt"`), and `graph.json`. Tests: `tests/test_bootstrap_test_mode.py`,
+`e2e/test_mode_*.sh`.
+
+## Limitations
+
+1. Source-build failure not recorded when fallback succeeds ([#1166](https://github.com/python-wheel-build/fromager/issues/1166)).
+2. Audit fallback via `build-order.json`, not graph `pre_built`.
+3. Serial bootstrap only.
+4. Pre-built version may differ from requested source version (warning logged).
+
+## Key source files
+
+`commands/bootstrap.py`, `bootstrapper.py`, `bootstrap_requirement_resolver.py`,
+`dependency_graph.py`