Yosys CSA tree - demo

[Goubermouche]

This PR aims to evaluate QoR of YosysHQ/yosys#5753, which introduces a new CSA pass, potentially improving critical path length. Note that the CSA pass runs before the wrapped operator flow (SWAP_ARITH_OPERATORS). The CSA pass consumes $alu/$macc chains, which may reduce the cells available for the wrapped operator flow, so hopefully this doesn't break things.

cc @maliberty @QuantamHD

[maliberty]

Please summarize the benefit you see

[widlarizer]

CSA_TREE isn't getting run in CI for some reason. Probably the yosys is not the one we set in the submodule, I can't rule it out since it reports Yosys 0.63+post (git sha1 UNKNOWN-dirty. I'm seeing in the logs that a docker image foss-fpga-tools-ext-openroad gets used. Does that mean we no longer can just bump the submodule to whatever experimental yosys version we'd like to try out then?

[maliberty]

Submodules point at a specific commit which I don't see updated here. You just changed repo path and are still getting the same commit

[Goubermouche]

Submodules point at a specific commit which I don't see updated here. You just changed repo path and are still getting the same commit

@maliberty Apologies, but I'm not sure what you mean by that - as far as I can see, the submodule points to the relevant commit (Enable CSA for ORFS eval.).

[maliberty]

@vvbandeira suggestions?

[oharboe]

If you are particularly adventurous, you can try #4094 with upcoming changes to https://github.com/The-OpenROAD-Project/bazel-orfs to run the full ORFS test suite against a patched Yosys.

Here is what that looks like, step by step:

Step 1: Install bazelisk

Bazelisk is a small launcher that auto-downloads the correct version of Bazel for the project. Think of it as a wrapper — you run bazelisk instead of bazel and it handles the rest.

# Download the binary
curl -L -o /usr/local/bin/bazelisk \
  https://github.com/bazelbuild/bazelisk/releases/latest/download/bazelisk-linux-amd64
chmod +x /usr/local/bin/bazelisk

Step 2: Checkout #4094

PR #4094 adds Bazel build definitions for 46 designs across 6 PDK platforms. Without it, the ORFS repo has no Bazel support.

git fetch origin pull/4094/head:bazel-orfs-beta
git checkout bazel-orfs-beta

Step 3: Bump Yosys with patches in MODULE.bazel

MODULE.bazel is a file at the root of the repo that pins the exact version of every external dependency (Yosys, OpenROAD, etc.). Think of it as a lockfile — it guarantees everyone uses the same tool versions. No Docker image can silently override it.

Open MODULE.bazel, find the Yosys entry, and change it to point to your patched Yosys (e.g. the one with the CSA tree pass from YosysHQ/yosys#5753). Save the file.

Step 4: Run the full test suite

bazelisk test ... --keep_going

For those used to Make, here is what each part means:

• bazelisk — launches Bazel (auto-downloading it if needed).
• test — builds all targets and runs their tests (like make test, but it also builds any missing dependencies first).
• ... — Bazel syntax for "everything, recursively." You type the three dots literally. This is equivalent to running make in every subdirectory at once.
• --keep_going — do not stop at the first failure (like make -k). You get the full picture of what passes and what breaks.

Behind the scenes, Bazel will:

1. Download the exact Yosys version you pinned in Step 3.
2. Run synthesis, floorplan, placement, CTS, routing, and final optimization for every design/platform combination.
3. Compare QoR metrics against baselines.
4. Report pass/fail per design.

Bazel parallelizes across all available cores automatically. On a 48-core machine, the full suite takes roughly 3–4 hours.

Step 5: Compare before/after QoR numbers

Each ORFS run produces a metadata.json file (in reports/) containing every metric from the flow — area, timing, wirelength, DRC errors, etc. The keys look like finish__timing__setup__ws, detailedroute__route__wirelength, and so on.

To get a before/after comparison, run the flow twice — once with the original Yosys and once with your patched version — and save the metadata.json from each run. Then use this Python script to plot the differences:

#!/usr/bin/env python3
"""Compare before/after QoR metrics from two ORFS metadata.json files."""

import json
import sys
import matplotlib.pyplot as plt
import numpy as np

def load_metrics(path):
    with open(path) as f:
        data = json.load(f)
    # Keep only numeric metrics (skip strings, lists, etc.)
    return {k: v for k, v in data.items() if isinstance(v, (int, float))}

def compare(before_path, after_path, filter_prefix=None):
    before = load_metrics(before_path)
    after = load_metrics(after_path)

    # Find common keys
    common = sorted(set(before) & set(after))
    if filter_prefix:
        common = [k for k in common if k.startswith(filter_prefix)]

    labels = []
    pct_changes = []

    for key in common:
        b, a = before[key], after[key]
        if b == 0 and a == 0:
            continue
        if b == 0:
            pct = float('inf')
        else:
            pct = ((a - b) / abs(b)) * 100
        labels.append(key)
        pct_changes.append(pct)

    return labels, pct_changes

def plot(labels, pct_changes, title="QoR: Before vs After"):
    # Color: green for improvement-direction, red for regression-direction
    # For timing (ws, tns): higher (less negative) is better → positive % = green
    # For area/wirelength: lower is better → negative % = green
    colors = []
    for label, pct in zip(labels, pct_changes):
        is_lower_better = any(x in label for x in ["area", "wirelength", "drc", "violations", "diodes"])
        if is_lower_better:
            colors.append("green" if pct <= 0 else "red")
        else:
            colors.append("green" if pct >= 0 else "red")

    fig, ax = plt.subplots(figsize=(12, max(4, len(labels) * 0.35)))
    y_pos = np.arange(len(labels))
    ax.barh(y_pos, pct_changes, color=colors, edgecolor="black", linewidth=0.5)
    ax.set_yticks(y_pos)
    ax.set_yticklabels(labels, fontsize=8)
    ax.set_xlabel("% change (after vs before)")
    ax.set_title(title)
    ax.axvline(x=0, color="black", linewidth=0.8)
    plt.tight_layout()
    plt.savefig("qor_comparison.png", dpi=150)
    print("Saved qor_comparison.png")
    plt.show()

if __name__ == "__main__":
    if len(sys.argv) < 3:
        print("Usage: python compare_qor.py <before.json> <after.json> [metric_prefix]")
        print("  metric_prefix: optional filter, e.g. 'finish__' or 'detailedroute__'")
        sys.exit(1)

    prefix = sys.argv[3] if len(sys.argv) > 3 else None
    labels, pct_changes = compare(sys.argv[1], sys.argv[2], prefix)
    plot(labels, pct_changes)

Usage example — compare just the final-stage metrics for gcd on nangate45:

# Run once with original Yosys, save the metadata
cp flow/reports/nangate45/gcd/base/metadata.json before.json

# (bump Yosys in MODULE.bazel, re-run)
cp flow/reports/nangate45/gcd/base/metadata.json after.json

# Plot all metrics
python3 compare_qor.py before.json after.json

# Or filter to just finish-stage metrics
python3 compare_qor.py before.json after.json finish__

This produces a horizontal bar chart where green bars are improvements and red bars are regressions, sized by percentage change.

Why this helps with the problem in this PR:

The confusion here was about which Yosys binary CI was actually running — the submodule commit with the CSA patch, or the pre-built one from the Docker image. With bazel-orfs, the Yosys version is explicitly written in MODULE.bazel. There is no ambiguity, no Docker override, and you can test all 46 designs with a single command.

I'll probably add a bazelisk run //:plot-merge-base or something that does this.

[widlarizer]

Ran ORFS locally, comparing with and without -csa in SYNTH_ARGS. Ibex, jpeg and gcd didn't have csa_tree kick in, so no difference there. csa_tree did optimize something inside ethmac but it was only triggering on a few of the $alu cells that can be found in the design, and the results were overall a wash. Carry save adders are a design-sensitive optimization. Within multipliers we already have wallace trees built by the existing booth command, which is integrated in the wrapped operator flow. We'll probably look for specific designs to show off csa_tree (a FIR filter? systolic array?) rather than just synthesizing ORFS test designs