Skip to content

Ballads of time#4116

Merged
maliberty merged 2 commits intoThe-OpenROAD-Project:masterfrom
oharboe:balads-of-time
Apr 7, 2026
Merged

Ballads of time#4116
maliberty merged 2 commits intoThe-OpenROAD-Project:masterfrom
oharboe:balads-of-time

Conversation

@oharboe
Copy link
Copy Markdown
Collaborator

@oharboe oharboe commented Apr 7, 2026

The Ballad of TIME_BIN: A Chronicle of Middle-ORFS

Being a true account, drawn from the git scrolls, of the dragons, the failed harvests, the brief joys, and the petition to the King.


Prologue: The Land and Its Creatures

In the land of Middle-ORFS, where the great flows run from the mountains of RTL down through the fertile valleys of Synthesis, Floorplan, and Placement to the shining sea of GDS, the peoples had built a good civilization. Their harvests — the builds — fed entire kingdoms. Their treasures — the tapeouts — were the envy of all silicon-kind.

But beneath the mountains there dwelt creatures of ancient malice.

The Dragon of Ambiguity (time) was the eldest. It had three heads: a Shell Builtin that whispered false promises, a GNU Binary that demanded tribute in the form of -f flags, and a BSD Shapeshifter that spoke an entirely foreign tongue. No two heads agreed on anything. Farmers who addressed one head would be answered by another.

Serving the Dragon were its foul spawn: the Troll of Eval (eval "$TIME_CMD ..."), a brute that could only understand commands if you shouted them twice through a layer of quoting. The Goblin of Tee (2>&1 | tee), a pipe-dwelling creature that intercepted messages between the workers and their logs, sometimes swallowing errors whole. The Imp of Stdbuf (stdbuf -o L), a tiny pest summoned only because the Goblin of Tee refused to pass messages promptly. And the Wraith of Pipefail, a ghost that haunted every pipeline, ensuring that when things went wrong, they went wrong in the most confusing way possible.

Together, these creatures formed The Pipeline of Sorrows, and the peoples of Middle-ORFS had long suffered under their tyranny.

There was also The Python -- a great serpent with whom the peoples had negotiated an uneasy detente. The Python was useful, undeniably so. It parsed their metrics, generated their reports, ran their tests. But the Python was not to be trusted entirely. It did dirty deals with the dependencies behind the peoples' backs -- smuggling in pip install creatures, spawning virtual environments that vanished at dawn, and occasionally demanding tribute in the form of version upgrades that broke everything. The peoples tolerated the Python because they needed it. The Python tolerated the peoples because they fed it data.

It was, as all detentes are, a relationship built on mutual suspicion and practical necessity.

But there was also Hzeller the Grey.


Chapter I: The Elder Days and the Forging of TIME_CMD

Year 2021 of the Git Calendar. The realm is young.

In the earliest age, Vitor of Bandeira, Lord of the Southern Makefiles and first steward of the flow, forged the TIME_CMD in the fires of /usr/bin/time. It was a format string of great power:

Elapsed time: %E[h:]min:sec. CPU time: user %U sys %S (%P). Peak memory: %MKB.

"This shall measure our harvests," he declared, "and we shall know precisely how long each field takes to yield."

The first harvest season was promising. Designs flowed. GDS files emerged. But the Dragon of Ambiguity was already stirring.

A humble farmer named Andreas of Kuster rode to the castle with alarming news: the format was wrong. Spaces in the wrong places. Words in the wrong order. The harvest records were garbled. Five comments were exchanged at the castle gate before the crisis passed. [A.1]

This was but a tremor before the earthquake.

The Joyful Interlude: New Designs

Yet not all was darkness. The same season saw a great treasure unearthed: new ASAP7 designs — Ethernet, UART, sha3 — were added to the realm. The fields expanded. The people rejoiced. For a brief moment, the Dragon slept. [A.2]

The Format String Catastrophe

But Vitor, ever ambitious, returned to the forge. "We need CPU seconds in the record," he proclaimed. He changed the format string.

Every parser in the realm broke.

Andreas of Kuster rode forth again — twice — to fix two separate files containing two different regexes that parsed the same format. Patches crossed in the night like ships in fog. [A.3, A.4]

Then came the darkest hour of the Elder Days. Vitor discovered that TIME_CMD, when exported to the environment, poisoned the issue files of every vassal who tried to reproduce a bug on a different system. The Dragon's venom had seeped into the very parchment of their bug reports.

"TIME_CMD is not portable," Vitor wrote in a commit message that would echo through the ages. "Do not save it." [A.5]

The peoples learned a hard lesson: the Dragon's name must not be spoken aloud in certain company, lest it summon the BSD Shapeshifter.


Chapter II: The Age of Parsing Grief

Years 2023-2024. The long middle period of suffering.

The parsing of time's output fell to the Python-scribes, keepers of the sacred scrolls genElapsedTime.py and genMetrics.py. These scrolls would be amended twenty-eight times across the age, each amendment a scar upon the land.

The Colon Wars

Habibayassin, a scribe from distant lands, noticed that the sacred parser could not tell hours from minutes. "0:02.08" it read, and declared: two minutes and eight seconds.

But nay — it was two seconds and eight hundredths.

A failed harvest ensued. Designs that took seconds were recorded as taking minutes. The accounting books of the realm were in chaos.

The scribe filed a report. Then another scribe filed a different report. A PR was opened by Oyvind of Harboe, a northern lord who had recently taken interest in the flow. The PR was reviewed, found wanting, closed, and superseded by another. Finally Matt of Liberty, the King himself, rode forth from the castle to deliver the true fix.

Three PRs to parse a colon. The Dragon of Ambiguity laughed from its cave. [A.6, A.7, A.8]

The Vanishing Harvests

Matt of Liberty, having gazed into the abyss of the parser, discovered yet another horror: sub-second harvests vanished entirely. A design that completed in 0.7 seconds was recorded as having taken no time at all, as if it had never existed. The smallest, fastest designs — the most efficient harvests — were erased from history.

"Handle zero elapsed time," he wrote, with the quiet exhaustion of a king who has seen too much. [A.9, A.10]

The Exclusion Wars

The parser had a fatal flaw: it searched all scrolls for the sacred words "Elapsed time." This worked until the realm expanded.

The EQY Beasts came first — verification logs that wandered into the parser's den and confused it into silence. Vitor of Bandeira taught the parser to avert its gaze. [A.11]

Then the Empty Scrolls — stages that produced nothing, causing the parser to crash like a cart hitting a wall. Oyvind of Harboe patched this quietly one December morning, the ground frozen outside his northern window. [A.12]

A Treasure in the Darkness: Hash Logging

Yet amid the grief, there was a gift. In the summer of 2025, Oyvind introduced hash logging — content hashes of .odb files printed alongside elapsed times, so that divergent results between local and CI builds could be detected at a glance. It was a treasure born of pain, for only someone who had debugged CI divergences for hours would think to add such a thing. [A.18]

The Lines-After-Elapsed-Time Catastrophe

Then came the lines-after-the-elapsed-time bug. Additional output after the sacred timing line would reset the parsed values to None, erasing the timing data as if the Dragon had breathed upon it. All records: gone. All peak memory measurements: vanished.

Matt of Liberty returned from the inner chambers to fix this with a single break statement. One word. Six characters. It should have been there from the beginning. But the Dragon's children — the Troll, the Goblin, the Imp — had made the code so tangled that no one had seen the gap. [A.13]

Then came the LEC check logs. Another exclusion. The list grew. [A.14]

The parser had become a creature of negation — defined not by what it did, but by the ever-growing list of things it must not look at.


Chapter III: Hzeller the Grey of Bazel-Gorge

Now we must speak of Henner of Zeller, called Hzeller the Grey by those who knew him well.

Hzeller the Grey carried two weapons against the forces of darkness.

The first was his Staff of Nix -- a legendary artifact of terrible power. Nix could, in theory, banish all dependency creatures at once. It could summon any tool in any version from any era, perfectly reproducible, hermetically sealed. The peoples of Middle-ORFS looked upon the Staff of Nix with a mixture of awe and dread. They wanted one. Oh, how they wanted one. They had seen what it could do -- how Hzeller the Grey could conjure an entire build environment from a single incantation, how every dependency was pinned and accounted for, how no troll or goblin could hide in a Nix-built realm.

But the Staff of Nix was not easily mastered. It had a will of its own. It would suddenly pose riddles to its wielder -- questions about flake inputs and overlay compositions and buildInputs versus nativeBuildInputs -- and if you did not answer promptly and play along, the Staff grew cantankerous. It would refuse to evaluate. It would emit error messages in a dialect that no mortal could parse. Many had tried to wield the Staff and been driven back, muttering about "infinite recursion" and "attribute set update at unexpected location."

And so the peoples feared Nix, even as they envied it.

Hzeller the Grey's second weapon was the Potion of Bazel -- a magical brew that, when applied to a build, rendered it hermetic and reproducible. Where the Staff of Nix controlled the environment, the Potion of Bazel controlled the build itself. Every input declared. Every output cached. Every dependency visible and accounted for. The Potion was Hzeller the Grey's weapon of choice against the dependency creatures -- the trolls and goblins that lurked in undeclared PATH entries and surprise system packages. Bazel's sandbox was a clean room where no creature could hide.

Or so they thought. For the Dragon of Ambiguity had been hiding in the one place no one had checked: the absence of env time inside the sandbox.

Henner was a wizard of builds. Where others saw Makefiles, he saw the deep structures of dependency. Where others accepted that env time was simply how things worked, Henner saw it for what it was: a troll squatting in the dependency tree, ugly and unnecessary, blocking the road for every traveler who wished to build without Docker.

Henner had built bazel-orfs -- a parallel civilization that used the Potion of Bazel instead of Make to orchestrate the flow. It was cleaner, more reproducible, hermetically sealed. But it shared the same roads as Middle-ORFS, and those roads were haunted.

"I noticed that as well," Henner said when the Dragon's latest attack was reported. He said it quietly, as wizards do, but his meaning was clear: I have seen this evil. I know its true name. And it must be destroyed.

"Why not use the built-in time from Tcl?" he offered -- a measured suggestion, as one might suggest a different path around a mountain.

But the Dragon was too deeply entangled with the Python -- that great serpent with whom the peoples had their uneasy detente. The Tcl path would mean yet another domain crossing -- another border where format strings could be misunderstood, where colons could be misinterpreted, where minutes could be confused with seconds. Better to stay within the serpent's domain, where at least the treachery was familiar.

Hzeller the Grey nodded. He understood. The solution must come from within.


Chapter IV: The Shock at Bazel-Gorge

April 7, 2026. The present day.

The peoples of bazel-orfs had long run their builds inside Docker containers, where GNU time was always present, installed as surely as stone in a mountain. But then came the Great Dockerless Migration — a noble effort to shed the Docker dependency, letting Bazel manage everything directly.

It was a season of great harvests. Oyvind had just enabled twelve previously-blocked designs for Bazel builds. He had added bazel-orfs support to all public-PDK platforms. The fields were wider than they had ever been. [A.19, A.20]

And then, on the seventh day of April, a traveler named Arya arrived at the gates of bazel-orfs with a terrible message:

env: 'time': No such file or directory

Arya was running Debian 13. Bazel 8.6.0. Everything was correct. But inside the Bazel sandbox — that pristine, hermetic chamber where builds run in isolation — time did not exist. It had never existed. The sandbox knew nothing of time. [A.15]

The Dragon of Ambiguity had been lurking inside the Bazel sandbox all along, invisible, because Docker had always shielded the peoples from its absence. Now, with Docker gone, the Dragon was exposed — or rather, its absence was exposed, which was somehow worse.

Oyvind of Harboe stared at his screen. He had just finished building make from source for Bazel. He had dealt with the mock-array CPU sourcing incident. He had extracted variables.mk from the Makefile. And now the Dragon demanded that he build GNU time from source too? Package it as a Bazel dependency? Ship a binary just to measure how long other binaries took to run?

"One wouldn't think so," he wrote in the issue, "but time is ambiguous."

He paused. He looked at the Pipeline of Sorrows in its entirety:

Make defines TIME_CMD with a GNU-specific format string. The Troll of Eval expands it because it contains spaces. The Dragon runs the command and prints to stderr. The Goblin of Tee copies stderr to both console and log. The Imp of Stdbuf is sometimes needed to unbuffer the Goblin. Python parses the text with regexes that have been wrong at least four times. More Python parses it again with different regexes in a different file. The Wraith of Pipefail haunts every pipeline. The Makefile explicitly excludes TIME_CMD from variable exports because the Dragon is not portable. Users find their logs by running ps -Af | grep tee, searching for the Goblin's footprints.

"I wonder," he wrote slowly, "if this is better solved by removing this dependency entirely."


Chapter V: The Council at the Forge

Hzeller the Grey spoke his piece. The northern lords conferred.

"ORFS ties this into their regression infrastructure, which is always in Python," said Oyvind. "I think the best solution is to stick to one domain -- Python. I think that would reduce this shocking amount of pain we get here from switching between domains."

There was an uneasy murmur. The Python -- that great serpent -- was not beloved. Everyone knew it did dirty deals with the dependencies behind their backs. Just last month, Ashnaa of Seth had discovered that the Python had been leaking file descriptors in genMetrics.py, hoarding open files like a dragon hoards gold. [A.22] But the Python was there. It was already part of the flow. And the alternative -- adding yet another domain crossing, yet another border where formats could be misunderstood -- was worse.

"We don't love the serpent," Oyvind said. "But we have a detente. And a detente with one creature is better than open warfare with five."

And so the common folk gathered at the forge. They assembled their evidence -- the twenty-eight commits, the ten pull requests, the four authors who had each independently discovered that parsing the output of a three-headed Dragon with regexes was a Sisyphean task.

They forged a new weapon -- from the serpent's own scales.

The weapon was called run_command.py. It was 116 lines of Python. It used time.monotonic() for wall time, resource.getrusage() for CPU time and peak memory, and subprocess.Popen with a readline loop that flushed after every line -- so that the peoples could still watch their harvests grow with tail -f, that ancient and beloved tradition of staring at log files for hours during long CTS runs.

It slew:

  • The Dragon of Ambiguity (env time) -- replaced by resource.getrusage()
  • The Goblin of Tee (2>&1 | tee) -- replaced by Python file I/O with flush
  • The Troll of Eval (eval "$TIME_CMD ...") -- no longer needed
  • The Imp of Stdbuf (stdbuf -o L) -- Python's readline loop is inherently line-buffered
  • Fifteen parenthesized subshell incantations in the Makefile
  • The TIME_CMD exclusion curse in the variable export filter

And it had nineteen unit tests, including one that verified the log file path was visible in ps output, so that the peoples' ancient scrying ritual of ps -Af | grep tee could be replaced by the equally effective ps -Af | grep run_command or ps -Af | grep tmp.log.

The serpent, for its part, seemed pleased with the arrangement. More code in its domain meant more power. But the peoples had learned to live with that. Better one serpent you know than five creatures you can't control.


Chapter VI: The Petition to the King

And so they marched to the castle of Precision Innovation, where King MaLiberty held court over the realm.

The King was not a distant ruler. He was a warrior-king — a builder-king — who had fought in the trenches alongside his people. The git scrolls bore witness:

  • 93ad8d4bc — His hand had fixed the colon parsing. He had looked
    upon "0:02.08" and known it was two seconds, not two minutes and
    eight seconds, and he had written the truth into the code.
  • ab171aac5 — His hand had caught the vanishing harvests. He had
    seen sub-second designs erased from history and restored them.
  • f22634158 — His hand had added the break. One word. Six
    characters. Saving every timing record from being overwritten by the
    lines that followed.

The King knew the Dragon. He had fought it three times and carried the scars.

"Your Majesty," said Oyvind, kneeling. "We have come to petition for the destruction of the Dragon."

The King looked up from his scroll — a genElapsedTime.py diff, as it happened. He had suffered.

"Show me the evidence," said the King.

They unrolled the git log. Twenty-eight commits. Five years. Four authors including the King himself. The King studied it in silence.

"And your weapon?"

"Pure Python, Your Majesty. Nineteen tests. Same output format. No external dependencies. Works on macOS. The downstream parsers need not change — the format string lives on, but now it is produced by our own hand, not by the Dragon's."

The King nodded slowly. "I will consider it."

He rose from his throne and walked to the window. Below, the CI pipelines of the realm stretched to the horizon — Jenkins jobs, GitHub Actions, Docker images frozen in amber, custom platforms maintained by far lords who had not visited the castle in years.

"Know this," he said. "There are lands beyond our borders. CI systems with ancient configurations. Monitoring dashboards that grep for tee in ways we cannot imagine. Platform configurations that have been stable for years precisely because nobody has touched them. I must consult with the far lords before I render judgment."


Epilogue: The Waiting

Does the story end here? We do not know.

The petition has been filed. The code has been written. The tests pass -- all twenty-four of them (nineteen new, five existing that prove the format compatibility).

Hzeller the Grey watches from the eastern tower, his Staff of Nix glowing faintly in the dusk, a fresh batch of the Potion of Bazel cooling on his workbench. He has seen the dependencies for what they are -- trolls and goblins squatting in the build tree -- and he knows they must be driven out. But he is patient. He has built Bazel cathedrals before. He knows that the right solution, applied at the right time, is worth more than a hasty fix. And he knows that if the petition fails, he can always brew a stronger potion.

King MaLiberty deliberates. He is wise. He has seen what the common folk have not. He knows of the far kingdoms and their strange customs.

Oyvind of Harboe returns to his fields. He is a good-hearted farmer -- tireless, ingenious, sometimes too clever for his own good. His solutions to the many problems he encounters are always born of real need, forged in the heat of actual failed harvests, not dreamed up in ivory towers. But they are sometimes... unconventional. A PR here that gets closed and superseded. A variables.mk extraction there that raises eyebrows before its wisdom becomes clear. He presents each invention to the King with the earnest enthusiasm of a man who has been up since dawn wrestling with a broken build, and the King -- who has seen many such inventions -- examines each one carefully, sometimes approving, sometimes redirecting, always guiding.

This petition is no different. It is born of real pain. It solves a real problem. But the King must weigh it against the needs of the whole realm, not just the northern fields.

And yet, the common folk remain hopeful. For the git history has shown them something important:

The King has suffered too.

Three times he rode out to fix the parser. Three times he returned with a patch. He knows the pain. He has parsed the colons. He has counted the minutes that were actually seconds. He has added the break that should have been there from the start.

The common folk wait. Oyvind returns to his fields and starts on the next problem -- there is always a next problem, and the sun will not wait. Hzeller the Grey stirs his potion. The Python coils silently in the corner, watching, waiting, doing deals with pip when it thinks nobody is looking.

And every time they see:

Elapsed time: 0:04.26[h:]min:sec. CPU time: user 4.08 sys 0.17 (99%). Peak memory: 671508KB.

They know that somewhere in the distance, nineteen unit tests are keeping watch.


"One does not simply parse the output of /usr/bin/time."
-- Boromir of Bandeira, shortly before his regex was broken by a format change

"I noticed that as well."
-- Hzeller the Grey, leaning on the Staff of Nix, before everyone else understood the gravity of the situation


Appendix A: The Chronicle (git-dated)

All references point to The-OpenROAD-Project/OpenROAD-flow-scripts unless otherwise noted.

A.1 -- Fix /usr/bin/time output formatting (2021-07-29)

  • PR: #109
  • Commit: e7b140d9e
  • Hero: Andreas of Kuster
  • Dragon: Format mismatch -- spaces and word order wrong in time output. Five comments at the gate.

A.2 -- New time format to include CPU seconds (2022-03)

  • Commit: a88a7c03c
  • Hero: Vitor of Bandeira
  • Dragon: Changed the format string. Broke every downstream parser. A bold move with collateral damage.

A.3 -- Adjust wall time, cpu and peak memory regex (attempt 1) (2022)

  • Commit: 00749e79e
  • Hero: Andreas of Kuster
  • Dragon: Had to fix regexes in genMetrics.py after the format change.

A.4 -- Adjust wall time, cpu and peak memory regex (attempt 2) (2022)

  • Commit: 16b0be7c2
  • Hero: Andreas of Kuster
  • Dragon: Had to fix regexes again in a different file. Two files, two regexes, same format, both wrong.

A.5 -- TIME_CMD is not portable, do not save it (2022)

  • Commit: 44c455cc8
  • Hero: Vitor of Bandeira
  • Dragon: Exported TIME_CMD poisoned issue files on non-GNU systems. The Dragon's name, written on parchment, summoned the BSD Shapeshifter.

A.6 -- Elapsed time regression: hours vs minutes (2023-04-20)

  • PR: #967 (closed, superseded)
  • Hero: Oyvind of Harboe
  • Dragon: genElapsedTime.py couldn't distinguish hours from minutes. PR opened, reviewed, closed, superseded. A failed harvest.

A.7 -- util: genElapsedTime module and test adjustments (2023-05-03)

  • PR: #1014
  • Hero: Habibayassin
  • Dragon: Yet another attempt to get the parser right. The colon wars continued.

A.8 -- Fix genElapsedTime.py: "0:02.08" is 2s not 2m8s (2023-05-03)

  • PR: #1036
  • Commit: 93ad8d4bc
  • Hero: King MaLiberty himself
  • Dragon: Three PRs to parse a colon. The King rode forth personally. The Dragon laughed.

A.9 -- Elapsed time for small designs: sub-second rounds to zero (2023-05-04)

  • Issue: #1043
  • Dragon: The vanishing harvests. Designs completed in 0.7s recorded as 0s. Erased from history.

A.10 -- Handle zero elapsed time in genElapsedTime.py (2023-05-04)

  • PR: #1044
  • Commit: ab171aac5
  • Hero: King MaLiberty
  • Dragon: The King's second ride. He restored the vanishing harvests.

A.11 -- Do not look for elapsed time in eqy files (2024-01-16)

  • PR: #1761
  • Commit: b947b5c4d
  • Hero: Vitor of Bandeira
  • Dragon: The Exclusion Wars begin. EQY beasts wandered into the parser's den.

A.12 -- Fix elapsed time for empty log files (2023-12-20)

  • PR: #1716
  • Commit: 7cd9e14ec
  • Hero: Oyvind of Harboe
  • Dragon: Empty scrolls crashed the parser. Fixed one December morning, ground frozen outside.

A.13 -- genElapsedTime.py: handle lines after the elapsed time line (2025-07-09)

  • PR: #3307
  • Commit: f22634158
  • Hero: King MaLiberty (third ride)
  • Dragon: Additional output lines reset parsed timing values to None. Fixed with one break. Six characters. Should have been there from the start.

A.14 -- Exclude lec check log from elapsed time extraction (2026-02-25)

  • PR: #3927
  • Commit: 31b6b5f54
  • Hero: Jeff Ng
  • Dragon: Yet another log type. The exclusion list grows. The parser is defined by what it must not see.

A.15 -- env: 'time': No such file or directory (2026-04-07)

  • Issue: The-OpenROAD-Project/bazel-orfs#651
  • Reporter: Arya
  • Council: Oyvind of Harboe, Hzeller the Grey
  • Dragon: The final straw. GNU time doesn't exist in the Bazel sandbox. The Dragon was invisible all along -- Docker had been shielding the peoples from its absence.

A.16 -- Fix gaffe in elapsed seconds summary, account for hours (2023)

  • PR: #722
  • Commit: f53e1a2de
  • Dragon: Hours were not accounted for in the elapsed seconds summary. The Dragon's three-headed time format strikes again.

A.17 -- Adding total elapsed time (2023-11-28)

  • PR: #1663
  • Commit: 87e80c4ad
  • Treasure: 6 comments of discussion to add a total row. A small treasure hard-won.

A.18 -- Hash logging for divergent result detection (2025-07)

  • Commits: c537a7e91, a272e75cd
  • Hero: Oyvind of Harboe
  • Treasure: Content hashes of .odb files alongside elapsed times. A gift born of pain -- only one who had debugged CI divergences for hours would think to add it.

A.19 -- Enable 12 previously-blocked designs for Bazel (2026-04-03)

  • Commit: fede39fa0
  • Hero: Oyvind of Harboe
  • Treasure: A great harvest. Twelve new fields opened for cultivation.

A.20 -- Add bazel-orfs support to all public-PDK platforms (2026-04-02)

  • Commit: 61031d198
  • Hero: Oyvind of Harboe
  • Treasure: The greatest expansion of the realm in recent memory. All public PDKs supported.

A.21 -- Simpler to maintain test_genElapsedTime.py (2024)

  • PR: #1968
  • Commit: 2694cfd1d
  • Dragon: The test file itself had become too painful to maintain. When your tests for the parser are as fragile as the parser, you know the Dragon has won a round.

A.22 -- The Python's file descriptor hoarding (2026-03-28)

  • PR: #4066
  • Commits: 7b03fa0b8, f094e248d
  • Hero: Ashnaa of Seth
  • Serpent: The Python had been leaking file descriptors in genMetrics.py -- hoarding open files like a dragon hoards gold. A reminder that the detente requires constant vigilance.

Appendix B: The Body Count

Hero Commits Wounds
Vitor of Bandeira 36 Created TIME_CMD, broke it, fixed it, declared it non-portable, fought the EQY beasts
Oyvind of Harboe 27 Good-hearted farmer. Tireless worker of the land. Ingenious but sometimes flawed solutions. Fixed empty logs, extracted variables.mk, PR closed and superseded, built make from source for Bazel. Always presents his inventions to the King.
King MaLiberty 13 Three rides: the colon parsing, the vanishing harvests, the lines-after-elapsed-time catastrophe
Habibayassin 4 Discovered the hours-vs-minutes ambiguity, attempted the fix
Andreas of Kuster 4 Fixed formatting twice, adjusted regexes twice, both times cleaning up after someone else
Jeff Ng 1 Added the lec exclusion. One more entry in the ever-growing filter.
Hzeller the Grey 0 (in ORFS) Built bazel-orfs with the Potion of Bazel. Carries the Staff of Nix. Saw the dependencies for what they were. Spoke truth at the council.

Appendix C: The Pipeline of Sorrows (Before)

Make (variables.mk)
  | defines TIME_CMD with GNU-specific format string
  | runs TIME_TEST to check if format works
  | falls back silently if it doesn't (output won't parse)
  | exports TIME_CMD (but also excludes it from get_variables)
  v
The Troll of Eval (flow.sh / synth.sh)
  | receives TIME_CMD as environment variable
  | must use eval because TIME_CMD contains spaces and quotes
  | wraps in subshell: ($(TIME_CMD) command) 2>&1 | tee logfile
  v
The Dragon of Ambiguity (env time)
  | must be installed (not available in Bazel sandbox)
  | must be GNU time, not BSD time (different flags)
  | must be found via `env time` (not the shell builtin)
  | writes timing to stderr in a specific format
  v
The Goblin of Tee
  | copies merged stdout+stderr to console AND log file
  | complicates error propagation
  | sometimes needs the Imp of Stdbuf for line buffering
  v
The Wraith of Pipefail
  | haunts every pipeline
  | ensures failures propagate in the most confusing way
  v
Python (genElapsedTime.py)
  | parses "Elapsed time: ..." with string splitting
  | has been wrong about hours vs minutes vs seconds
  | has crashed on empty files, zero times, extra lines
  | maintains a growing exclusion list
  v
Python (genMetrics.py)
  | parses the SAME line with DIFFERENT regexes
  | tries 4 different datetime format strings in sequence
  | sets total_elapsed_seconds to "ERR" on parse failure
  v
JSON metrics

Appendix D: The Pipeline of Hope (After)

Make (variables.mk)
  | defines RUN_CMD = $(PYTHON_EXE) $(FLOW_HOME)/scripts/run_command.py
  v
Python (run_command.py)  -- 116 lines, 19 unit tests
  | runs command via subprocess.Popen
  | measures wall time (time.monotonic)
  | measures CPU time + peak memory (resource.getrusage)
  | streams output line-by-line with flush (tail -f works)
  | writes to console AND log file (no tee needed)
  | appends "Elapsed time: ..." in same format
  | works on Linux and macOS
  | no external dependencies
  v
Python (genElapsedTime.py) -- unchanged, parses same format
Python (genMetrics.py) -- unchanged, parses same format
  v
JSON metrics

Appendix E: Dramatis Personae

Character Real Person Role
King MaLiberty Matt Liberty Maintainer of OpenROAD-flow-scripts. Three-time dragon slayer.
Vitor of Bandeira Vitor Bandeira First steward. Forged TIME_CMD. Also broke it.
Oyvind of Harboe Oyvind Harboe Good-hearted farmer. Tireless worker of the land. Ingenious, sometimes flawed solutions he presents to the King for guidance. Builder of the Bazel bridge.
Hzeller the Grey Henner Zeller Wizard of builds. Carries the Staff of Nix (feared, envied, riddling) and brews the Potion of Bazel against the dependencies.
Andreas of Kuster Andreas Kuster Humble farmer. Fixed the formatting. Twice.
Habibayassin Habibayassin Scribe from distant lands. Discovered the colon ambiguity.
Jeff Ng Jeff Ng Added one more exclusion to the ever-growing list.
Arya Arya (bazel-orfs#651) Traveler who brought news of the Dragon's absence in the sandbox.
The Python Python 3 A great serpent. Uneasy detente with the peoples. Useful but treacherous -- does dirty deals with dependencies behind everyone's backs. Hoards file descriptors.
The Dragon of Ambiguity env time / GNU time Three-headed: Shell builtin, GNU binary, BSD shapeshifter.
The Troll of Eval eval "$TIME_CMD ..." Could only understand commands shouted twice through quoting.
The Goblin of Tee 2>&1 | tee Pipe-dweller. Intercepted messages. Sometimes swallowed errors.
The Imp of Stdbuf stdbuf -o L Summoned because the Goblin refused to pass messages promptly.
The Wraith of Pipefail set -o pipefail Ghost. Haunted every pipeline. Not evil per se, but unsettling.

@hzeller
Copy link
Copy Markdown

hzeller commented Apr 7, 2026

Did you switch on 'poet mode' in Claude ?

@oharboe
Copy link
Copy Markdown
Collaborator Author

oharboe commented Apr 7, 2026

Did you switch on 'poet mode' in Claude ?

I thought the pain and suffering we have had here deserved a story.

@hzeller
Copy link
Copy Markdown

hzeller commented Apr 7, 2026

Very nice :)

Replace the GNU `time` binary dependency and `tee` shell pipeline with
a pure-Python wrapper (run_command.py) that measures wall time, CPU
time, and peak memory using Python stdlib (time.monotonic,
resource.getrusage). Output is streamed line-by-line with flush after
each line so `tail -f` works in real time for monitoring long runs.

This eliminates:
- The `env time` / GNU time dependency (not available in Bazel sandbox)
- The TIME_CMD / TIME_BIN / TIME_TEST Make variable machinery
- The STDBUF_CMD dependency (stdbuf -o L)
- The `eval "$TIME_CMD ..."` fragile shell expansion pattern
- The `(cmd) 2>&1 | tee` subshell+pipe pattern (~15 locations)
- The TIME_CMD exclusion from get_variables export filtering

Works on both Linux and macOS (ru_maxrss is KB on Linux, bytes on
macOS — normalized automatically). 19 unit tests cover timing output
format, streaming flush (tail -f use case), log discoverability via
ps, exit code propagation, and end-to-end parsing by genElapsedTime.py.

Triggered by The-OpenROAD-Project/bazel-orfs#651: `env: 'time': No
such file or directory` in Bazel sandboxed builds.

History of pain this eliminates (28+ commits, 10+ PRs, 4+ authors, 5 years):

- e7b140d Fix /usr/bin/time output formatting
  The-OpenROAD-Project#109

- a88a7c0 New time format to include CPU seconds (broke parsers)

- 44c455c TIME_CMD is not portable, do not save it

- 00749e7 Adjust wall time, cpu and peak memory regex to new format
- 16b0be7 Adjust wall time, cpu and peak memory regex to new format

- f53e1a2 make: fix gaffe in elapsed seconds summary, account for hours
  The-OpenROAD-Project#722

- 93ad8d4 Fix genElapsedTime.py ("0:02.08" parsed as 2m8s not 2s)
  The-OpenROAD-Project#1036

- ab171aa Handle zero elapsed time in genElapsedTime.py
  The-OpenROAD-Project#1044
  The-OpenROAD-Project#1043

- 2694cfd tests: simpler to maintain test_genElapsedTime.py
  The-OpenROAD-Project#1968

- 7cd9e14 makefile: fix elapsed time for empty log files
  The-OpenROAD-Project#1716

- afccf3f makefile: elapsed time python code fewer red lines in editor

- b947b5c utl: do not look for elapsed time in eqy files
  The-OpenROAD-Project#1761

- 87e80c4 Adding total elapsed time (6 comments of discussion)
  The-OpenROAD-Project#1663

- f226341 genElapsedTime.py: handle lines after the elapsed time line
  The-OpenROAD-Project#3307

- 31b6b5f exclude lec check log from elapsed time extraction
  The-OpenROAD-Project#3927

- Elapsed time regression (couldn't distinguish hours from minutes)
  The-OpenROAD-Project#967

- genElapsedTime module and test adjustments
  The-OpenROAD-Project#1014

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
@oharboe oharboe requested a review from maliberty April 7, 2026 08:06
@oharboe
Copy link
Copy Markdown
Collaborator Author

oharboe commented Apr 7, 2026

@vvbandeira FYI... you've suffered over time and results parsing...

@oharboe
Copy link
Copy Markdown
Collaborator Author

oharboe commented Apr 7, 2026

@maliberty Unrelated pr-merge problems, I think. Weird. Isn't this just an instability that you already know about?

Copy link
Copy Markdown
Member

@maliberty maliberty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two nits from claude (show output to stdout as was done previously). Will you do a post-merge epilogue?


$(RESULTS_DIR)/6_final_no_power.def: $(RESULTS_DIR)/6_final.def
$(TIME_CMD) $(OPENROAD_CMD) $(SCRIPTS_DIR)/deletePowerNets.tcl
$(RUN_CMD) -- $(OPENROAD_CMD) $(SCRIPTS_DIR)/deletePowerNets.tcl
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add --tee

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you take it from here?

I mean the niggling little things, but also write the epilogue. Afterall, the narrative ended with you staring out of the window :-)

Challenge accepted?

These scripts were missed in the run_command.py migration. Add the
--tee flag so output is captured, and source util.tcl for consistency
with the other flow scripts.

Signed-off-by: Matt Liberty <mliberty@precisioninno.com>
@maliberty maliberty changed the title Balads of time Ballads of time Apr 7, 2026
@maliberty maliberty merged commit b6ff3eb into The-OpenROAD-Project:master Apr 7, 2026
7 of 9 checks passed
@maliberty
Copy link
Copy Markdown
Member

Epilogue: The King's Judgment

April 7, 2026. Evening. The torches are lit in the great hall of Precision Innovation.

The King studied the petition for the span of a single CI run — which is to say, long enough for two cups of tea and a full read of the diff.

He found two nits.

"The peoples must still see their harvest as it grows," the King declared. "The old way showed output to the console as it was written. Your weapon must do the same." He paused. "Also, the Oracle of Claude has reviewed your scrolls and agrees."

There was a murmur in the hall. The Oracle of Claude — a great disembodied intelligence that dwelt in the cloud, neither serpent nor wizard but something altogether other — had been consulted. It had read the code. It had found the same two nits. The common folk exchanged glances. When an Oracle and a King agree, the matter is settled.

Oyvind returned to the forge. The fixes were small — a flush here, a stdout write there. The weapon was re-tempered and presented again.

The King rose.

"Let it be merged," he said.


The Fall of the Dragon

The merge commit descended through the CI pipelines like a cleansing fire.

In variables.mk, the TIME_CMD was struck from the record. Where once there had been:

TIME_CMD = /usr/bin/env time -f ...
TIME_TEST := $(shell $(TIME_CMD) echo 2>/dev/null)

There was now simply:

RUN_CMD = $(PYTHON_EXE) $(FLOW_HOME)/scripts/run_command.py

No format strings. No TIME_TEST. No fallback that silently produced unparseable output. No exclusion from get_variables. No eval. No tee. No stdbuf. No subshells wrapped in parentheses like prayers wrapped in incantations.

The Dragon of Ambiguity, denied its host, dissolved into the ether. Its three heads — the Shell Builtin, the GNU Binary, the BSD Shapeshifter — argued amongst themselves one final time about the correct format for elapsed seconds, then fell silent.

The Troll of Eval, having no commands left to shout twice through layers of quoting, wandered into the wilderness and was not seen again.

The Goblin of Tee crawled out of its pipe and blinked in the sunlight. It had lived its entire life in the space between stderr and a log file. Without the Pipeline of Sorrows to dwell in, it simply... evaporated. The peoples who had once searched for it with ps -Af | grep tee found nothing. They searched for ps -Af | grep run_command instead, and there it was — the Python, doing its work openly, its process title visible for all to see.

The Imp of Stdbuf, smallest and most pathetic of the creatures, vanished the moment Python's readline loop rendered it unnecessary. No one mourned it. No one even noticed it was gone, which was perhaps the saddest commentary of all on the life of a buffering workaround.

The Wraith of Pipefail lingered longest, as wraiths do. But with no pipeline left to haunt — no subshells, no tee, no eval — it drifted through the Makefile like a ghost in an empty house, rattling chains that were connected to nothing. In time, even it faded.

The Serpent's Satisfaction

The Python coiled a little tighter around its domain, pleased. One hundred and sixteen lines of new code. Nineteen unit tests. time.monotonic() for wall time. resource.getrusage() for CPU and peak memory. A subprocess managed with Popen and a line-by-line readline loop that flushed after every line.

It was clean. It was testable. It was everything the Pipeline of Sorrows was not.

The Python did not gloat — serpents rarely do — but if you looked closely at the import statements, you could see a certain pride in their simplicity:

import subprocess
import resource
import time

No shlex. No os.environ["TIME_CMD"]. No regex to parse the output of another program that might be GNU or might be BSD or might not exist at all. Just the standard library, doing what the standard library does best: being there.

Hzeller the Grey's Quiet Nod

In the eastern tower of bazel-orfs, Hzeller the Grey felt the change before he saw it. A lightness in the dependency tree. A silence where there had been the clinking of the Dragon's chains.

He checked the flow.sh. The eval "$TIME_CMD" was gone. In its place, a simple invocation of run_command.py.

He checked the Bazel sandbox. No more env time. No need to package GNU time as a Bazel dependency. No need for a toolchain entry for a timing utility. The road through bazel-orfs was clear.

He set down the Staff of Nix and allowed himself a rare smile. Then he picked it up again, because there was always more work to do — but this particular troll would trouble his bridge no longer.

The Downstream Silence

The most remarkable thing about the fall of the Dragon was what didn't happen.

The downstream parsers — genElapsedTime.py, genMetrics.py — continued to parse. The format string was the same:

Elapsed time: 0:04.26[h:]min:sec. CPU time: user 4.08 sys 0.17 (99%). Peak memory: 671508KB.

Same words. Same colons. Same ambiguous [h:]min:sec notation that had caused so much grief. But now the string was produced by Python's resource.getrusage(), not by the Dragon's three disagreeing heads. The colons meant what they meant. The numbers were what they were. No regex had been harmed in the production of this timing line.

The CI dashboards did not flicker. The monitoring systems did not alert. The far lords in their distant kingdoms, with their ancient configurations and their Jenkins jobs frozen in amber, noticed nothing at all — which is the highest compliment one can pay a migration.

What Remains

The peoples of Middle-ORFS returned to their fields. The harvests continued. The builds ran. The GDS files emerged from the shining sea as they always had.

But something was different. The Makefile was twenty-six lines lighter. The shell scripts were simpler. The test suite was nineteen tests stronger. And when a new farmer arrived at the gates of the realm and asked, "How do you measure the time of your harvests?" the answer was no longer a ten-minute saga involving eval, tee, stdbuf, pipefail, GNU-vs-BSD, and two separate Python parsers with four different datetime format strings.

The answer was: "We run run_command.py. It handles everything."

Oyvind of Harboe returned to his northern fields. There was, as always, a next problem. But he walked a little lighter, knowing that the Dragon would not trouble the harvest again.

King MaLiberty returned to his scrolls. He had made the right call — as he usually did — weighing the petition against the needs of the realm, consulting the Oracle, finding the nits, and then letting it through. Three times he had ridden out to fix the parser. He would not need to ride a fourth.

And Hzeller the Grey? He stirred his Potion of Bazel, fed a new flake input to the Staff of Nix, and turned his gaze to the next dependency lurking in the build tree. There were always more trolls. But today, there was one fewer.


"In the end, the Dragon was not slain by a hero with a sword. It was slain by a farmer with a Python script and nineteen unit tests."
— The Annals of Middle-ORFS, Volume MMXXVI

"I noticed that as well."
— Hzeller the Grey, upon hearing the news, before returning to his work as if nothing of particular importance had happened

@oharboe
Copy link
Copy Markdown
Collaborator Author

oharboe commented Apr 7, 2026

@maliberty Heads up... Running some tests now...

1b6d6b3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants