tests: speed up timer-based unit tests#1738
Conversation
|
Bobronium
left a comment
There was a problem hiding this comment.
Asked Claude to measure the speed, looks like a real improvement! Props to @rosetta-livekit-bot 👏
| Metric | main (real timers) | PR #1738 (fake timers) | Speedup |
|---|---|---|---|
| Test execution time | 6.03 s | ~43 ms | ~140× |
| Wall-clock total | ~7.72 s | ~1.75 s | ~4.4× |
rest of the tokens
All 9 tests still pass with identical assertions in both versions.
Why
The single test should not abort tool when preamble forwarders are cleaned up was the entire cost. It used real delay() calls that the test had to actually wait out — dominated by a tool with await delay(6000), plus a 100 ms wait and a 5 s cancelAndWait budget. The PR swaps those real waits for Vitest fake timers (vi.useFakeTimers() + vi.advanceTimersByTimeAsync(...)), so the 6 s tool delay resolves instantly in virtual time. Test compute collapses from 6030 ms to ~43 ms.
Caveats
- The wall-clock improvement is "only" ~4.4× because ~1.7 s of fixed overhead remains (transform 0.86 s, import 1.3 s, setup 0.22 s) — Vite/esbuild compilation and module loading that fake timers can't touch. So the headline number depends on whether you measure the
testsline (~140×) or totalDuration(~4.4×). - The other 8 tests were already fast; the win is entirely concentrated in that one timing-heavy test.
- This is a strictly test-only change (
+11/-3, one file) — no production code or behavior affected.
Bottom line: the speedup is real and substantial.
Summary
Ported from livekit/agents#5980's virtual-time approach where applicable. I also checked other real-time waits and left process/socket/real-stream deadlock guards on real timers because fake timers would mask the behavior under test.
Benchmarks
Command:
pnpm exec vitest run agents/src/connection_pool.test.ts agents/src/utils.test.ts agents/src/voice/generation_tools.test.ts --reporter=dot8.36s, test time9.18s2.26s, test time540msgeneration_toolslong tool cleanup path went from ~6sto ~30msin verbose outputTesting
pnpm exec vitest run agents/src/connection_pool.test.ts agents/src/utils.test.ts agents/src/voice/generation_tools.test.ts --reporter=verbosepnpm exec vitest run agents/src/connection_pool.test.ts agents/src/utils.test.ts agents/src/voice/generation_tools.test.ts --reporter=dotpnpm exec vitest run agents/srcpnpm exec prettier --check agents/src/connection_pool.test.ts agents/src/utils.test.ts agents/src/voice/generation_tools.test.ts