Skip to content

Add 'Debugging from the UI' section with parallel-worker URLs and known limitations#58

Open
samgutentag wants to merge 1 commit into
mainfrom
sam-gutentag/test-error-debugging
Open

Add 'Debugging from the UI' section with parallel-worker URLs and known limitations#58
samgutentag wants to merge 1 commit into
mainfrom
sam-gutentag/test-error-debugging

Conversation

@samgutentag
Copy link
Copy Markdown
Member

Summary

  • Add a "Debugging a flaky test from the UI" section to flaky-tests/dashboard.mdx covering the parallel-worker URL gap (with per-CI env-var workaround), Bundle Upload ID lookups (support-only today), and CI artifact retention vs ticket resolution lag.
  • Add a "Known limitations" subsection inside that block with <Info> callouts for RSpec::Core::MultipleExceptionError (single exception surfaced today, gem extension pending) and Go subtests reporting 0ms duration (AI analysis caveat + post-process workaround).
  • Cross-link from flaky-tests/get-started/frameworks/rspec/index.mdx to the MultipleExceptionError note and from flaky-tests/get-started/frameworks/gotestsum.mdx to the Go-subtest note.

Why

Sourced from customer feedback mining (cluster test-error-log-detail-debugging-info, verdict partial, 9 pairs across 6 customers). The dashboard page documents the failure-detail UI but doesn't address the recurring gaps: parallel-worker drill-down, Bundle Upload ID search, RSpec MultipleExceptionError visibility, Go-subtest 0ms AI caveat, artifact retention lag.

Items flagged for review

  • Parallel-worker env var workaround — confirm CIRCLE_BUILD_URL / BUILDKITE_BUILD_URL / CI_JOB_URL / the constructed GitHub Actions URL is the right surface to recommend, and that appending it to a JUnit property/system-out block is the recommended approach (vs. a CLI flag we haven't documented).
  • RSpec MultipleExceptionError fix status — copy says "is being extended". Confirm whether the gem extension has shipped; if so, update or remove the limitation.
  • Bundle Upload ID — copy says "contact support" is the only option today. Confirm there's no internal lookup URL pattern we'd rather direct users to in the interim.
  • CI artifact retention guidance — recommendation to push artifacts to S3 is generic; if the team has a preferred pattern (e.g., a Trunk-side artifact storage option on the roadmap), point me at it.
  • Page placementdashboard.mdx is already a split candidate (3 clusters target it). If you'd rather this become a standalone flaky-tests/debugging-from-the-ui.mdx page, the section is self-contained and can be lifted out without restructuring the rest of the dashboard doc.
  • Go cross-link target — pointed gotestsum.mdx (the only Go framework page) at the dashboard limitations section. If there's a more specific Go troubleshooting page planned, the link should move there.

Customer signal

@samgutentag samgutentag added the needs review PR sourced from customer-feedback-mining; needs human scrutiny for accuracy before merge label May 20, 2026
@mintlify
Copy link
Copy Markdown
Contributor

mintlify Bot commented May 20, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
trunk 🟢 Ready View Preview May 20, 2026, 11:53 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs review PR sourced from customer-feedback-mining; needs human scrutiny for accuracy before merge

Development

Successfully merging this pull request may close these issues.

1 participant