feat(bundle-analysis): JSONL-only gap analysis with full six-signal attribution#23
Open
colombod wants to merge 4 commits into
Open
feat(bundle-analysis): JSONL-only gap analysis with full six-signal attribution#23colombod wants to merge 4 commits into
colombod wants to merge 4 commits into
Conversation
…ttribution Implements a complete bundle usage gap analysis system for the context-intelligence bundle. Replaces the CI-graph-first approach with a JSONL-only, inventory-first pipeline that works offline. Key changes: - Rewrite inventory scanner — three-tier schema (always_active / agent_level / mode_gated), reads behaviors/*.yaml, mode.contributes blocks, and agent-level frontmatter. Disk cache keyed on bundle slug+hash for near-instant re-runs. - Rewrite processor.py — named sets throughout (not counts), inventory reverse lookup enables attribution of all six signal types: agents, skills, recipes, context, tools, and modes. - Add tool_call event kind — parses tool:pre events for non-recipe tools, attributed via agent_level tool declarations in inventory. - Add mode_activated event kind — parses mode:activated and mode:changed events, attributed via inventory modes map. - Rewrite gap.py — four gap types with specific component names: tree-shake, mode-refactor, config-gap, mode-never-activated. Two-tier evaluation: always_active and agent_level get tree-shake/mode-refactor; mode_gated gets mode-never-activated. - Update orchestration order — inventory built first, signals attributed using inventory as reverse lookup, then gap computed. - Remove CI client dependency — run_bundle_analysis no longer requires AsyncCIClient. tool-bundle-usage works JSONL-only. - Delete GraphFetcher and 18 Cypher query files — no CI graph path. - Add bundle-usage mode (advertised: false) — contributes bundle-usage-analyst agent and tool-bundle-usage tool. - Add bundle-usage-analyst agent — calls bundle_usage tool once, interprets gap findings, writes structured markdown report to disk. - Support bundle.md and bundle.yaml — fixes silent drop of python-dev and lsp bundles which use bundle.yaml format. - 134 tests pass, ruff clean, pyright 0 errors. Co-authored-by: Amplifier <amplifier@microsoft.com>
…h references, fix processor docstring and __all__
Collaborator
|
Reviewed for parent_id and Session B merge interaction: zero impact. This PR reads One mechanical note: this PR and #19 both touch Approved. The deletion of GraphFetcher + 18 Cypher query files is a one-way door — fine and consistent with the stated JSONL-first direction, just flagging. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements a complete bundle usage gap analysis system. Answers "which named components of which bundles did I actually use versus what was declared?" across sessions and workspaces — using local JSONL only, no CI graph required.
What changed
Core library (
context_intelligence/bundle_analysis/)always_active/agent_level/mode_gated). Readsbehaviors/*.yamlfor always-active agents/context/recipes,modes/*.mdcontributesblocks for mode-gated components,agents/*.mdfrontmatter for agent-level tool declarations. Disk cache keyed on bundle{slug}-{hash}for near-instant re-runs. Supports bothbundle.mdandbundle.yamlformats.tool:preevents) and modes (viamode:activated/mode:changedevents). Previously tools and modes were always-zero.tree-shake,mode-refactor,config-gap,mode-never-activated. Two-tier evaluation:always_activeandagent_levelcomponents get tree-shake/mode-refactor;mode_gatedcomponents get mode-never-activated.AsyncCIClientrequirement entirely.Deleted
GraphFetcherclass and all 18 Cypher query files — no CI graph path remains.Tooling
tool-bundle-usage— removed CI client dependency, works JSONL-only.agents/bundle-usage-analyst.md— new agent contributed bybundle-usagemode. Callsbundle_usageonce, interprets gap findings, writes structured markdown report.modes/bundle-usage.md—advertised: falsemode that contributes the analyst agent and tool.Tests
ruff: all checks passed.pyright: 0 errors, 0 warnings.Live verification
Against the
bundle-use-inspectorsworkspace (real JSONL data):Gap findings include
tree-shakefor 7 unused skills,mode-never-activatedfor thecontext-intelligencemode (never activated in this workspace), and 24 totalmode-never-activatedfindings across all bundles — all based on real event data.