feat(toolkit-docs-generator): latest-version coherence filter#907
Draft
feat(toolkit-docs-generator): latest-version coherence filter#907
Conversation
Engine API returns tools at mixed versions for the same toolkit (e.g. Github tools at @3.1.3 alongside stale notification tools at @2.0.1). These stale tools survive through the pipeline because the data source has no version filtering, and --skip-unchanged preserves them indefinitely since the API response stays consistent. Add filterToolsByMajorityVersion() that computes the version shared by the most tools in a toolkit and drops tools at other versions. Applied in both fetchAllToolkitsData() and fetchToolkitData() (when no explicit version is passed), so stale tools are removed before reaching the diff or merger layers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
- Move extractVersion to utils/fp.ts as the single source of truth; data-merger.ts re-exports it for backward compatibility - Replace lexicographic version tie-break with numeric semver comparison so multi-digit components (e.g. 9.0.0 vs 10.0.0) sort correctly - Remove redundant array spread in fetchAllToolkitsData filter - Add test for multi-digit version tie-break edge case Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
|
Should it be "most" or "greatest"? What if we drop 80 bad tools and keep 10 good ones? |
arcade-mcp's normalize_version() allows semver with pre-release tags
(1.2.3-alpha.1) and build metadata (1.2.3+build.456). The previous
compareVersions used .split(".").map(Number) which produced NaN for
these suffixes. Now strips pre-release and build metadata before
parsing numeric MAJOR.MINOR.PATCH for comparison.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
The majority-version approach was wrong: if a new release has fewer tools (e.g. consolidation), the filter would keep the OLD version with more tools and drop the new one. The correct logic is to always keep tools at the highest (newest) version — stale tools are always at older versions. Renamed: getMajorityVersion → getHighestVersion, filterToolsByMajorityVersion → filterToolsByHighestVersion. Added test: "keeps newer version even when it has fewer tools". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 417a51f. Configure here.
Member
|
wouldn't we want to continue documenting the deprecated tools (but with a deprecated banner) since the engine is still hosting them? |
jottakka
force-pushed
the
fix/majority-version-coherence-filter
branch
from
417a51f to
ea5ab6b
Compare
Remove the re-export of extractVersion from data-merger.ts since it's already exported via utils/index.ts. This eliminates the ambiguous binding that TypeScript treats as a package API regression. Update test imports to use the canonical utils/index.ts export. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update toolkit-diff.ts to import extractVersion from the canonical utils/index.ts location instead of data-merger.ts (which no longer re-exports it). This fixes the SyntaxError when running the CLI: 'The requested module data-merger.js does not provide an export named extractVersion' Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jottakka
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
filterToolsByMajorityVersion()utility that computes the version shared by the most tools in a toolkit and drops tools at other versionsfetchAllToolkitsData()andfetchToolkitData()(when no explicit version is passed) inCombinedToolkitDataSource@2.0.1alongside 35+ tools at@3.1.3) surviving through the docs pipeline indefinitelyProblem
The Engine
/v1/tool_metadataendpoint returns tools at mixed versions for the same toolkit. The docs generator trusts this as-is, grouping tools by toolkit ID with no version filtering. The--skip-unchangedflow compounds this — it correctly sees no change run after run (because the API keeps returning the same stale tools), preserving old output indefinitely.Solution
After grouping tools by toolkit ID, compute the majority version (the version shared by the most tools), then keep only tools at that version. This drops stale tools before they reach the diff or merger. Explicit version parameters bypass the filter.
Test plan
getMajorityVersionandfilterToolsByMajorityVersion(11 tests)CombinedToolkitDataSourceversion filtering (3 new tests)🤖 Generated with Claude Code
Note
Medium Risk
Changes which tools are included in generated docs/diffs by automatically dropping older-version tools when Engine returns mixed versions, which can cause large output churn if current data has inconsistencies.
Overview
Adds a highest-version coherence filter so each toolkit keeps only tools at the numerically highest
@versionwhen no explicit version is requested, dropping stale tools from older releases in bothfetchToolkitData()andfetchAllToolkitsData().Introduces
utils/version-coherence.ts(plus exports) and movesextractVersionintoutils/fp.tswhile re-exporting it fromdata-merger.tsfor compatibility. Updates the merger to preserve a previous toolkit summary when the tool signature changes but summary generation is disabled (no LLM), and adds unit/scenario tests covering mixed-version filtering and diff/summary behavior.Reviewed by Cursor Bugbot for commit 417a51f. Bugbot is set up for automated code reviews on this repo. Configure here.