CONSOLE-5194: Audit and fix Claude Code skills for accuracy#16316
CONSOLE-5194: Audit and fix Claude Code skills for accuracy#16316sg00dwin wants to merge 5 commits intoopenshift:mainfrom
Conversation
Assisted by Claude code
…from Claude Code and replaced by TaskCreate/TaskUpdate
openshift-ci-robot
added
the
jira/valid-reference
|
@sg00dwin: This pull request references CONSOLE-5194 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
📝 WalkthroughWalkthroughThis pull request updates Claude Code skill documentation across the 🚥 Pre-merge checks | ✅ 12✅ Passed checks (12 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In @.claude/skills/skill-audit/SKILL.md:
- Line 81: The fenced code block that opens before the "## Skill Audit Report"
example lacks a language tag (triggering MD040); update the opening fence for
that example to include a language identifier (e.g., change the opening "```" to
"```markdown") so the block is correctly recognized by markdown-lint and docs
pipelines.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: 43404fdc-d47e-4a9b-a7db-218a67b993d3
📒 Files selected for processing (4)
.claude/skills/gen-rtl-test/SKILL.md.claude/skills/pre-push-review/SKILL.md.claude/skills/skill-audit/SKILL.md.claude/skills/update-package/SKILL.md
📜 Review details
🧰 Additional context used
🪛 markdownlint-cli2 (0.22.0)
.claude/skills/skill-audit/SKILL.md
[warning] 81-81: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🔀 Multi-repo context openshift/console-operator
[::openshift/console-operator::] .claude/commands/ci-prep.md: line 96 — contains guidance "Use the TodoWrite tool to track progress..." (TodoWrite referenced).
[::openshift/console-operator::] CLAUDE.md: line 7 — references AGENTS.md.
[::openshift/console-operator::] vendor/github.com/openshift/api/.coderabbit.yaml: lines 27,49-50 — references AGENTS.md and CLAUDE.md.
Notes: this repository has no .claude/skills directory and no matches for @console/test-utils or @console/shared test-utils imports.
🔇 Additional comments (4)
.claude/skills/pre-push-review/SKILL.md (1)
134-134: Good consistency update on context + workflow tooling.Switching project guidance to
AGENTS.mdand standardizing progress tracking onTaskCreate/TaskUpdateis coherent across the file and reduces stale-tool drift.Also applies to: 159-159, 320-323, 373-374, 454-454, 476-476, 492-492
.claude/skills/update-package/SKILL.md (1)
73-73: Nice alignment with current Claude task APIs.Using
TaskCreate/TaskUpdatehere keeps package-update workflow tracking consistent with other skills and avoids deprecated tool names..claude/skills/gen-rtl-test/SKILL.md (2)
512-512: Rule 14 update is a solid improvement.Preferring
userEventwhile still allowingfireEventfor simple synchronous cases is the right balance for realism and maintainability in generated tests.Also applies to: 556-575, 578-587
196-196: No action needed — the import alias is properly configured.The alias
@console/shared/*is explicitly mapped infrontend/tsconfig.json(line 22) to./packages/console-shared/*, which correctly resolves@console/shared/src/test-utils/unit-test-utilsto the existing file atfrontend/packages/console-shared/src/test-utils/unit-test-utils.tsx. Generated tests placed in thefrontend/directory will inherit this configuration and resolve the import without issues.
|
|
||
| Output a summary grouped by skill: | ||
|
|
||
| ``` |
There was a problem hiding this comment.
Add a language tag to the fenced report example.
Line 81 opens a fenced code block without a language specifier, which triggers MD040 and can break markdown-lint gating in strict docs pipelines.
Suggested fix
-```
+```markdown
## Skill Audit Report
...</details>
<details>
<summary>🧰 Tools</summary>
<details>
<summary>🪛 markdownlint-cli2 (0.22.0)</summary>
[warning] 81-81: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
</details>
</details>
<details>
<summary>🤖 Prompt for AI Agents</summary>
Verify each finding against the current code and only fix it if needed.
In @.claude/skills/skill-audit/SKILL.md at line 81, The fenced code block that
opens before the "## Skill Audit Report" example lacks a language tag
(triggering MD040); update the opening fence for that example to include a
language identifier (e.g., change the opening "" to "markdown") so the
block is correctly recognized by markdown-lint and docs pipelines.
</details>
<!-- fingerprinting:phantom:triton:hawk:9f1da3fc-354b-4584-a9cc-ba88b5182442 -->
<!-- This is an auto-generated comment by CodeRabbit -->
There was a problem hiding this comment.
LGTM (but I did not write the skill :)
Assisted by Claude code
sg00dwin
force-pushed
the
CONSOLE-5194-AI-Tooling-Review
branch
from
4f19328 to
58c6de6
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: sg00dwin The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
@sg00dwin: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
| // Simulate clicking | ||
| fireEvent.click(button); | ||
| expect(screen.getByText(/success/i)).toBeVisible(); | ||
| }, 30000); // userEvent.type is slow — extend timeout for tests with significant typing |
There was a problem hiding this comment.
Would it be better to remove , 30000) and show proper async handling instead? Rather than mask timing issues with extended timeouts.
6 participants
Analysis:
AI coding skills in
.claude/skills/accumulate stale references over time — deprecated tool names, incorrect file paths, outdated config file references — with no automated way to detect the drift.Note: Assigning review of skill-specific changes to the associated creator of each skill.
/assign @cajieh to review
.claude/skills/gen-rtl-test/SKILL.md/assign @Leo6Leo to review
.claude/skills/pre-push-review/SKILL.md/assign @logonoff to review
.claude/skills/update-package/SKILL.md/assign @jhadvig to review
claude/skills/skill-audit/SKILL.mdDescription:
gen-rtl-test: Corrected@console/test-utilsimport path to@console/shared/src/test-utils/unit-test-utils; updated Rule 14 to recommenduserEventoverfireEvent(no longer blocked by Jest version)pre-push-review: Replaced 6 references to deprecatedTodoWritewithTaskCreate/TaskUpdate; corrected.claude/CLAUDE.mdreferences toAGENTS.mdupdate-package: ReplacedTodoWritereference withTaskCreate/TaskUpdateskill-auditskill (/skill-audit) that detects:@console/*import paths.claude/skills/directories (bidirectional: config→skills and skills→config)Test cases:
/skill-auditand verify it produces a report with no false positives/skill-audit gen-rtl-testand verify single-skill mode works/gen-rtl-teston a component and verify the corrected import path appears in generated tests/pre-push-reviewand verify it usesTaskCreate/TaskUpdate(notTodoWrite)Summary by CodeRabbit
New Features
Documentation
userEventoverfireEventfor simulating user interactions while notingfireEventremains acceptable for simple synchronous cases.