Skip to content

Commit a45b1b7

Browse files
committed
docs: add commit generation test results tracking to CLAUDE.md
1 parent 21c820c commit a45b1b7

1 file changed

Lines changed: 13 additions & 1 deletion

File tree

CLAUDE.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -297,6 +297,18 @@ Common mistake: calling a new safeguard/check `fix` — if there was no bug, it'
297297
- **No think-then-compress**: Explicit `<thought>` prompting is not used — small models (<10B) exhaust their token budget on analysis instead of JSON output. The pre-computed EVIDENCE/CONSTRAINTS/SYMBOLS sections serve this role. Revisit for 70B+/cloud APIs.
298298
- **Retry**: `validate_and_retry()` runs up to 3 attempts (`MAX_RETRIES: 3`), logging each violation individually before retry. Future: prioritized violation ordering, per-group retry for split commits.
299299

300+
### Commit Generation Test Results
301+
302+
Real-world test results are tracked in auto-memory at `test-results.md`. After every manual test of commit message generation (`commitbee --dry-run`), record:
303+
304+
- The staged changes (files, type of change)
305+
- Expected vs actual commit type
306+
- Subject and body quality assessment
307+
- Prompt observations (signatures, connections, evidence flags)
308+
- Any issues (retry warnings, display bugs, misclassifications)
309+
310+
Compare new tests against previous results to detect regressions or improvements. The goal is generating fantastic commit messages with small local LLMs (qwen3.5:4b).
311+
300312
### Deferred Issues
301313

302314
A tracked list of review findings, design decisions, and improvement ideas that were identified but deferred lives in auto-memory at `deferred-issues.md`. Rules:
@@ -308,4 +320,4 @@ A tracked list of review findings, design decisions, and improvement ideas that
308320

309321
### Documentation Sync
310322

311-
Keep README.md test count in sync (currently 334).
323+
Keep README.md test count in sync (currently 339).

0 commit comments

Comments
 (0)