Add codebase scanner, CLAUDE.md init, and usage-focused workflow

- ctc scan: runs linters/tests/type-checkers to find errors without
  burning AI tokens. Auto-detects Python, Node, Rust, Go projects.
- ctc init: drops a CLAUDE.md into any repo teaching Claude Code to
  work efficiently (chunk work, commit often, read scan results first).
- Updated README with the actual workflow: scan -> paste -> fix.

https://claude.ai/code/session_01Fzv8baXnEVVhnrffAb3Ucc
This commit is contained in:
Claude
2026-03-29 22:48:58 +00:00
parent 2d4ead1e3f
commit e211583214
4 changed files with 529 additions and 67 deletions
+103 -63
View File
@@ -4,116 +4,156 @@ Break large coding tasks into self-contained chunks for efficient Claude Code se
## The Problem ## The Problem
You ask Claude Code to make a big change — add auth, refactor a module, migrate a database — and it runs out of usage halfway through. You're left with half-finished work and no easy way to pick up where you left off. You ask Claude Code to make a big change — add auth, refactor a module, fix errors across a 10,000-line codebase — and it runs out of usage halfway through. You're left with half-finished work and no easy way to pick up where you left off.
## The Solution ## The Solution
`ctc` decomposes large tasks into independent, self-contained chunks. Each chunk is a complete prompt you can paste into a fresh Claude Code session. It includes just enough context for Claude to work without needing the prior conversation. Two things working together:
1. **`ctc scan`** — Runs linters, tests, and pattern matching on your codebase *without using any AI tokens*. Finds the errors so Claude doesn't have to read 10,000 lines to find them itself.
2. **`ctc plan`** — Breaks big tasks into self-contained chunks with ready-to-paste prompts for Claude Code.
3. **`ctc init`** — Drops a `CLAUDE.md` into your repo that teaches Claude Code to be usage-efficient automatically.
## Install ## Install
```bash ```bash
# Clone this repo, then:
pip install -e . pip install -e .
``` ```
## Usage ## Quick Start: The Workflow
### 1. Plan — decompose a task ### Step 1: Set up your project (one time)
```bash
cd /path/to/your/big-project
ctc init
```
This creates a `CLAUDE.md` in your project that teaches Claude Code to work efficiently — chunk its own work, commit often, and not waste tokens exploring.
### Step 2: Find errors fast (no AI needed)
```bash
ctc scan
# or with a specific focus:
ctc scan "authentication is broken"
```
This runs your project's linters, tests, and type checkers automatically. It detects what kind of project you have (Python, Node, Rust, Go) and runs the right tools. The output goes to `.claude-chunks/scan_results.md`.
### Step 3: Open Claude Code and paste the results
Open Claude Code connected to your repo as normal. Then:
```
Fix the errors in .claude-chunks/scan_results.md
```
Claude reads the pre-scanned results and goes straight to fixing — no wasted tokens reading 10,000 lines to find the bugs.
### Step 4: For big tasks, plan chunks first
```bash ```bash
ctc plan "Add OAuth2 authentication with login, logout, and RBAC" ctc plan "Add OAuth2 authentication with login, logout, and RBAC"
``` ```
This analyzes your codebase and creates a set of numbered chunks in `.claude-chunks/prompts/`. This creates numbered prompt files in `.claude-chunks/prompts/`. Each one is a self-contained session:
### 2. Execute — one chunk at a time
```bash ```bash
# See what's next ctc next # see the next chunk's prompt
ctc next ctc next -c # copy it to clipboard
# Or copy it straight to clipboard
ctc next -c
# Paste the prompt into Claude Code and let it work
``` ```
### 3. Track — mark chunks as done Open Claude Code, paste the prompt, let it work, then:
```bash ```bash
ctc done 1 ctc done 1 # mark chunk 1 complete
ctc status ctc next # get chunk 2
``` ```
### 4. Repeat until the full task is complete ## Your Actual Workflow
Here's what this looks like day-to-day:
```bash
ctc next # get the next chunk
# paste into Claude Code
ctc done 2 # mark it complete
ctc next # and so on
``` ```
You: cd my-big-project
You: ctc scan # 10 seconds, finds 3 errors
You: [open Claude Code]
You: "Fix the errors in .claude-chunks/scan_results.md"
Claude: [reads the file, fixes the 3 errors, done in one session]
```
For big features:
```
You: ctc plan "migrate database from MySQL to Postgres" -s phase
You: ctc next -c # copies chunk 1 prompt
You: [open Claude Code, paste]
Claude: [does chunk 1: research & plan, commits]
You: ctc done 1
You: ctc next -c # copies chunk 2 prompt
You: [paste into Claude Code — same session or new one]
Claude: [does chunk 2: core implementation, commits]
You: ctc done 2
... and so on
```
## Commands
| Command | What it does |
|---------|-------------|
| `ctc init` | Add CLAUDE.md to your project (teaches Claude to be efficient) |
| `ctc scan` | Find errors with linters/tests (no AI tokens used) |
| `ctc scan "description"` | Same, but adds your description to the output prompt |
| `ctc plan <task>` | Break a big task into chunks |
| `ctc plan <task> -s component` | Chunk by directory/module instead of phase |
| `ctc plan <task> -s manual --chunks-file chunks.txt` | Use your own chunk list |
| `ctc status` | See progress on current plan |
| `ctc next` | Print the next chunk's prompt |
| `ctc next -c` | Copy it to clipboard |
| `ctc done <id>` | Mark a chunk as complete |
| `ctc show <id>` | View a specific chunk's prompt |
## What `ctc scan` Detects
It auto-detects your project type and runs the right tools:
| Project | What it runs |
|---------|-------------|
| **Python** | pytest, ruff, mypy, syntax check |
| **Node/TypeScript** | npm/yarn test, eslint, tsc --noEmit |
| **Rust** | cargo check, cargo test --no-run |
| **Go** | go build, go vet, go test |
| **Any** | grep for TODO/FIXME/HACK/BUG markers |
## Decomposition Strategies ## Decomposition Strategies
### `phase` (default) ### `phase` (default)
Breaks work into standard phases: Research & Plan Core Implementation Tests Integration & Cleanup. Best for most tasks. Breaks work into: Research & Plan -> Core Implementation -> Tests -> Integration & Cleanup.
```bash
ctc plan "Migrate from REST to GraphQL" -s phase
```
### `component` ### `component`
Groups work by directory/module. Best when changes are spread across independent components. One chunk per top-level directory. Good when changes are spread across independent modules.
```bash
ctc plan "Add input validation everywhere" -s component
```
### `manual` ### `manual`
You define the chunks yourself in a text file, one per line. You write a text file with one chunk per line:
```bash ```
# chunks.txt # chunks.txt
Set up database models for users and roles Set up database models for users and roles
Build registration and login API endpoints Build registration and login API endpoints
Add JWT middleware and session management Add JWT middleware and session management
Build role-based access control decorators Build role-based access control decorators
Add frontend login/logout pages
``` ```
```bash ```bash
ctc plan "Add user auth" -s manual --chunks-file chunks.txt ctc plan "Add user auth" -s manual --chunks-file chunks.txt
``` ```
## Commands
| Command | Description |
|---------|-------------|
| `ctc plan <task>` | Decompose a task into chunks |
| `ctc status` | Show progress on current plan |
| `ctc next` | Print the next chunk's prompt |
| `ctc next -c` | Copy next chunk's prompt to clipboard |
| `ctc done <id>` | Mark a chunk as complete |
| `ctc show <id>` | Show a specific chunk's prompt |
## How It Works
1. **Analyzes** your codebase structure (languages, key files, directory layout)
2. **Decomposes** the task using the chosen strategy
3. **Generates** self-contained prompts with codebase context, targeted instructions, file lists, and verification steps
4. **Tracks** progress so you always know what's done and what's next
Each generated prompt tells Claude Code:
- What the codebase looks like (summary, not full contents)
- Exactly what to do in this chunk
- Which files to read and modify
- What depends on what
- How to verify the work is correct
## Tips ## Tips
- **Start with `phase` strategy** — it works well for most tasks - **Always `ctc scan` before asking Claude to fix errors** — saves huge amounts of usage
- **Use `manual` for complex tasks** — you know your codebase best; write chunks that make sense for your architecture - **`ctc init` once per project** — the CLAUDE.md stays in the repo and works every session
- **Keep chunks independent** — the less each chunk depends on others, the cleaner the handoff - **Keep chunks independent** — the less each chunk depends on others, the cleaner the handoff
- **Add `.claude-chunks/` to `.gitignore`** — it's working state, not source code - **Add `.claude-chunks/` to `.gitignore`** — it's working state, not source code
- **Commit CLAUDE.md to your repo** — it benefits every Claude Code session in that project
+134 -3
View File
@@ -5,7 +5,8 @@ import sys
from pathlib import Path from pathlib import Path
from .analyzer import analyze_codebase from .analyzer import analyze_codebase
from .decomposer import TaskChunk, decompose_by_phase, decompose_by_component, STRATEGIES from .decomposer import TaskChunk, decompose_by_phase, decompose_by_component
from .scanner import scan_codebase
from .tracker import save_plan, load_plan, mark_done, get_next_chunk, get_status_summary, TRACKER_DIR from .tracker import save_plan, load_plan, mark_done, get_next_chunk, get_status_summary, TRACKER_DIR
@@ -54,8 +55,8 @@ def cmd_plan(args: argparse.Namespace) -> None:
print(f"\nPlan saved to: {plan_file}") print(f"\nPlan saved to: {plan_file}")
print(f"Prompts saved to: {root / TRACKER_DIR / 'prompts'}/") print(f"Prompts saved to: {root / TRACKER_DIR / 'prompts'}/")
print(f"\nTo start, copy the prompt from chunk_01.md into Claude Code.") print("\nTo start, copy the prompt from chunk_01.md into Claude Code.")
print(f"After each chunk, run: ctc done <chunk_id>") print("After each chunk, run: ctc done <chunk_id>")
def cmd_status(args: argparse.Namespace) -> None: def cmd_status(args: argparse.Namespace) -> None:
@@ -128,6 +129,74 @@ def cmd_show(args: argparse.Namespace) -> None:
sys.exit(1) sys.exit(1)
def cmd_scan(args: argparse.Namespace) -> None:
"""Scan the codebase for errors, then output a ready-to-paste prompt."""
root = Path(args.project).resolve()
if not root.is_dir():
print(f"Error: {root} is not a directory.", file=sys.stderr)
sys.exit(1)
print(f"Scanning {root}...", file=sys.stderr)
result = scan_codebase(root)
task = args.task or ""
prompt = result.to_prompt(task)
output_file = root / ".claude-chunks" / "scan_results.md"
output_file.parent.mkdir(exist_ok=True)
output_file.write_text(prompt)
if args.copy:
try:
import subprocess
process = subprocess.Popen(
_get_clipboard_cmd(),
stdin=subprocess.PIPE,
)
process.communicate(prompt.encode())
print("Scan complete! Prompt copied to clipboard.", file=sys.stderr)
print(f"Also saved to: {output_file}", file=sys.stderr)
except (FileNotFoundError, OSError):
print(f"Scan complete! Saved to: {output_file}", file=sys.stderr)
print(prompt)
else:
print(f"Scan complete! Saved to: {output_file}", file=sys.stderr)
print(file=sys.stderr)
print(prompt)
# Summary stats to stderr
n_err = len(result.errors)
n_warn = len(result.warnings)
n_files = len(set(result.files_with_issues))
print(f"\nFound: {n_err} errors, {n_warn} warnings across {n_files} files", file=sys.stderr)
def cmd_init(args: argparse.Namespace) -> None:
"""Initialize a CLAUDE.md in the target project to teach Claude Code chunking."""
root = Path(args.project).resolve()
claude_md = root / "CLAUDE.md"
if claude_md.exists() and not args.force:
print(f"CLAUDE.md already exists at {claude_md}. Use --force to overwrite.", file=sys.stderr)
sys.exit(1)
template = _get_claude_md_template()
# If CLAUDE.md exists and --force, append rather than overwrite
if claude_md.exists() and args.force:
existing = claude_md.read_text()
if "# Usage Efficiency" not in existing:
claude_md.write_text(existing.rstrip() + "\n\n" + template)
print(f"Appended chunking instructions to {claude_md}")
else:
print("CLAUDE.md already has chunking instructions.")
else:
claude_md.write_text(template)
print(f"Created {claude_md}")
print("Claude Code will now follow usage-efficient chunking practices in this repo.")
def _get_clipboard_cmd() -> list[str]: def _get_clipboard_cmd() -> list[str]:
"""Get the platform-appropriate clipboard command.""" """Get the platform-appropriate clipboard command."""
import platform import platform
@@ -160,6 +229,57 @@ def _parse_manual_chunks(filepath: str, task: str) -> list[dict]:
return chunks return chunks
def _get_claude_md_template() -> str:
return '''# Usage Efficiency
When working on large tasks in this codebase, follow these practices to minimize
token usage and avoid running out of session capacity.
## Before Starting a Large Task
1. Check if a pre-scan exists: look for `.claude-chunks/scan_results.md`
2. If it exists, read it FIRST — it contains linter/test/build errors already identified
3. Use those results to target your work. Do NOT re-read the entire codebase.
## Chunking Strategy
For any task that will touch more than 5 files or require significant analysis:
1. **Plan first, implement second.** Before writing code, list:
- Which files need to change
- What order to change them in
- How to verify each change works
2. **Work in commits.** Make a commit after each logical unit of work. This creates
save points if the session ends unexpectedly.
3. **Stay focused.** Only read files directly relevant to the current task. Don't
explore the codebase "to understand it better" — that burns tokens fast.
4. **Use tools efficiently:**
- Use `grep`/`rg` to find what you need instead of reading whole files
- Read specific line ranges instead of entire files when possible
- Don't re-read files you've already read in this session
## If the Task Is Too Large
If you realize mid-task that you won't finish in this session:
1. Commit everything you've done so far
2. Create a file called `.claude-chunks/handoff.md` with:
- What was completed
- What remains to be done
- Any gotchas or context the next session needs
3. Tell the user to run `ctc status` to see progress
## Error Fixing Workflow
When asked to find and fix errors in the codebase:
1. First check for `.claude-chunks/scan_results.md` — it may already have the errors
2. If not, ask the user to run `ctc scan` first (it's faster than AI-powered search)
3. Focus on the specific files and line numbers from the scan results
4. Don't read unrelated files
'''
def main() -> None: def main() -> None:
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
prog="ctc", prog="ctc",
@@ -184,6 +304,15 @@ def main() -> None:
help="File with one chunk description per line (for manual strategy)", help="File with one chunk description per line (for manual strategy)",
) )
# scan
p_scan = sub.add_parser("scan", help="Scan codebase for errors and generate a prompt")
p_scan.add_argument("task", nargs="?", default="", help="Optional: what you're trying to fix")
p_scan.add_argument("-c", "--copy", action="store_true", help="Copy prompt to clipboard")
# init
p_init = sub.add_parser("init", help="Add CLAUDE.md with chunking instructions to a project")
p_init.add_argument("--force", action="store_true", help="Overwrite/append to existing CLAUDE.md")
# status # status
sub.add_parser("status", help="Show progress on the current plan") sub.add_parser("status", help="Show progress on the current plan")
@@ -203,6 +332,8 @@ def main() -> None:
commands = { commands = {
"plan": cmd_plan, "plan": cmd_plan,
"scan": cmd_scan,
"init": cmd_init,
"status": cmd_status, "status": cmd_status,
"done": cmd_done, "done": cmd_done,
"next": cmd_next, "next": cmd_next,
+1 -1
View File
@@ -42,7 +42,7 @@ class TaskChunk:
if self.depends_on: if self.depends_on:
deps = ", ".join(f"Task {d}" for d in self.depends_on) deps = ", ".join(f"Task {d}" for d in self.depends_on)
lines.append(f"## Prerequisites") lines.append("## Prerequisites")
lines.append(f"This task depends on: {deps}. Those changes should already be committed.") lines.append(f"This task depends on: {deps}. Those changes should already be committed.")
lines.append("") lines.append("")
+291
View File
@@ -0,0 +1,291 @@
"""Fast codebase scanner - finds errors/issues WITHOUT using AI tokens.
Runs linters, tests, and pattern matching to identify problems before
you even open Claude Code. Feed the results to Claude so it doesn't
waste usage reading 10,000 lines to find the bug itself.
"""
import subprocess
import shutil
from dataclasses import dataclass, field
from pathlib import Path
@dataclass
class ScanResult:
"""Results from a codebase scan."""
errors: list[str] = field(default_factory=list) # actual errors found
warnings: list[str] = field(default_factory=list)
files_with_issues: list[str] = field(default_factory=list)
test_output: str = ""
lint_output: str = ""
build_output: str = ""
raw_sections: dict[str, str] = field(default_factory=dict)
def to_prompt(self, task: str = "") -> str:
"""Generate a prompt you can paste into Claude Code."""
lines = []
if task:
lines.append(f"# Task: {task}")
lines.append("")
lines.append("# Pre-scan Results")
lines.append("The following issues were found by automated scanning (linters, tests, grep).")
lines.append("Use these results to focus your work — don't re-read the entire codebase.")
lines.append("")
if self.errors:
lines.append(f"## Errors ({len(self.errors)})")
for e in self.errors:
lines.append(e)
lines.append("")
if self.warnings:
lines.append(f"## Warnings ({len(self.warnings)})")
for w in self.warnings[:20]: # cap at 20 to keep prompt manageable
lines.append(w)
if len(self.warnings) > 20:
lines.append(f"... and {len(self.warnings) - 20} more warnings")
lines.append("")
if self.test_output:
lines.append("## Test Output")
lines.append("```")
# Only include the tail — failures are at the end
test_lines = self.test_output.splitlines()
if len(test_lines) > 60:
lines.append("... (truncated, showing last 60 lines)")
lines.extend(test_lines[-60:])
else:
lines.extend(test_lines)
lines.append("```")
lines.append("")
if self.lint_output:
lines.append("## Lint Output")
lines.append("```")
lint_lines = self.lint_output.splitlines()
if len(lint_lines) > 60:
lines.append("... (truncated, showing last 60 lines)")
lines.extend(lint_lines[-60:])
else:
lines.extend(lint_lines)
lines.append("```")
lines.append("")
if self.build_output:
lines.append("## Build Output")
lines.append("```")
build_lines = self.build_output.splitlines()
if len(build_lines) > 60:
lines.append("... (truncated, showing last 60 lines)")
lines.extend(build_lines[-60:])
else:
lines.extend(build_lines)
lines.append("```")
lines.append("")
if self.files_with_issues:
lines.append("## Files with Issues")
for f in sorted(set(self.files_with_issues)):
lines.append(f"- `{f}`")
lines.append("")
lines.extend([
"## Instructions",
"- Fix the errors above. Start with the files listed in 'Files with Issues'.",
"- Read only the files relevant to the errors — don't explore the whole codebase.",
"- Commit your fixes when done.",
])
return "\n".join(lines)
def _run(cmd: list[str], cwd: Path, timeout: int = 120) -> tuple[int, str]:
"""Run a command and capture output. Returns (exit_code, combined_output)."""
try:
result = subprocess.run(
cmd, cwd=cwd, capture_output=True, text=True, timeout=timeout,
)
output = result.stdout
if result.stderr:
output += "\n" + result.stderr
return result.returncode, output.strip()
except subprocess.TimeoutExpired:
return -1, f"Command timed out after {timeout}s: {' '.join(cmd)}"
except FileNotFoundError:
return -1, ""
def _extract_file_paths(output: str, root: Path) -> list[str]:
"""Extract file paths from linter/compiler output lines like 'path/to/file.py:10: error'."""
files = []
for line in output.splitlines():
if ":" in line:
candidate = line.split(":")[0].strip()
if candidate and (root / candidate).exists():
files.append(candidate)
return files
def scan_codebase(root: Path) -> ScanResult:
"""Run all available scanners against the codebase."""
root = root.resolve()
result = ScanResult()
# Detect project type and run appropriate tools
_scan_python(root, result)
_scan_node(root, result)
_scan_rust(root, result)
_scan_go(root, result)
_scan_generic_patterns(root, result)
return result
def _scan_python(root: Path, result: ScanResult) -> None:
"""Run Python-specific checks."""
has_python = (
(root / "pyproject.toml").exists()
or (root / "setup.py").exists()
or (root / "requirements.txt").exists()
)
if not has_python:
return
# pytest
for cmd in [["python", "-m", "pytest", "--tb=short", "-q"], ["pytest", "--tb=short", "-q"]]:
code, output = _run(cmd, root)
if code == -1 and not output:
continue
if code != 0 and output:
result.test_output = output
result.files_with_issues.extend(_extract_file_paths(output, root))
break
# ruff (fast linter)
if shutil.which("ruff"):
code, output = _run(["ruff", "check", "."], root)
if code != 0 and output:
result.lint_output = output
result.files_with_issues.extend(_extract_file_paths(output, root))
# mypy
if shutil.which("mypy"):
code, output = _run(["mypy", ".", "--no-error-summary"], root, timeout=180)
if code != 0 and output:
for line in output.splitlines():
if ": error:" in line:
result.errors.append(line)
elif ": warning:" in line:
result.warnings.append(line)
result.files_with_issues.extend(_extract_file_paths(output, root))
# Python syntax errors (fast, no deps needed)
code, output = _run(["python", "-m", "py_compile", "--help"], root)
py_files = list(root.rglob("*.py"))
for py_file in py_files[:500]: # cap to avoid huge repos
rel = py_file.relative_to(root)
if any(part in {".venv", "venv", "node_modules", "__pycache__", ".git"} for part in rel.parts):
continue
code, output = _run(["python", "-c", f"import ast; ast.parse(open('{py_file}').read())"], root)
if code != 0:
result.errors.append(f"{rel}: SyntaxError")
result.files_with_issues.append(str(rel))
def _scan_node(root: Path, result: ScanResult) -> None:
"""Run Node.js-specific checks."""
if not (root / "package.json").exists():
return
# npm test or yarn test
pkg_manager = "yarn" if (root / "yarn.lock").exists() else "npm"
code, output = _run([pkg_manager, "test", "--", "--reporter=min"], root, timeout=180)
if code != 0 and output:
result.test_output = output
result.files_with_issues.extend(_extract_file_paths(output, root))
# eslint
if shutil.which("npx"):
code, output = _run(["npx", "eslint", ".", "--format=compact", "--no-warn-ignored"], root, timeout=120)
if code != 0 and output:
result.lint_output = output
result.files_with_issues.extend(_extract_file_paths(output, root))
# TypeScript check
if (root / "tsconfig.json").exists():
code, output = _run(["npx", "tsc", "--noEmit"], root, timeout=180)
if code != 0 and output:
result.build_output = output
for line in output.splitlines():
if ": error " in line:
result.errors.append(line)
result.files_with_issues.extend(_extract_file_paths(output, root))
def _scan_rust(root: Path, result: ScanResult) -> None:
"""Run Rust-specific checks."""
if not (root / "Cargo.toml").exists():
return
code, output = _run(["cargo", "check", "--message-format=short"], root, timeout=300)
if code != 0 and output:
result.build_output = output
for line in output.splitlines():
if "error" in line.lower():
result.errors.append(line)
result.files_with_issues.extend(_extract_file_paths(output, root))
code, output = _run(["cargo", "test", "--no-run"], root, timeout=300)
if code != 0 and output:
result.test_output = output
def _scan_go(root: Path, result: ScanResult) -> None:
"""Run Go-specific checks."""
if not (root / "go.mod").exists():
return
code, output = _run(["go", "build", "./..."], root, timeout=180)
if code != 0 and output:
result.build_output = output
result.files_with_issues.extend(_extract_file_paths(output, root))
code, output = _run(["go", "vet", "./..."], root, timeout=120)
if code != 0 and output:
for line in output.splitlines():
result.warnings.append(line)
result.files_with_issues.extend(_extract_file_paths(output, root))
code, output = _run(["go", "test", "-short", "-count=1", "./..."], root, timeout=180)
if code != 0 and output:
result.test_output = output
def _scan_generic_patterns(root: Path, result: ScanResult) -> None:
"""Grep for common error patterns across any codebase."""
patterns = [
("TODO.*FIXME", "TODO/FIXME markers"),
("HACK", "HACK markers"),
("BUG", "BUG markers"),
("XXX", "XXX markers"),
]
if not shutil.which("grep"):
return
for pattern, label in patterns:
code, output = _run(
["grep", "-rn", "--include=*.py", "--include=*.js", "--include=*.ts",
"--include=*.tsx", "--include=*.jsx", "--include=*.go", "--include=*.rs",
"--include=*.java", "--include=*.rb", "--include=*.cpp", "--include=*.c",
"-E", pattern, "."],
root,
)
if code == 0 and output:
lines = output.splitlines()
for line in lines[:10]:
result.warnings.append(f"[{label}] {line}")