🪨 why use many token when few token do trick — Claude Code skill that cuts 65% of tokens by talking like caveman
Book to Agent Skills converter for Claude Code and Copilot
This tool takes technical books, docs, or source folders and turns them into a reusable skill with a chapter index, glossary, patterns, and cheatsheet. It keeps the heavy extraction work out of your prompt and lets your agent load only the parts it needs when you ask about a topic.
Videos about this repo
Builders who want to turn books and internal docs into something their agent can consult while they work.
You can ask about a book later without re-reading it or dumping the whole PDF into context.
What it does
Convert books and doc folders into skills
Accepts PDFs, EPUBs, DOCX, HTML, RTF, MOBI, Markdown, and plain text, plus folders, globs, or file lists.
Build on-demand chapter files
Generates `SKILL.md` plus per-chapter Markdown files, a glossary, patterns, and a cheatsheet so the skill stays usable inside context limits.
Choose the right extractor
Uses tools like `pdftotext`, `pypdf`, `pdfminer.six`, or `docling` depending on the book type and available dependencies.
Handle scans and bad inputs safely
Detects scanned PDFs, unreadable files, and other extraction problems early instead of producing empty or broken output.
Validate generated skills
Includes `tools/validate_skill.py` and scan utilities to check a generated skill against host rules.
Measure discovery cost
Includes performance tooling that compares the token cost of a full context dump with the structured skill workflow.
How to get it
- 1Scanned PDFs need OCR first. A PDF that is page images with no text layer — a…
ocrmypdf input.pdf output.pdf
README
book-to-skill
Turn any technical book, document folder, or collection of sources into a unified agent skill — ready to study, reference, and use while you work in GitHub Copilot CLI, Amp, Claude Code, or Hermes Agent.
Why · What it generates · Beyond books · How it works · Usage · Install · FAQ · Performance · Architecture · Changelog
24×–51× fewer tokens than dumping the book into context to answer one question, measured on real books (how it's measured).
How it works, in 3 steps:
- Point it at a file, folder, or glob —
/book-to-skill ./my-book.pdf - It distills the book into a skill — frameworks, decision rules, anti-patterns, and per-chapter files. Structure, not a summary.
- Your agent loads it on demand — ask
/my-book replicationand it reads the right chapter and answers from the real content, no hallucination.
🤔 Why
You buy a great technical book. You read it once. Three months later you can't remember chapter 7 existed.
The usual workarounds don't help:
- 📄 "Let me just search the PDF" → you get a list of pages, not answers
- 🧠 "I'll ask the agent about this book" → it either hallucinates or says it doesn't have the content
- 📝 "I'll take notes as I read" → you end up with a 200-line doc you never open again
book-to-skill solves this by turning the book into a structured skill your agent loads on demand.
Once installed, you just type /your-book-slug replication and the agent reads the right chapter and answers from the actual content. No hallucination. No digging through PDFs. The book becomes part of your workflow.
Works with any host that supports the open Agent Skills standard — GitHub Copilot CLI, Amp, Claude Code, and Hermes Agent all read the same SKILL.md format.
📦 What it generates
Running /book-to-skill your-book.pdf (or a folder, glob, or list of files) creates a full skill in your agent's skills directory (~/.copilot/skills/<slug>/ for Copilot CLI, ~/.agents/skills/<slug>/ for Amp or cross-agent, ~/.claude/skills/<slug>/ for Claude Code, or $HERMES_HOME/skills/<category>/<slug>/ for Hermes Agent):
| File | Purpose | Size |
|---|---|---|
SKILL.md | Core mental models + chapter index | ~4,000 tokens |
chapters/ch01-*.md … | One file per chapter, loaded on-demand | ~1,000 tokens each |
glossary.md | Every key term, alphabetically sorted with chapter refs | ~1,500 tokens |
patterns.md | All techniques, algorithms, and design patterns | ~2,000 tokens |
cheatsheet.md | Decision tables and quick-reference rules | ~1,000 tokens |
Chapter files are loaded on-demand — they don't count against the skill budget until you ask about that topic.
🏢 Beyond books
The name says "book", but the input is any structured prose. The same extraction works on knowledge you own and re-read constantly:
- Internal documentation — architecture decision records, runbooks, onboarding guides. Fold a whole
docs/folder into one skill and ask it while you code. - Brand & design systems — voice guidelines, tone-of-voice docs, component principles. Turn a brand book into a skill your team queries instead of skimming a 60-page PDF.
- Research clusters — a stack of papers plus your own notes, merged into a single unified skill and updated as new material lands (see Update / fold-in).
- Specs & standards — RFCs, API contracts, compliance docs you reference but never memorize.
If you re-open a document often enough to wish you'd memorized it, it's a candidate.
🧾 The Discovery Loop Tax
A PDF-reading agent doesn't just read — it navigates: it re-fetches the ToC, backtracks, and re-processes all of it on every turn. book-to-skill pays that structuring cost once, at conversion, so queries stay proportional to the answer — 24×–51× fewer tokens than dumping the book into context, measured on real books.
📊 Full methodology, numbers, and per-book tables → docs/performance.md
⚙️ How it works
Two halves: a deterministic Python extractor (document → clean text + metadata) and a spec-driven generator (your agent follows SKILL.md to turn that into a structured skill). On-demand chapter files keep the loaded skill small.
🔧 Full walkthrough (Steps 0–10, extraction modes, token budgets) → docs/how-it-works.md
🚀 Usage
/book-to-skill <path|folder|glob> [skill-name] — plus analyze-only, generate-from-analysis, and update/fold-in modes. After a conversion, the converter can publish the skill to GitHub (private by default) so any host installs it with npx skills add.
▶️ All modes and examples → docs/usage.md
💬 In practice → use cases — a DevEx book became a survey of 300+ engineers; a scanned PDF that stalled became #130. Add yours: the account lives in your own Gist, the index takes a one-line PR.
📥 Install
# One command, any host — via the cross-agent skills CLI:
npx skills add virgiliojr94/book-to-skill
# Or manually — clone into your skills folder (registers /book-to-skill):
git clone https://github.com/virgiliojr94/book-to-skill.git ~/.claude/skills/book-to-skill
# (Copilot CLI: ~/.copilot/skills/ · Amp/cross-agent: ~/.agents/skills/)
# (Hermes Agent: ${HERMES_HOME:-$HOME/.hermes}/skills/<category>/)
📥 All hosts, optional extractors, and the standalone CLI → docs/install.md
❓ FAQ
Common questions — "why not just dump the PDF?", cost, privacy, non-book inputs, multi-file books.
❓ Answers → docs/faq.md
🔧 Requirements
The extractor tries tools in order per format and uses the first available. If nothing is installed, it tells you which command to run. Plain text, Markdown, reStructuredText and AsciiDoc need no extra deps.
Check your setup in one command:
python3 scripts/extract.py --checkprints which extractors are installed for every format and the exact command to install anything missing — no file needed.
PDF — choose by book type:
| Book type | Tool | Install | Speed |
|---|---|---|---|
| Text-heavy (prose, few tables) | pdftotext (poppler) | sudo apt install poppler-utils | ⚡ instant |
| Text-heavy fallback | pypdf | pip3 install pypdf | ⚡ instant |
| Text-heavy fallback | pdfminer.six | pip3 install pdfminer.six | ⚡ instant |
| Technical (code, tables, formulas) | docling | pip3 install docling | ~1.5s/page |
Before extraction begins, the skill asks you whether the book is technical or text-heavy and picks the right tool automatically. Docling preserves markdown tables and code blocks; pdftotext is faster for prose-only books.
Scanned PDFs need OCR first. A PDF that is page images with no text layer — a photographed or scanned book — has nothing for these tools to extract. The extractor checks the first pages and stops immediately with an explanation, rather than working through the whole book to produce an empty skill. Run OCR yourself, then convert the result:
ocrmypdf input.pdf output.pdf
EPUB:
| Tool | Install | Quality |
|---|---|---|
ebooklib + beautifulsoup4 | pip3 install ebooklib beautifulsoup4 | ⭐⭐⭐ Best |
stdlib zipfile | built-in — no install needed | ⭐⭐ Always available |
Other formats:
| Format | Tool | Install |
|---|---|---|
| DOCX | python-docx (fallback: stdlib ZIP/XML) | pip3 install python-docx |
| HTML | beautifulsoup4 (fallback: stdlib html.parser) | pip3 install beautifulsoup4 |
| RTF | striprtf (fallback: regex) | pip3 install striprtf |
| MOBI / AZW / AZW3 | Calibre ebook-convert (external app, not pip) | https://calibre-ebook.com/download |
| TXT / Markdown / reStructuredText / AsciiDoc | built-in | — |
📁 Repository structure
book-to-skill/
├── SKILL.md # Skill definition + step-by-step instructions (the generator spec)
├── scripts/
│ ├── extract.py # Thin entrypoint wrapper
│ └── extractor/ # Modular extraction package
│ ├── config.py # Extensions, paths, dependency constants
│ ├── dependencies.py # optional-dep probing + --check
│ ├── exceptions.py # ExtractionError (per-source failures, batch-safe)
│ ├── utils.py # CLI parsing, multi-source resolution, chapter detection, runner
│ └── parsers/ # Format-specific parsers (pdf, epub, docx, html, rtf, calibre, text)
├── tools/
│ ├── discovery_tax.py # measures token cost vs context-dump / discovery loop
│ └── validate_skill.py # checks a generated SKILL.md against host rules (--lens claude|copilot|amp)
├── tests/ # pytest suite (extraction, detection, discovery tax)
├── docs/
│ ├── performance.md # measured benchmarks, discovery tax, cost
│ └── architecture.md # pipeline + component map
├── CHANGELOG.md # release history (semver)
├── CONTRIBUTING.md # dev setup, PR conventions, release process
├── SECURITY.md # vulnerability reporting
└── README.md # This file
⚖️ Copyright & fair use
book-to-skill ships no book content — not a single page. It's a converter you point at files you already own.
- Processing is local. Extraction and analysis run on your machine. Your files are never uploaded by this tool. (If your agent's model runs in the cloud, the text you feed it follows that provider's normal data terms — same as any prompt.)
- You use your own copy. Bring a book you bought, docs your company owns, or papers you have the right to read.
- The output is your notes. A generated skill is a structured, synthesized derivative — framework names, definitions, takeaways — not a reproduction of the text. The skill explicitly never copies raw passages (see Quality Rule #7). Treat it like handwritten study notes: yours, for personal use.
- Don't redistribute. Publishing or sharing a generated skill of a copyrighted work can infringe the rights holder. Keep skills of third-party books private. Internal docs, your own writing, and openly-licensed material are fine to share within the bounds of their license.
When in doubt, follow the license or terms of the source document. This project is a tool; how you use it is on you.
💖 Sponsors
book-to-skill is free and MIT-licensed, maintained on personal time. If it saves you tokens or study hours, consider sponsoring its upkeep: PR reviews, multilingual fixes, releases, and docs.
Become a sponsor → github.com/sponsors/virgiliojr94
Every sponsor is listed in BACKERS.md. Thank you for keeping open, privacy-first tooling alive. ✨
License
MIT — applies to the converter (code + skill definition) in this repository, not to any book or document you process with it.
Files in the repo
- .github
- book_to_skill
- docs
- evals
- overrides
- scripts
- tests
- tools
- .gitignore
- AGENTS.md
- BACKERS.md
- CHANGELOG.md
- CLAUDE.md
- cliff.toml
- CONTRIBUTING.md
- LICENSE.md
- mkdocs.yml
- pyproject.toml
- README.md
- README.ru.md
- README.zh-CN.md
- SECURITY-NOTICE.md
- SECURITY.md
- SKILL.md
Discussion (0)
Ask about usage, or say what you built with itSign in to join the discussion.
No comments yet. Be the first to say what this is good for.
More tools
The best-benchmarked open-source AI memory system. And it's free.
Orca is the ADE for working with a fleet of parallel agents. Run any coding agent with your own subscription. Available on desktop, mobile and remote runtime.

A cross-platform desktop All-in-One assistant for Claude Code, Codex, OpenCode, OpenClaw, Grok Build & Hermes Agent. Only official website: ccswitch.io
Never stop coding. Free MIT AI gateway: one endpoint, 352 providers (150+ free), 1200+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 550+ contributors
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.