We spent $2,893 of compute studying how to set up Claude Code. This is what 46 popular repositories are worth.

Over a month, 34 research sessions read about 17,000 search results and pages and cited 1,040 sources, and Claude Code sessions reviewed 46 public repositories of skills, plugins, hooks and agents, each cloned at a fixed commit and read file by file. We then installed 24 configurations from them into Claude Code and measured what they cost and what they do.

Every run, by what it tested

All 1,539 scored runs of Claude Code in the study, in the order they ran. Most marks are filled: on build tasks, every hidden test passed in 75% of runs, which leaves a configuration little room to show a gain.

The repositories

Of the 46 repositories, we would install parts of 9 as they ship, borrow a mechanism or a prompt from 22, read 13 for reference, and skip 2. The repositories page says, for each one, what is worth taking, what to watch out for, and which of its claims held up when we read the code.

Worth installing, at least in part:

• ai-evals-course/evals-skills: the judge-validation and judge-prompt skills are worth using as written; the other six are reading material.
• anthropics/claude-plugins-official: security-guidance and the code-review rubric are worth using; do not install pr-review-toolkit as a whole.
• anthropics/skills: three skills worth installing as shipped (skill-creator, webapp-testing, frontend-design); the other five we read are reading-only.
• dmmulroy/anti-slop: copy the source into your repository as the README says, then prune the taste-heavy rules.
• hankimis/self-preference: a small harness that works as shipped for judge-bias checks; change the model list and prompts for your own task.
• mattpocock/skills: grilling, diagnosing-bugs, tdd and to-tickets are worth installing; the rest is personal workflow.
• obra/episodic-memory: useful cross-session recall once the mandatory wording is removed from the skill description.
• shimo4228/codex-review: the wrapper and its test suite are worth using as shipped; the skill text needs rewriting.
• spences10/svelte-claude-skills: the hook improves skill activation at a real token cost, and the measurement probe is the part most worth copying.

What we learned reading them

• Read the code, not the README. Widely repeated descriptions often did not match the source. Anthropic's security-guidance plugin is described as a hook that blocks unsafe edits; it is a reminder system with 25 pattern rules that never blocks one. garrytan/gstack's /ship is described as a deterministic gate; it is a 21-step pipeline with human questions.
• Numbers rarely come with a model or a method. A cookbook's "2.5× cheaper, 3× faster" was withdrawn by its authors. A "100% skill activation" result records no model, and its harness never counts a correct abstention. A "~95% more reliable" and a "saves 50–100× context" have no measurement behind them anywhere in their repositories.
• The few repositories that measure are the most useful. habit-hooks reports a pre-registered third-party test (coaching guides raised genuine fixes from 5.6% to 83.3%). obra/superpowers publishes dated phrasing experiments that cut against some of its own capital-letter rules.
• Small, sharp pieces travel best. A skill that will not start debugging until a failing command exists, a review wrapper that exits 64 on anything but a review, lint rules whose messages say what to do instead. Large bundles carried personas, rule walls and preambles that were half the file.
• Copies drift. The plugins bundled in Anthropic's Claude Code repository lag the marketplace versions, two different commands ship under the name code-review, and one popular loop repository is a copy of another.

What we measured on them

We installed 24 configurations from 22 of the repositories into stock Claude Code, each the way its README describes, and ran them beside stock on the same tasks.

• None broke a headless run. Three did not fully install as their READMEs describe: gstack's skills need its setup step, superpowers-skills is not discovered on its own, and episodic-memory's search server did not start from its shipped build.
• Most are cheap to carry. 15 of 24 add under 1,000 tokens to every session; the heaviest add 3,999 (AutomationPanda/gherkin-guidelines-for-ai), 2,432 (paulnsorensen/easy-cheese), 2,383 (EveryInc/compound-engineering-plugin).
• Skills often do not load on their own. Given prompts written to call for them, skill packs loaded the right skill anywhere from 0 to 10 times in 10. 3 packs scored 10 of 10; 2 scored 0, though their skills were installed and listed. The packs that ship a hook pushing skill use did best. None fired on a near-miss.
• They change the bill more than the result. On 12 real build tasks, per-task cost ranged from ×0.94 to ×1.13 of stock's (median ×1.00), rising about 2.6% for every 1,000 tokens of standing context. None showed a pass-rate drop large enough to flag, and none showed evidence of a gain.
• On real work, their skills and agents went unused. In 300 build runs with these repositories installed, Claude Code never invoked one of their skills or spawned one of their subagents. They acted only through the text they add to every request.

The repositories page has every measurement, per repository.

If you are choosing a setup

Start from stock Claude Code; on these tasks it is a strong baseline. Add small pieces that do one checkable thing, read their source before you install them, and measure what they cost you per task: whatever a repository puts in front of the model, you pay for again on every request.

A separate part of the project built one setup from the research and benchmarked it against stock Claude Code; that is written up under Our setup, with the method both parts share.