Overview
What It Does
Eval Harness First packages a focused coding & development workflow for an AI agent. Build the evaluation harness that gates every fine-tuning run — golden sets, per-failure-mode graders, judge calibration, and base-model baselines. Use when starting a fine-tuning effort, when converting traces into an eval set, or when calibrating a judge against human labels. It is best suited to users who can review the resulting actions and provide only the accounts, files, or command access needed for the task. It is not a substitute for human approval on destructive, financial, security-sensitive, or public-facing actions.
Task ideas
Popular Use Cases
- Review and improve a code change
- Automate a repeatable development task
- Investigate failures and prepare a fix
- Work with repositories and developer tooling
Installation
Install this Agent Skill
Claude Code
npx skills add https://github.com/wshobson/agents --skill eval-harness-firstCommands derived from the public GitHub SKILL.md record. Checked 2026-09-02. Review the source before running them.
Before you start
Requirements
| Claude Code | Required / review |
| Codex | Required / review |
| Public SKILL.md source | Required / review |
| Review instructions and requested permissions before installation | Required / review |
| Paid service | Check source |
| Supported system | Check source |
Popularity context
Why It’s Popular
Eval Harness First is a verified Agent Skill from wshobson with a public SKILL.md, compatible with Claude Code, Codex.
Alternatives
