Agentic Engineering Handbook

The definitive OpenAI, Anthropic, Google, MCP, Harness, Evals, and Production Agent Systems learning roadmap.

If this repository helps you, consider giving it a ⭐

---

Why This Repository?

The AI industry has entered the Agentic Era. Building production-grade AI systems now requires mastering agents, tool use, MCP, memory, long-running workflows, coding agents, agent harnesses, evals, and safety — but the knowledge is scattered across OpenAI blogs, Anthropic engineering posts, SDK docs, cookbooks, and research papers.

This repository consolidates 179 curated resources into one structured learning roadmap.

The goal: Become a world-class Agentic Engineer.

---

How To Use This Handbook

Pick the path that matches your starting point:

• New to agents: follow the Learning Roadmap from Phase 0 to Phase 6. Treat each Read First, Then Read, and Build Exercise as a checklist.
• Already building LLM apps: start at Phase 2 or Phase 3, then fill gaps in agent loop, tool calling, evals, and production engineering.
• Trying to build projects: use the phase-level Build Exercise prompts, then branch into Applied Practice Tracks for coding agents, security, code review, or SRE.
• Looking for references: jump to the Full Reading Table. Read P0 first, use P1 for implementation detail, and keep P2 as optional background.

---

Learning Roadmap

Phase 0 — Agent Loop From Scratch

If you treat Claude Code as a coding CLI, many capabilities can feel like magic: it reads files, runs commands, edits code, delegates work, and stays oriented during complex tasks.

From an engineering perspective, the core is much simpler:

model + tools + one loop.

Understanding that loop makes the rest of the system easier to reason about:

• When the agent should plan first, and when it should act immediately
• Why an explicit todo list reduces drift in longer tasks
• Why subagents improve exploration while protecting the main context
• How skills, MCP, and hooks each add capability around the same core loop

These pages are based on the upstream English Markdown tutorials from shareAI-lab/mini-claude-code, with added Study Notes and inline source code for this handbook.

Step	Page	Code
v0	Bash is All You Need	v0_bash_agent.py
v1	Model as Agent	v1_basic_agent.py
v2	Structured Planning	v2_todo_agent.py
v3	Subagent Mechanism	v3_subagent.py
v4	Skills Mechanism	v4_skills_agent.py

Supporting files are included in the same folder: requirements.txt, .env.example, v0_bash_agent_mini.py, and skills/.

Next reference: minion.py is a compact, single-file coding agent worth reading after this lab. It shows how an OpenAI-compatible agent loop grows to include tool calls, sessions, resume, approvals, memory, and context compaction.

---

Phase 1 — Agent Foundations

Build shared vocabulary for workflow vs agent, tool loop, handoff, guardrails.

Key Mental Models

Should I build an agent? (4-question checklist from Barry Zhang's talk - Anthropic)

Question	If No → Workflow	If Yes → Agent
Is the task complex enough?	Decision tree is fully mappable	Ambiguous problem space
Is the task valuable enough?	<$0.10 per run	>$1 per run, cost doesn't matter
Are all core capabilities doable?	Weak links break the chain	Model handles every step well
Is error cost low & detectable?	High cost + hard to detect → human-in-the-loop	Errors caught by tests/CI

Think like the agent. Most failures come from designing with a human perspective. Put yourself inside the agent's context window: you only see ~10K–20K tokens (system prompt + tool descriptions + recent observations). Ask: does the agent have enough information to act correctly at each step?

→ Source: How We Build Effective Agents

Read First

#	Title	Vendor
1	System Prompts	Anthropic
2	Prompt guidance	OpenAI
3	Function Calling	OpenAI
4	Tool use overview	Anthropic
5	Function calling - Gemini API	Google
6	Building effective agents	Anthropic
7	New tools for building agents	OpenAI
8	Agents SDK overview	OpenAI

Then Read

Title	Vendor
How We Build Effective Agents: Barry Zhang, Anthropic	Anthropic
Phistory — Claude Code & Codex CLI System Prompt Diff History	Community
Coding Agents 101: The Art of Actually Getting Things Done	Cognition
OpenAI Agents SDK examples	OpenAI
Structured Outputs for Multi-Agent Systems	OpenAI

Build Exercise

Build a customer service/ticket triage agent: router → specialist → evaluator, with all outputs constrained by structured schemas.

---

Phase 2 — MCP & Tool Ecosystem

Understand MCP server/client, remote vs local, tool loading, approval, connector boundaries.

Read First

#	Title	Vendor
1	Introducing the Model Context Protocol	Anthropic
2	MCP and Connectors	OpenAI
3	Building MCP servers for ChatGPT Apps and API integrations	OpenAI

Then Read

Title	Vendor
Code execution with MCP: Building more efficient agents	Anthropic
Writing effective tools for AI agents - with AI agents	Anthropic
Model Context Protocol - Codex	OpenAI
Build a Remote MCP server	Cloudflare
Introducing the MCP Registry	MCP
OpenAI Docs MCP	OpenAI
Build your ChatGPT UI	OpenAI

Build Exercise

Build a read-only repo/docs MCP server, then create an eval to verify the agent correctly cites documentation.

---

Phase 3 — Context, Memory & Skills

Learn to control context window, short/long-term memory, skills/plugins, CLAUDE.md/AGENTS.md.

Read First

#	Title	Vendor
1	Agent Skills Specification	Agent Skills
2	Effective context engineering for AI agents	Anthropic
3	The new rules of context engineering for Claude 5 generation models	Anthropic
4	How the Open Knowledge Format can improve data sharing	Google Cloud
5	How Long Contexts Fail	Drew Breunig
6	Context Rot	Chroma
7	Progressive disclosure	Claude-Mem
8	Equipping agents for the real world with Agent Skills	Anthropic
9	Agent Skills	Anthropic
10	Skills	OpenAI
11	Building Reliable Agents with Memory and Compaction	OpenAI

Then Read

Title	Vendor
Custom instructions with AGENTS.md - Codex	OpenAI
Best practices for Claude Code	Anthropic
Agent Skills - Codex	OpenAI
Skills in OpenAI API	OpenAI

Build Exercise

Implement the same task as a Skill/Plugin, then measure accuracy and token cost across three variants: no skill, long prompt, and skill-based.

---

Phase 4 — Harness & Long-Running Agents

Master agent runtime: event stream, thread, tool execution, state, sandbox, approval, recovery.

Read First

#	Title	Vendor
1	Unrolling the Codex agent loop	OpenAI
2	Unlocking the Codex harness: how we built the App Server	OpenAI
3	Agent Harness Engineering: A Survey	Academic
4	Effective harnesses for long-running agents	Anthropic
5	Orchestrate subagents at scale with dynamic workflows	Anthropic
6	Deep Agents	LangChain

Then Read

Title	Vendor
Deep research	OpenAI
Open Deep Research	LangChain
The next evolution of the Agents SDK	OpenAI
A harness for every task: dynamic workflows in Claude Code	Anthropic
Using PLANS.md for multi-hour problem solving	OpenAI
Build long-running AI agents that pause, resume, and never lose context with ADK	Google
Harness design for long-running application development	Anthropic
Scaling Managed Agents: Decoupling the brain from the hands	Anthropic

Build Exercise

Build a mini coding harness: plan file, shell tool, apply patch, test gate, event log, and resume capability.

---

Phase 5 — Coding & Workspace Agents

Compare Codex vs Claude Code product/SDK forms; learn multi-agent, IDE, workspace collaboration.

Read First

#	Title	Vendor
1	AGENTS.md	Agentic AI Foundation
2	Introducing Codex	OpenAI
3	Best practices for Claude Code	Anthropic
4	How Claude Code works in large codebases	Anthropic
5	Enabling Claude Code to work more autonomously	Anthropic

Then Read

Title	Vendor
Introducing the Codex app	OpenAI
Introducing workspace agents in ChatGPT	OpenAI
Apple's Xcode now supports Claude Agent SDK	Anthropic
Building Consistent Workflows with Codex CLI & Agents SDK	OpenAI
Best practices for Claude Code	Anthropic
The spec is dead, long live the spec!	Ravi on Product
How Anthropic teams use Claude Code	Anthropic
Multi-stack Web App Builds	Community

Build Exercise

Run both OpenAI/Codex and Claude Code style workflows on the same repo: issue → plan → patch → tests → PR summary.

---

Phase 6 — Evals, Safety & Production

Build pre/post-launch eval loop, trace loop, safety boundaries, permissions, regression monitoring.

Read First

#	Title	Vendor
1	Demystifying evals for AI agents	Anthropic
2	The six generations of AI agents and how to eval them	Braintrust
3	Agent observability powers agent evaluation	LangChain
4	Agent Evaluation Readiness Checklist	LangChain
5	Build an Agent Improvement Loop with Traces, Evals, and Codex	OpenAI
6	Macro Evals for Agentic Systems	OpenAI
7	Testing Agent Skills Systematically with Evals	OpenAI

Then Read

Title	Vendor
How we build evals for Deep Agents	LangChain
Deep Research Bench	FutureSearch
How to Evaluate Tool-Calling Agents	Arize
AI agent evaluation: How to test, debug, and improve agents in production	Arize
A Survey on Agent-as-a-Judge	Academic
Running Codex safely at OpenAI	OpenAI
How we contain Claude across products	Anthropic
Evals API Use-case - MCP Evaluation	OpenAI
Measuring AI agent autonomy in practice	Anthropic

Build Exercise

Build a smoke/macro eval suite for your agent: task success rate, tool misuse, prompt injection resistance, latency, cost, and human approval count.

---

Applied Practice Tracks

Use these tracks after the core roadmap when you want to practice agentic engineering in real engineering workflows.

Track	Start Here	Why It Matters
Agentic coding workflow	Coding Agents 101, How Claude Code works in large codebases, How Anthropic teams use Claude Code	Turns agent theory into day-to-day engineering habits: prompting, checkpoints, verification, parallel work, and team rollout.
Spec-driven building	The spec is dead, long live the spec!, Multi-stack Web App Builds	Treats specs, prompts, and assignments as executable source material for agents.
Context failure modes	How Long Contexts Fail, Context Rot, Progressive disclosure	Helps diagnose context poisoning, distraction, confusion, context degradation, and retrieval overload.
Evals and observability	Demystifying evals for AI agents, Agent observability powers agent evaluation, Agent Evaluation Readiness Checklist	Builds the feedback loop for traces, datasets, graders, offline/online evals, and regression gates.
Deep research agents	Deep research, Open Deep Research, Alibaba-NLP/DeepResearch	Practices long-running research agents: planning, search, MCP, citations, report synthesis, and benchmark-driven improvement.
MCP operations	Build a Remote MCP server, Introducing the MCP Registry	Shows how MCP moves from local prototypes to authenticated, discoverable, production-grade tool ecosystems.
Agent security	OWASP Top Ten, SAST vs. DAST vs. RASP, Copilot Remote Code Execution via Prompt Injection	Grounds agent security in classic AppSec plus new prompt-injection and tool-permission failure modes.
Code review systems	How to Review Code Effectively, AI-Assisted Assessment of Coding Practices in Modern Code Review, AI Code Review Implementation Best Practices	Connects human review quality with AI-assisted review, automated comments, and review policy design.
Production and SRE agents	ML and LLM system design, Introduction to Site Reliability Engineering, Observability Basics You Should Know	Extends agents beyond coding into incidents, observability, root-cause analysis, on-call, and production operations.

---

Full Reading Table

Priority guide: P0 = must-read (architectural/conceptual), P1 = highly useful (implementation detail), P2 = optional context (background/releases).

Priority	Title	Vendor	Topic	Key Idea	Date
P0	OpenAI for Developers in 2025	OpenAI	Agents; MCP; Platform	Annual overview: systematic walkthrough of Responses API, Agents SDK, AgentKit, Codex, MCP, Apps SDK, and AGENTS.md.	2025-12-30
P0	New tools for building agents	OpenAI	Agents; Responses API; Tools	Key starting point for OpenAI's agent platform: Responses API, built-in web/file/computer tools, Agents SDK, tracing/observability.	2025-03-11
P0	Introducing AgentKit	OpenAI	Agents; Evals; AgentKit	AgentKit, expanded evals, agent RFT: the official agent toolchain from prototype to production.	2025-10-06
P0	Prompt guidance	OpenAI	Prompting; Models; Agent UX	Official model-specific prompting guidance for outcome-first prompts, reasoning effort, preambles, and validation rules in tool-heavy workflows.	Current docs
P0	System Prompts	Anthropic	System prompts; Claude; Behavior	Claude web/mobile system prompt release notes; useful for studying production prompting patterns and behavioral scaffolding.	Current docs
P0	Agents SDK overview	OpenAI	Agents; SDK	Official SDK entry point: concepts and boundaries of agent, tool, handoff, guardrail, and tracing.	Current docs
P0	Introducing the Model Context Protocol	Anthropic	MCP; Standards	The origin article for MCP: an open standard connecting AI assistants to data, tools, and systems.	2024-11-25
P0	Building effective agents	Anthropic	Agents; Patterns; Frameworks	Essential agent primer: workflow vs agent, prompt/tool/retrieval, orchestrator-worker, evaluator-optimizer patterns.	2024-12-19
P0	Coding Agents 101: The Art of Actually Getting Things Done	Cognition	Coding agents; Workflows; Practice	Product-agnostic guide to prompting, delegation, verification, environment setup, security, and cost management for coding agents.	2025-06
P0	minion.py	Sentdex	Agent loop; Coding agents; Reference implementation	Compact single-file coding agent showing OpenAI-compatible model calls, tool-call parsing, sessions, resume, approvals, memory, and context compaction in runnable Python.	Current repo
P0	AGENTS.md	Agentic AI Foundation	Codin