10 frameworks analyzed, 50+ sources synthesized, ranked by popularity. Everything you need to write skills that actually work.
SWE-agent's ACI design tripled benchmark success from 3.8% โ 12.5% without changing the model. Tool interface design is a primary performance multiplier.
Model Context Protocol has been adopted by Anthropic, OpenAI, Google, Microsoft, and every major IDE. 22,000+ servers indexed. The "USB-C for AI tools."
271M monthly downloads for LangChain, 109k stars for browser-use, 1,000+ Composio integrations. The agent tool ecosystem is exploding.
Clear naming, strict schemas, actionable errors, context-conscious output, single responsibility, and sandboxed safety โ validated by academic research.
Frameworks ranked by GitHub stars, PyPI downloads, and enterprise adoption.
| Rank | Framework | Stars | Category |
|---|---|---|---|
| 1 | AutoGPT | โญ 187k | Autonomous Agent Platform |
| 2 | LangChain | โญ 144k | Orchestration Framework |
| 3 | browser-use | โญ 109k | Browser Automation |
| 4 | awesome-mcp-servers | โญ 92k | MCP Community Index |
| 5 | MCP Reference Servers | โญ 79k | Universal Protocol |
| 6 | AutoGen | โญ 60k | Multi-Agent Research |
| 7 | CrewAI | โญ 57k | Role-Based Multi-Agent |
| 8 | Agno (Phidata) | โญ 42k | Multi-Modal Agent OS |
| 9 | DSPy | โญ 37k | Declarative LM Pipelines |
| 10 | Composio | โญ 29k | Tool Integration Platform |
| 11 | OpenAI Agents SDK | โญ 29k | Official OpenAI Framework |
| 12 | Semantic Kernel | โญ 28.5k | Enterprise .NET/Python |
| 13 | smolagents | โญ 26k | Minimalist Code Agents |
| 14 | Pydantic AI | โญ 19k | Type-Safe Python Agents |
| 15 | Mastra | โญ 19k | TypeScript Agent Stack |
| Rank | Package | Monthly Downloads |
|---|---|---|
| 1 | langchain | 271,000,000 |
| 2 | langchain-core | 170,000,000 |
| 3 | langgraph | 71,000,000 |
| 4 | browser-use | 55,700,000 |
| 5 | crewai | 11,000,000 |
| 6 | dspy | 6,600,000 |
| 7 | semantic-kernel | 2,700,000 |
| 8 | smolagents | 583,000 |
| 9 | pyautogen | ~450,000 |
| Framework | Notable Enterprise Users |
|---|---|
| LangChain / LangGraph | Klarna, Cisco, Stripe, Uber, LinkedIn, Databricks, PagerDuty |
| Semantic Kernel | Fujitsu (38k users), Accenture, KPMG, Microsoft 365 Copilot |
| CrewAI | PwC, IBM, AWS Bedrock, Citi, Capgemini, Gelato |
| MCP | Block (60+ servers), Cloudflare, Atlassian, JetBrains |
How each major framework defines, structures, and executes agent skills.
The most widely-adopted LLM orchestration framework. Tools defined via @tool decorator, BaseTool subclassing, or BaseToolkit collections.
Multi-agent orchestration with role/goal/backstory personas. Task-level tool scoping keeps context windows clean.
Official production framework with strict: true constrained decoding. Built-in code_interpreter, file_search, web_search.
Universal open protocol โ the "LSP for AI tools." Tools, Resources, and Prompts over JSON-RPC 2.0. 22k+ servers.
Enterprise SDK powering Microsoft 365 Copilot. First-class C#, Python, and Java support. 15M+ NuGet downloads.
Universal tool integration platform. Managed OAuth for 1,000+ SaaS apps. Bridges to LangChain, CrewAI, OpenAI, and MCP.
Breakout star of 2025-2026. Dual vision + DOM browser automation for LLM agents. Fastest-growing agent repo.
Hugging Face's minimalist ~1,000 LOC framework. Code Agents write Python instead of JSON โ loops, conditionals, multi-tool piping in one step.
Validated by academic research and production experience across all frameworks.
Use namespaced domain_verb_noun names. Write 3-4 sentence descriptions covering: what it does, when to use, when NOT to use, expected return format. Include explicit negative guidance.
Define Pydantic/Zod/JSON Schema with described parameters, enum constraints, numeric bounds, and format examples. Enable strict: true or additionalProperties: false.
Never return raw stack traces. Return structured diagnostics explaining what failed, why, and what the model should do differently. Use ToolException, isError: true, or handle_tool_error.
{"error": 500}{"error": "INVALID_FORMAT", "message": "Expected ISO 8601 'YYYY-MM-DD'", "fix": "Reformat to '2024-12-31'"}Never dump raw HTML, 5,000-row SQL results, or massive JSON blobs. Use pagination, column projection, content summarization, and artifact separation.
Each tool performs one logical operation. Decompose "God Tools" into purpose-built tools. Exception: tightly-coupled CRUD can use a strict action enum.
Read-only database defaults, sandboxed file paths, command allow/deny lists, Human-in-the-Loop gates for destructive operations, and idempotency keys for mutations.
Production-grade examples showing the principles in action.
from pydantic import BaseModel, Field from langchain_core.tools import tool, ToolException class WeatherQuery(BaseModel): city: str = Field(description="City name, e.g., 'San Francisco'") units: str = Field(default="celsius", description="'celsius' or 'fahrenheit'") @tool("get_weather", args_schema=WeatherQuery, response_format="content_and_artifact") def get_weather(city: str, units: str = "celsius"): """Retrieve current weather. Returns summary + full sensor data.""" if not city.strip(): raise ToolException("City name cannot be empty.") raw = {"city": city, "temp": 22, "humidity": 65} summary = f"Weather in {city}: {raw['temp']}ยฐ {units}" return summary, raw # (LLM content, full artifact)
server.tool( "query_customer", "Retrieve customer records by ID or email.", { customerId: z.string().optional().describe("Unique UUID"), email: z.string().email().optional().describe("Customer email") }, async ({ customerId, email }) => { try { const result = await db.query({ customerId, email }); return { content: [{ type: "text", text: JSON.stringify(result) }] }; } catch (err) { return { content: [{ type: "text", text: `Error: ${err.message}` }], isError: true // Enables LLM self-correction }; } } );
{
"type": "function",
"function": {
"name": "execute_readonly_sql",
"description": "Execute read-only SELECT queries. Mutations rejected.",
"parameters": {
"type": "object",
"properties": {
"query": { "type": "string", "description": "PostgreSQL SELECT query." },
"limit": { "type": "integer", "description": "Max rows (1-500)." },
"reasoning": { "type": "string", "description": "Why this query." }
},
"required": ["query", "limit", "reasoning"],
"additionalProperties": false
},
"strict": true
}
}
Common mistakes that break agent tool usage.
| Anti-Pattern | What Goes Wrong | Fix |
|---|---|---|
| "Tool Soup" (20+ overlapping tools) | LLM can't decide, infinite deliberation | Expose 3โ5 tools per step; dynamic retrieval for large catalogs |
| "God Tool" (one tool does everything) | Massive parameter hallucinations | Decompose into single-purpose tools or strict action enum |
| Context Blowout (raw HTML/SQL dumps) | Attention degradation, high cost | Pagination, summarization, artifact separation |
| Silent Failures (empty strings, generic errors) | Endless retry loops | Structured errors with specific fix suggestions |
| Unsandboxed Mutations (open DELETE, rm -rf) | Data loss, security vulnerabilities | Read-only defaults, HITL gates, allow/deny lists |
| Missing Async (sync-only implementations) | Blocks event loop in multi-agent | Implement both _run and _arun |
| Vague Parameters (no formats or constraints) | Wrong formats, type mismatches | Explicit examples, patterns, enum constraints |
Key papers that established the science of effective tool design.
| Paper | Year | Key Finding |
|---|---|---|
| SWE-agent / ACI Princeton ยท NeurIPS | 2024 | Interface design tripled SWE-bench from 3.8% โ 12.5% without changing the model |
| EASYTOOL ACL 2024 | 2024 | Purified docs: +25-30% accuracy, -60-80% tokens |
| ToolLLM / ToolBench Tsinghua | 2023 | Tree-search planning doubled multi-tool success vs. linear ReAct |
| Gorilla UC Berkeley | 2023 | Clean docs + retrieval drastically reduce API hallucination |
| Toolformer Meta AI | 2023 | LLMs can self-teach tool use; tools need discrete return formats |
| MetaTool | 2023 | LLMs over-call tools due to keyword matching; semantic boundaries critical |
| TALM Google Research | 2022 | Tools learned best with distinct, delineated text spans |
How Cursor, Windsurf, Cline, Copilot, and Antigravity customize agent behavior.
| # | Rule Category | Evidence |
|---|---|---|
| 1 | Next.js App Router โ Server Components, Server Actions | #1 on cursor.directory, 150k+ views |
| 2 | TypeScript Strict Mode โ Ban any, strictNullChecks | Top in awesome-cursorrules |
| 3 | Tailwind + Shadcn UI โ cn(), Radix accessibility | Top 3 on cursor.directory |
| 4 | Python FastAPI + Pydantic v2 โ Async, Depends() | Top Python rule |
| 5 | Rust Safety โ Ban unwrap(), thiserror enums | AGENTS.md collections |
| 6 | Go Clean Architecture โ if err != nil, context.Context | Enterprise copilot-instructions.md |
| 7 | TDD Self-Verification โ Test before claiming completion | Core pattern across all agents |
| 8 | AGENTS.md Git Workflow โ Feature branches, conventional commits | Linux Foundation standard |
| Repository | Stars | Focus |
|---|---|---|
| PatrickJS/awesome-cursorrules | โญ 40.6k | 100+ stack-specific rule files |
| cursor.directory | Millions of views | Searchable web directory + CLI |
| AGENTS.md Standard | Linux Foundation | Cross-agent open standard |
any" beats "write clean code"components/**/*.tsxnpm test, ruff check, tsc --noEmitStep-by-step checklist for building production-grade agent tools.
service_action_target)def query_users(role: str = "all", limit: int = 20) -> str: """ Search and list enterprise user accounts by role. WHEN TO USE: - Looking up user profiles, permissions, or listings. WHEN NOT TO USE: - Do NOT use for support tickets (use 'search_support_tickets'). - Do NOT use for modifying accounts (use 'admin_manage_user'). RETURNS: - JSON: {"users": [...], "total_count": 42, "has_more": true} EXAMPLES: - List admins: query_users(role="admin", limit=20) - Recent users: query_users(created_after="2024-01-01") """