AI Agent Fundamentals

Everything you need to reason about agents from first principles — before any framework gets involved.

What is an AI agent?

An AI agent is a system in which a language model decides the control flow: it chooses which action to take next (call a tool, ask a question, answer, hand off), observes the result, and repeats until the goal is met. That loop is what separates an agent from a single LLM call or a fixed pipeline.

Anatomy of an agent

• Model — the reasoning engine (Claude, GPT, Gemini, Llama…). Must support tool/function calling.
• Instructions — the system prompt: role, goals, constraints, output format, when to stop.
• Tools — functions with a name, description and JSON schema the model can call (APIs, DB queries, search, code execution).
• Memory / state — short-term (the message history of this run) and long-term (facts persisted across sessions).
• Orchestration loop — the code that calls the model, executes the tool calls it asks for, appends the results and loops.
• Guardrails — input/output validation, permission checks, human approval, step and cost limits.

The agent loop, without a framework

Write this once by hand. Every framework — LangGraph, CrewAI, the OpenAI Agents SDK, the Claude Agent SDK — is a more robust version of these ~40 lines.

import json
import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

def get_weather(city: str) -> str:
    return json.dumps({"city": city, "temp_c": 21, "sky": "clear"})

TOOLS = [{
    "name": "get_weather",
    "description": "Get the current weather for a city.",
    "input_schema": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"],
    },
}]
REGISTRY = {"get_weather": get_weather}

def run(user_msg: str, max_steps: int = 8) -> str:
    messages = [{"role": "user", "content": user_msg}]
    for _ in range(max_steps):                      # guardrail: bounded loop
        resp = client.messages.create(
            model="claude-sonnet-5-5", max_tokens=1024,
            tools=TOOLS, messages=messages,
        )
        messages.append({"role": "assistant", "content": resp.content})
        if resp.stop_reason != "tool_use":          # model is done
            return "".join(b.text for b in resp.content if b.type == "text")

        results = []
        for block in resp.content:
            if block.type == "tool_use":
                out = REGISTRY[block.name](**block.input)   # act
                results.append({"type": "tool_result",
                                "tool_use_id": block.id, "content": out})
        messages.append({"role": "user", "content": results})  # observe
    return "Stopped: step limit reached."

print(run("Should I bring an umbrella in Paris today?"))

Notice the three ingredients: reason (model call), act (execute tool), observe (append result) — repeated with a hard stop.

Tool calling in depth

The model never executes anything. It emits a structured request (name + JSON arguments); your code validates and runs it, then returns the result as a message.

Designing good tools

• Names and descriptions are prompts. Say what it does, when to use it, and when not to.
• Narrow, typed parameters. Use enums and required fields; validate with Pydantic before executing.
• Return compact, model-friendly output. Trim large JSON; include IDs the model can reference later.
• Return errors as data ("error: order not found") so the model can recover instead of crashing the loop.
• Make side-effecting tools idempotent (idempotency keys) — retries happen.
• Keep the toolset small per agent (≈5–15). Too many similar tools cause wrong selections; split into sub-agents instead.

MCP — Model Context Protocol

MCP is an open protocol for exposing tools, resources and prompts to any compatible agent through a standard client/server interface (stdio or HTTP). Instead of writing a bespoke integration per framework, you run an MCP server once (e.g. GitHub, Postgres, Slack) and any MCP-capable agent can use it. LangGraph consumes MCP tools via the langchain-mcp-adapters package.

Reasoning & planning patterns

Modern models also expose built-in extended thinking / reasoning modes. These improve multi-step planning inside a single call, but you still need the outer loop for acting in the world.

Memory

• Short-term (thread) memory — the conversation and intermediate state of one session. In LangGraph this is persisted by a checkpointer keyed by thread_id.
• Long-term memory — facts that survive across sessions (user preferences, past resolutions). Stored in a database or vector store, namespaced per user, written by the agent or a background process.
• Types of long-term memory: semantic (facts), episodic (past experiences / few-shot examples), procedural (updated instructions).

Managing the context window

• Trim old messages (keep the system prompt + last N turns).
• Summarise older history into a running summary message.
• Store bulky tool outputs outside the context and pass references.
• Use prompt caching for long, stable prefixes (system prompt, tool definitions) to cut cost and latency.

RAG and agentic RAG

Retrieval-Augmented Generation grounds the model in your data: chunk documents → embed → store in a vector DB → at query time retrieve the top-k similar chunks → put them in the prompt.

Agentic RAG exposes retrieval as a tool. The agent decides whether to search, rewrites the query, grades retrieved documents for relevance, and searches again if needed. This handles multi-hop questions that one-shot RAG can't.

Multi-agent systems

Why split into multiple agents? Focused prompts, smaller toolsets per agent, isolated context windows, independent testing, and the option to use different models per role. Why not? Every hop adds latency, tokens and failure modes. Start with one agent.

Guardrails & safety

• Prompt injection: treat tool outputs and retrieved docs as untrusted data. Never let them grant new permissions.
• Least privilege: tools get scoped credentials; read-only where possible.
• Human-in-the-loop for irreversible or high-value actions (refunds, emails, deletes).
• Budgets: max steps, max tokens, timeouts, per-user rate limits.
• Validation: schema-validate tool args and structured outputs; PII redaction on input/output.
• Audit: log every tool call with who/what/when for traceability.

Evaluation & observability

• Tracing: capture every LLM call, tool call, latency and token count (LangSmith, Langfuse, Arize Phoenix, OpenTelemetry).
• Offline evals: a dataset of inputs + expected outcomes, scored by exact match, heuristics, or LLM-as-judge. Run in CI on every prompt/model change.
• Trajectory evals: did the agent call the right tools in a sensible order, not just produce a good final answer?
• Online metrics: task success rate, escalation rate, thumbs up/down, cost per resolved task, p95 latency.

Framework landscape