Agentic System Architecture

A single LLM call (large language model — a program that generates text by predicting the next token, a token being a chunk of text about the size of a short word) is a one-shot transaction: you send a prompt, you get a reply, the interaction is over. It cannot look anything up, take an action in the world, or notice whether its answer was any good. It is a function from text to text.

An agent is an LLM placed in a loop with the ability to take actions. Instead of answering once, the model is given a goal, a set of tools (functions it can call — search the web, query a database, send an email), and permission to keep going: it decides on a step, takes it, observes the result, and decides again, repeating until the goal is met or a limit is hit. The single call answers a question; the agent pursues an objective across many calls, adjusting as it learns.

The engine of every agent is a four-phase cycle: perceive → reason → act → observe, repeated. Perceive is assembling what the model can currently see — the goal, the conversation so far, and any results from previous steps. Reason is the model deciding what to do next: answer directly, or call a tool, and which one with what arguments. Act is executing that decision — running the tool call. Observe is feeding the tool's result back into the model's view so the next cycle can use it. The loop ends when the model decides the goal is reached (it emits a final answer instead of another tool call) or when a safety limit stops it.

The critical difference from a plain call is the feedback edge from observe back to perceive: the agent's later decisions depend on the results of its earlier actions, which is what lets it correct course, retry, and handle tasks it could not plan out fully in advance.

A working agent is more than a model; it is a small system with named parts. The lead-in idea is that each part supplies something the raw model lacks — actions, memory, direction, or control.

• Model. The reasoning engine that decides each step. Its capability sets the ceiling on how hard a task the agent can handle.
• System prompt / policy. The standing instructions that define the agent's role, its rules, what it may and may not do, and how to use its tools. It is the agent's constitution, sent on every step.
• Tools / function calling. The actions the agent can take, described to the model as named functions with typed arguments. Function calling is the mechanism by which the model, instead of writing prose, emits a structured request to invoke one of those functions; the runtime executes it and returns the result.
• Memory. What the agent remembers. Short-term memory is the current context window — the bounded amount of text (measured in tokens) the model can see at once, holding the recent conversation and results. Long-term memory is an external store (a database or vector store) the agent writes to and reads from to remember beyond that window.
• Planning. Breaking a goal into ordered steps, either up front or incrementally as results arrive.
• Orchestration runtime. The code that actually runs the loop: it calls the model, executes the tool the model chose, appends the result, enforces limits, and decides when to stop. The model reasons; the runtime is what turns that reasoning into a running process.

Tools are what make an agent more than a talker — they let it read real data and change the real world. Each tool is described to the model with a name, a purpose, and a typed argument list, so the model knows when and how to call it. When the model wants to act, it returns a function-call request (tool name plus arguments as structured data); the runtime validates and executes it, then returns the result as the next observation.

Historically every framework described tools its own way, so a tool built for one agent could not be reused by another. MCP (Model Context Protocol) is the standard that fixes this: it is an open protocol for exposing tools, data, and context to models in a uniform way, so a capability is implemented once as an MCP server and any MCP-aware agent (the MCP client) can use it. It is to agent tooling what a common API schema was to the LLM gateway of the previous day — one contract, many consumers.

MCP matters architecturally because it decouples tool providers from agent builders: the same server works across agents and models, so tools become a shared, composable ecosystem instead of per-app glue.

The context window is finite, and a long-running agent quickly generates more history than fits. Context management is the discipline of deciding what to keep in that limited space: the system prompt and goal always, the most relevant recent steps, and summaries of older ones — dropping or compressing the rest so the model always sees what matters most within its token budget.

A key technique for feeding in outside knowledge is RAG (retrieval-augmented generation): before the model answers, the runtime fetches relevant documents from an external store (often by embedding the query and finding nearby stored text) and places them into the context, so the model reasons over fresh, specific facts instead of only what it memorized during training. RAG is how an agent grounds its answers in your data without retraining the model.

Many tasks fit a single agent: one model, one loop, one set of tools, pursuing the goal. It is the simplest thing that works, and it should be the default — a single agent is far easier to reason about, trace, and debug than several cooperating ones.

Some tasks are large or varied enough to split across multiple agents, each specialized. The common shape is orchestrator–worker: a lead agent decomposes the goal and delegates sub-tasks to worker agents (a researcher, a coder, a writer), each with its own tools and context, then integrates their results. A related pattern is handoff, where one agent passes control entirely to another better suited to the next phase (a triage agent hands a billing question to a billing agent).

Multi-agent buys specialization and parallelism — workers can run at once, each focused with a smaller, cleaner context — but it costs coordination overhead, more tokens, and harder debugging, because now failures can hide in the seams between agents. Rule of thumb: start with one agent; split into multiple only when a single context genuinely cannot hold the task or when clearly separable sub-tasks would benefit from running in parallel.

To make the parts concrete, trace a single agent asked to "find the three most-cited papers on X and write a one-paragraph summary of each." It has two tools: a search tool and a fetch-document tool, both exposed over MCP.

Notice the loop in action: the agent does not plan all steps up front. It searches, sees the results, then decides to fetch — each decision depends on the previous observation. Short-term memory (the context window) holds the running conversation and the fetched texts; if the papers were too long to all fit, the runtime would summarize older ones or store them in long-term memory and retrieve the relevant parts via RAG. A single agent is the right choice here because the task is one coherent thread of work, not separable sub-tasks.

This table collapses the day into concept-to-role pairs, each answering "what does this part contribute?"

• A single LLM call is a one-shot text-to-text function; an agent is an LLM in a loop with tools, pursuing a goal across many steps and adjusting as it learns.
• The agent loop is perceive → reason → act → observe, repeated; the feedback edge from observe back to perceive is what lets the agent self-correct.
• Core components are the model, the system prompt/policy, tools via function calling, memory (short-term context window vs long-term store), planning, and the orchestration runtime that actually runs the loop.
• Function calling lets the model emit structured tool requests; the runtime executes them and returns results as observations.
• MCP (Model Context Protocol) is the standard for exposing tools/data/context, so a capability is built once and reused across any MCP-aware agent — one contract, many consumers.
• Context management keeps the finite window filled with what matters; RAG retrieves outside facts into context so answers are grounded in your data without retraining.
• Prefer a single agent; adopt orchestrator–worker or handoffs only when one context cannot hold the task or separable sub-tasks benefit from parallelism, accepting the coordination and debugging cost.

• [ ] I can state the difference between a single prompt→response call and an agent.
• [ ] I can name and explain the four phases of the agent loop and why the feedback edge matters.
• [ ] I can list the core components and say what each supplies that the raw model lacks.
• [ ] I can explain function calling and how a tool result becomes an observation.
• [ ] I can define MCP and explain why a shared tool protocol matters architecturally.
• [ ] I can distinguish short-term context from long-term memory and describe RAG in one line.
• [ ] I can describe orchestrator–worker and handoff topologies and when a single agent is the better choice.