What is an AI agent and how does it actually work?
The word “agent” is everywhere in the AI world today — agentic coding, agentic workflows, multi-agent systems. Beneath the marketing layer, however, lies a surprisingly simple principle: a language model running in a loop that is allowed to call tools. In this article we break that loop down step by step, build a minimal agent in plain Python and in LangChain, and look at what separates a weekend demo from an agent you’d trust with production data.
Map of the article
First we separate an agent from a workflow, then we dissect the agent loop and finally move on to operations: memory, tools, multi-agent systems, typical failures and rules for production deployment.
A chatbot answers, an agent acts
The classic use of an LLM is one-shot: you send a prompt, you get an answer, the end. The model is a pure function text → text — it has no access to your data, can’t run anything and doesn’t know what happened after its answer. When you want the answer to take the contents of a database into account, you have to put it there yourself. When it generates code, you have to run it yourself. And when it got it wrong, you find out — the model doesn’t.
An AI agent extends this model with two key things: tools, with which it can act on the surrounding world — search the web, read files, call APIs, run code — and a loop, in which it decides what to do next until it completes the task. An agent thus maintains state between steps and adapts its next move to what it learned in the previous ones. It is precisely the feedback from the environment that makes an agent an agent: it can verify its plan, hit a wall, correct itself and try a different way.
A useful distinction was introduced by Anthropic in the often-cited text Building effective agents:
- Workflow — the LLM and tools are connected by a predefined path in code. The number of steps and their order are determined by the programmer. Example: “take an e-mail, classify it, generate a reply with template A or B according to the category”.
- Agent — the LLM itself controls which tools it uses and in what order. We don’t know the number of steps in advance. Example: “find the cause of this failing test and fix it”.
Agents make sense where the problem is open-ended and the number of steps unpredictable. If the task can be described by a fixed procedure, a workflow is usually cheaper, faster and more reliable. This decision is the very first one you should make when designing — and honestly: most tasks companies solve with an “agent” today would be handled better by a workflow.
The agent loop: Thought → Action → Observation
The core of every agent is a cycle first systematically described in the paper ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 2022). In it the model alternates three phases:
- Thought — the model works out in text what it needs to find out or do.
- Action — the model calls a specific tool with specific parameters.
- Observation — the tool’s result returns to the context and the model continues based on it.
The loop repeats until the model concludes that it has the final answer. The strength of ReAct is that thoughts and actions anchor each other: every tool call is justified and every further thought rests on a real result, not a hallucination. In the original paper the authors showed that this combination beats both pure chain-of-thought reasoning (which makes up facts) and blind action without deliberation.
In 2022 ReAct meant a text prompt with
Thought:/Action:/Observation:lines and a regular expression for parsing. Today the same loop is provided by native tool calling and the models’ internal reasoning (reasoning tokens) — the format has changed, the principle remains.
Anatomy of an agent: four building blocks
If you look at any agentic system — from a weekend script to Cursor or Claude Code — you’ll always find the same four components in it:
1. The model (brain)
The LLM is the only place where “decisions” are made. The quality of the agent therefore stands and falls with the model’s ability to plan, follow instructions and call tools correctly. Models are now deliberately trained for agentic use (post-training on trajectories with tools) and benchmarks like SWE-bench or τ-bench measure exactly this. No framework will save a weaker model.
2. Tools (hands)
A tool is any function with a description and a parameter schema. The quality of the descriptions is crucial — the model decides based on them when and how to use the tool. A good tool has a clear name, a description written for the model (not for a human), returns concise and structured outputs and is idempotent where possible, so that repeated calls do no harm.
3. Context and memory
The agent’s short-term memory is the conversation itself — a list of messages that grows with every step. Long-term memory (facts about the user, results of previous runs) must live outside the model: in a database, a vector store or files, and is pulled into the context as needed. More on that below.
4. Orchestration (skeleton)
The code that drives the loop: it assembles messages, executes tools, watches limits, saves state, logs. In a demo it’s a while loop, in production a state machine with persistence and recovery. This is where frameworks like LangChain or LangGraph live.
Tool calling: a structured JSON response
Modern models (GPT, Claude, Gemini and a number of open-source models) are fine-tuned to be able to return a structured function call instead of a text answer. You describe the available tools to the model (name, description, JSON schema of parameters) and the model, when it concludes it needs a tool, returns JSON with the function name and arguments. This is what the raw response of a model that decided to call a tool looks like:
{
"role": "assistant",
"content": null,
"tool_calls": [{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"city\": \"Brno\"}"
}
}]
}
Your code executes the function and sends the result back as a message with the tool role. It’s important to realise that the model itself runs nothing. It only generates a request; execution is always under your control. This is exactly where safety checks belong — validating arguments (the model can invent non-existent values), restricting permissions or asking the user for confirmation on destructive operations.
A minimal agent in Python
The whole principle fits into roughly thirty lines. The following example uses the OpenAI SDK, but the same scheme works with any model that supports tool calling:
import json
from openai import OpenAI
client = OpenAI()
def get_weather(city):
# zde by bylo volani skutecneho API
return f"V miste {city} je 24 stupnu a jasno."
TOOLS = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Vrati aktualni pocasi v zadanem meste.",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}]
messages = [{"role": "user",
"content": "Mam si zitra v Brne vzit bundu?"}]
while True: # agentni smycka
response = client.chat.completions.create(
model="gpt-4.1", messages=messages, tools=TOOLS)
msg = response.choices[0].message
messages.append(msg)
if not msg.tool_calls: # finalni odpoved
print(msg.content)
break
for call in msg.tool_calls: # Action
args = json.loads(call.function.arguments)
result = get_weather(**args) # Observation
messages.append({"role": "tool",
"tool_call_id": call.id,
"content": result})
Notice that the agent’s “intelligence” is not in our code — that just forwards messages. The model decides when to call a tool and when to answer. This is exactly what large agentic systems like Cursor, Claude Code or Deep Research do under the hood; they differ mainly in the number and quality of tools, context handling and production robustness.
The same with LangChain
As soon as agents and tools multiply, you don’t want to write the loop, argument parsing and history management over and over. This is exactly the work wrapped by LangChain — the most widespread open-source framework for building LLM applications. Its ecosystem has three main parts: LangChain itself (agents, tools, integrations for dozens of models), LangGraph (a low-level engine for more complex orchestration, on which the agents run) and LangSmith (observability and evaluation).
Our weather agent looks like this in LangChain:
from langchain.agents import create_agent
from langchain.tools import tool
@tool
def get_weather(city: str) -> str:
"""Vrati aktualni pocasi v zadanem meste."""
return f"V miste {city} je 24 stupnu a jasno."
agent = create_agent(
model="openai:gpt-4.1",
tools=[get_weather],
system_prompt="Jsi uzitecny asistent. Odpovidej cesky.",
)
vysledek = agent.invoke(
{"messages": [{"role": "user",
"content": "Mam si zitra v Brne vzit bundu?"}]})
print(vysledek["messages"][-1].content)
The whole agent loop from the previous chapter has disappeared into create_agent. The tool decorator also generated the tool’s JSON schema automatically from the type annotations and docstring — no manual writing of definitions. And because you pass the model as just a string ("openai:gpt-4.1"), you switch to Claude or a locally hosted model by changing one line.
The real added value, though, shows up with things you’d otherwise write yourself: the checkpointer parameter adds conversation persistence (the agent survives a process restart), middleware lets you step into the loop — e.g. summarise a long history, have a human approve sensitive tools or limit the number of iterations — and hooking up LangSmith records every trajectory for debugging.
A framework isn’t a requirement, though. For simple agents a bare loop is often more readable and easier to debug; reach for LangChain when you need persistence, orchestration of multiple agents or ready-made integrations. And above all: even with a framework your knowledge of the bare loop stays with you — that is exactly what happens inside.
Memory and context management
This is the most underrated part of agent engineering. The context window is finite and every tool result takes a bite out of it — a file listing can easily be thousands of tokens, a search result tens of thousands. A long agent run will therefore sooner or later hit a wall: either the hard limit of the window, or the softer problem that models reason worse with a growing and “cluttered” context (so-called context rot; it relates to the attention U-curve, which we cover in a separate article). In practice three techniques are used:
- Trimming and summarisation — old steps of the conversation are progressively replaced by a summary. You lose detail, you keep the thread.
- Offloading outside the context — bulky tool outputs are saved to disk or storage and only a reference and a short extract go into the context; the agent requests the detail only when it needs it.
- Structured state — instead of relying on free text in the history, the agent keeps explicit state (a to-do list, notes, intermediate results), which is always inserted into the prompt whole and current.
This discipline has come to be called context engineering, and in long agentic tasks it determines success more than the choice of framework.
Multi-agent systems: when one agent isn’t enough
When one agent stops being enough — typically because the task has more independent parts than fit into one context — it’s time to divide the work among multiple agents. The most proven pattern is orchestrator–workers: the main agent breaks the task into subtasks and assigns each to a specialised sub-agent with its own clean context and a narrower set of tools. The sub-agent returns only a condensed result, so the orchestrator works with the essence, not with hundreds of thousands of tokens of raw data. This is exactly how, for example, Anthropic’s multi-agent research system works.
At the same time a sober warning applies: a multi-agent system is many times more expensive (every agent consumes tokens), harder to debug, and errors compound in it. Start with one agent and split it only when you hit a real limit — not because it’s fashionable.
How agents fail
Before you go to production, expect the typical failures:
- Hallucinated arguments — the model calls a tool with an ID that doesn’t exist or a path it made up. Defence: validation at the tool’s input and error messages written to help the model correct itself.
- Looping — the agent repeats the same action with the same result (typically after an unclear error). Defence: an iteration limit, detection of repeated calls and the instruction “if it fails three times, stop and describe the problem”.
- Premature surrender or false success — the agent declares the task done even though it isn’t (the most insidious failure, because it looks like success). Defence: independent verification of the result — by a test, a check query, a second model in the role of critic.
- Context poisoning — a wrong intermediate result settles in the history and all further decisions build on it. Defence: structured state, checkpoints and the ability to go back to before the error.
What production needs on top
Between a thirty-line demo and a reliable production agent lies a chasm. In practice you mainly deal with these things:
- Iteration limit and budget — a loop without a ceiling can run (and spend) forever. Always set a maximum number of steps and tokens per task.
- Context management — see above; without summarisation and offloading, long runs degrade.
- Observability — without a record of the whole trajectory (thoughts, calls, results, latency, cost) you can’t debug an agent failure at all. Tools: LangSmith, Langfuse, OpenTelemetry.
- Guardrails — tool permissions (read-only vs. write), sandboxing of executed code, human confirmation of irreversible actions (human-in-the-loop), protection against prompt injection from data the agent reads.
- Evaluation (evals) — agents are non-deterministic; without a set of test scenarios you won’t know whether a prompt change improved something or broke it. Evals for agents measure not only the final result but also the quality of the trajectory (number of steps, unnecessary calls, cost).
LangChain, mentioned above, helps with most of these things, as does its low-level foundation LangGraph or the Microsoft Agent Framework. For standardised tool connection there is the MCP (Model Context Protocol) — an MCP server with tools written once can be connected to any agent that supports the protocol. We devoted a separate article to MCP and the gateways that keep it under control. It pays, though, to understand the bare loop first — frameworks only wrap it.
When (not) to deploy an agent
The last piece of advice is perhaps the most valuable: an agent is the most expensive and least predictable tool in the box. If one well-written prompt can handle the task, use a prompt. If the task has a fixed procedure, use a workflow. Deploy an agent where you really need dynamic decision-making — exploring an unknown codebase, debugging an error, research across sources. And even there give it clear tools, clear limits and the option to say “I don’t know”.
Want to get hands-on with agents?
This is exactly what our AI Agent Developer course is about — from the basic loop through MCP and RAG to production deployment with observability and evals. If you’re rather looking for AI assistants in the developer workflow (Cursor, Claude, MCP), take a look at Vibe Coding and AI in Development.
Sources and further reading
- Yao et al.: ReAct: Synergizing Reasoning and Acting in Language Models (2022)
- Anthropic: Building effective agents (2024)
- Anthropic: How we built our multi-agent research system (2025)
- OpenAI: Function calling — documentation
- LangChain: Agents — documentation
- Lilian Weng: LLM Powered Autonomous Agents (2023)
- Jimenez et al.: SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (2023)
- Model Context Protocol: modelcontextprotocol.io
