How to Build an AI Agent: The Anatomy Plus a Worked Example
The bottom line: An AI agent sounds like magic as long as you treat it as a black box. Open it up and you always find the same five parts. The anatomy stays identical.
Why I Think in Parts, Not Syntax
I built several products and workflows in nine months without writing a single line of code myself, here in our workshop in Hamburg. That was possible not because I secretly know how to program, but because I stopped thinking in syntax and started thinking in parts.
In short: you do not need to train a model. You only need to know which part does which job. If you are not yet clear on what separates an agent from a chatbot, read What is an AI agent first. This article picks up from there and shows the blueprint. For the wider context, two good starting points are Anthropic's Building effective agents and the LangChain documentation.
What Are the Five Parts of Every Agent?
Picture an agent as a new colleague on their first day. They need a head to think with, a clear task, tools, a memory, and a working rhythm. Those five things make up every agent.
1. The Model: the brain
The model is the language model at the core, such as Claude or a Generative Pre-trained Transformer (GPT). It is the thinking part. It understands language, draws conclusions, and decides what to do next. The catch: the brain alone does nothing. It cannot reach your inbox, cannot remember anything, and cannot start on its own. It waits until it is given a task and tools. A brilliant brain with no hands and no brief is just a very expensive conversation partner.
2. The Instructions: the job description
The instructions tell the model who it is and what its job is. This is the system prompt: role, goal, rules, tone, limits. A good job description is concrete. "Be helpful" is not an instruction, it is a wish. "Sort every incoming email into one of four categories and never reply to invoices on your own" is an instruction. The sharper the job description, the more reliable the agent.
3. The Tools: the hands
Tools are the agent's hands. Without them it can only talk. With them it can act: read an email, create a record in a database, run a search, save a draft. Each tool is a clearly defined action with an input and an output. Anthropic's tool use documentation explains the mechanics: the model decides when to reach for a tool and writes the matching input. Through open standards like the Model Context Protocol, these tools can plug into existing systems without building a custom connector for every connection.
4. The Memory: what it remembers
The model itself forgets everything after each call. Memory is the layer that fills that gap. It ranges from short term (what has happened so far in this task) to long term (what the agent has stored about earlier cases, customers, or rules). Without memory the agent starts from zero every time, like a colleague who forgets their own name after every break.
5. The Loop: perceive, act, repeat
The loop is the working rhythm. An agent rarely solves a task in one step. It perceives (what is on my plate?), acts (reaches for a tool), checks the result, and decides whether it is done or needs another round. Anthropic describes this same loop in Building effective agents as the core of any real agent. It is what separates a real agent from a simple command: it keeps working until the goal is reached, instead of answering just once.
The table below sums up the five parts.
| App | Role | Example inbox agent |
|---|---|---|
| Model | thinks, understands language | reads: complaint, invoice, or request? |
| Instructions | set the role and limits | "never reply to invoices on your own" |
| Tools | act in real systems | read email, save draft, route |
| Memory | remembers across calls | learns: this sender always sends invoices |
| Loop | keeps the rhythm | perceive, act, check, next email |
What Does a Concrete Agent Look Like? The inbox triage desk
Theory is easy to forget. Let us walk the five parts through an example every leadership team knows: the overflowing inbox. Our agent reads incoming emails, classifies them, and either drafts a reply or routes them to the right person. Not a theoretical showcase, but a task that burns hours in every company, every day.
Part 1: the model as the brain
We use a strong language model as the brain. It needs to be good at understanding text and reading intent: is this email a complaint, an invoice, a request, or spam? That is exactly where today's models are strong. We train nothing new. We use the model as it is.
Part 2: writing the instructions
This is where the real work sits. The job description sets out how the agent sorts and where its limits are. Here is what such an instruction block looks like, simplified:
Role: You triage incoming emails for a team.
Task:
1. Read every new email.
2. Assign it to exactly one category:
Request, Invoice, Complaint, Other.
3. For "Request": write a reply draft,
but never send it yourself.
4. For "Invoice": route to accounting.
5. When unsure: flag the email and ask.
Limits:
- Never reply or delete on your own.
- Every action is shown to a human for approval.
Note the last rule. It is short, but it decides whether the agent becomes a helpful assistant or a liability.
Part 3: choosing the tools
This is exactly why the hands matter so much. According to Capgemini's Rise of agentic AI study (February 2026), fewer than one in five organizations report high maturity in the data and system infrastructure that AI agents need. The brain is not the bottleneck. The connection to the systems where the work actually happens is. Our agent needs four hands, so four clearly scoped tools:
- Read email: pulls the content of an incoming message.
- Assign category: stores the label on the email.
- Save draft: creates a reply draft without sending it.
- Route: forwards the email to a stored address.
That is all. There is deliberately no tool for sending directly. What the agent cannot do, it cannot get wrong.
Part 4: shaping the memory
Short term, the agent remembers which emails it already handled in this run, so it does not sort the same one twice. Long term, it can remember patterns: this sender always writes invoices, this subject line usually means a complaint. The sorting gets better over the weeks instead of starting from zero every day.
Part 5: closing the loop
Now it all comes together. The agent perceives a new email, reads it (tool), decides the category (model plus instructions), runs the matching action (draft or route), records the result (memory), and moves on to the next email. It repeats until the inbox is cleared. Five parts, one smooth workflow.
The software layer that holds these five parts together and keeps the loop running is called the agent harness. You do not have to build it yourself: ready-made harnesses ship the loop, tool calls, and memory out of the box. And if you would rather have an agent built for you, the packages and pricing are on our AI agents for business page.
Where Do First Agents Break?
The five parts are simple. Your first agent still tends to go wrong, almost always for the same four reasons. Knowing them saves you the expensive lessons.
The Four Most Common Failure Points
Vague instructions. "Sort the inbox sensibly" leaves too much open. The agent guesses, and guessing is unreliable. Write the job description concretely enough that a new human could follow it without a single question.
Too many tools. The more hands, the more ways to reach for the wrong thing. Give your first agent as few tools as possible. Expand only once the basics run reliably.
No human in the loop. An agent that sends emails or deletes data unchecked is not an assistant, it is a liability. Every serious action needs a human approval step, at least until trust is earned.
Hallucinated actions. Models hallucinate. An agent can claim it routed an invoice when it never did. Run every action through real tools and log what actually happened, instead of taking the agent at its word.
Notice the pattern: most of these failure points come not from the model, but from the instructions, the tools, and the missing approval step. That is the good news: the parts most likely to break are exactly the ones you can control yourself, with no coding knowledge.
The bottom line is to start small. The skepticism is warranted: the same Capgemini study found that trust in fully autonomous AI agents fell from 43% to 27% in a single year. A tightly scoped first agent with a human approval step is not a step back, it is the only sensible way in.
From Blueprint to a Running Agent
You now know the anatomy. An agent is not a magical creature, it is five nameable parts working together. The next question is not "How do I program this?" but "Which task in my company do I hand to an agent first?".
If you are weighing whether a ready-made tool is enough or a custom build makes sense, the decision guide is in Build vs. Buy for custom software. And if you want the fastest path from blueprint to a working system, look at AI prototype development in four weeks: instead of planning for months, you have a running agent after four weeks and learn real lessons from it.
If you want to know which tasks are even suited to a first agent, read on in the catalog of AI agent use cases for the Mittelstand.
FAQ
Do I need to know how to code to build an agent?
No. Once you understand the five parts, you can plan, judge, and steer an agent without writing a line of code. The parts most likely to break are the instructions, the tool selection, and the approval step, which are exactly the points you control at a domain level. The technical plumbing comes from ready-made frameworks like LangChain or an agent harness.
How long does a first agent take?
A tightly scoped first agent, such as one for inbox triage, can be set up in days rather than months. The key is the narrow scope: one task, a few tools, a human in the loop. We typically ship a first working system in four weeks instead of planning for months.
Which part breaks most often?
Almost never the model. In practice, the problem sits in vague instructions, too many tools, or a missing human approval step. Once you know the five parts, every failure tells you straight away where to look.
Do I need a custom model for every agent?
No. For most business tasks, a strong off-the-shelf model like Claude or GPT works as it is. Custom training only pays off for very specialized requirements and is neither needed nor sensible for a first agent.
