Classical AI theory often frames intelligent behavior through the lens of an agent: an entity that perceives its environment (via sensors) and acts upon it (via actuators).
Why this matters: This taxonomy helps you (1) choose an architecture that fits your environment and constraints, (2) debug failures by identifying what information/optimization the agent is missing, and (3) map classical ideas cleanly onto modern RL policies and LLM tool-using agents.
This is the textbook taxonomy popularized by Russell & Norvig, Artificial Intelligence: A Modern Approach (AIMA). It’s intentionally simplified—modern systems are often hybrids—but it remains a useful mental model.
What makes something an “agent”?
At minimum, an agent implements an agent function:
- Percept history → Action
This is a conceptual mapping: the agent does not need to store the entire percept history explicitly. In implementation, it may compress history into an internal representation.
Many readers (especially from RL) will map this to a policy:
- Policy (RL): typically (\pi(a\mid s)) or (\pi(a\mid o)), mapping state (s) (or observation (o)) to actions.
- Agent function (AIMA): mapping from percept sequence to actions.
A convenient bridge between the two is internal state (sometimes called a belief state in partially observable settings):
- Percepts update internal state (b_t)
- The policy maps (b_t) (or (s_t)) to an action
Fully observable vs partially observable (one-sentence definition)
- Fully observable: the current percept contains all information needed to choose an optimal action.
- Partially observable: the percept is incomplete/noisy, so the agent must infer hidden state.
Task environments and PEAS (with a mini-example)
Classical theory also emphasizes the task environment, often summarized as PEAS:
- Performance measure (what “success” means)
- Environment (the world the agent operates in)
- Actuators (how it acts)
- Sensors (what it perceives)
Mini-example (robot vacuum):
- P: % of floor cleaned, time, collisions
- E: apartment layout, dirt distribution
- A: move/turn/suction power
- S: bump sensor, lidar/camera, dirt sensor
With that framing, the categories below differ mainly in whether the agent uses only the current percept, maintains internal state (belief), reasons with models/goals/utilities, and/or improves via learning.
Quick comparison table
| Category | Observability assumption | Internal state (belief) | World model | Planning horizon | Objective type | Learning built-in? | |---|---|---:|---:|---|---|---:| | Simple reflex | Typically fully observable | No state beyond current percept | No | Immediate/reactive | Rules / condition-action | No | | Model-based reflex | Works under partial observability | Yes (state estimate / belief) | Yes (deterministic or probabilistic) | Short/reactive | Rules over inferred state | No | | Goal-based | Often needs state + model | Usually | Often | Longer (search/planning) | Goal satisfaction | Not required | | Utility-based | Often uncertain / multi-objective | Usually | Often probabilistic | Longer (optimize) | Max (expected) utility | Not required | | Learning agent | Any | Any | Any (may be learned) | Any | Improves performance measure | Yes |
1) Simple reflex agents
If the environment is simple and fully observable, you can skip state estimation and just react.
Defining idea: Choose an action based solely on the current percept, using condition–action rules.
- Mechanism:
if percept matches condition → do action - “No memory” in practice: no conceptual internal state used to summarize history. (An implementation may still have variables like timers or debouncing logic, but the action choice does not depend on an inferred world state.)
- Best for: fully observable, simple, stable environments
Example: A thermostat:
- If temperature < setpoint → turn heater on
- Else → turn heater off
Common failure mode (simple reflex): gets stuck in loops/oscillation when observation is ambiguous (e.g., a vacuum bouncing between two spots).
Strengths
- Fast and easy to implement
- Predictable behavior (often easier to verify)
Limitations
- Breaks down in partially observable environments
- Brittle when the right action depends on context not present in the current percept
2) Model-based reflex agents
If reflex rules aren’t enough because you can’t observe everything, you add state.
Defining idea: Still uses rules, but maintains an internal state derived from percept history and a model of how the world evolves.
- Key concept: state estimation / belief state (an inferred summary of what’s going on)
- Model: can be deterministic (next state is fixed) or probabilistic (next state is a distribution)
- Mechanism: update internal state using percept + world model → apply rules
Example: A robotic vacuum that tracks which rooms it has likely cleaned, even if sensors are noisy.
Common failure mode (model-based reflex): model mismatch → wrong belief state → systematically wrong actions.
Strengths
- Handles partial observability better than simple reflex agents
- Can avoid some repetitive/looping behaviors
Limitations
- Requires a reasonably accurate model; poor models lead to poor state estimates
- Still mostly reactive: rules may not capture long-horizon planning
3) Goal-based agents
Once you care about where you’re trying to get, rules over the current (or believed) state are often replaced by search/planning.
Defining idea: Select actions to achieve explicit goals.
- Mechanism: evaluate possible future action sequences → choose one that reaches the goal
- Goal satisfaction vs optimal planning: a goal-based agent may be satisfied with any plan that reaches the goal, whereas optimal planning additionally minimizes cost (time, distance, energy).
- Typical algorithms: graph search like BFS/DFS, informed search like A*, and classical planning formalisms such as STRIPS-style planning.
Example: Route planning:
- Goal: reach destination
- Agent considers roads and chooses a path that gets there
Common failure mode (goal-based): planning blow-up (state-space explosion) or timeouts in large environments.
Strengths
- More flexible than reflex agents: change the goal, and behavior changes without rewriting rules
- Supports deliberation and longer-horizon reasoning
Limitations
- Planning can be computationally expensive
- If multiple goal-achieving options exist, goals alone don’t specify which is best
4) Utility-based agents
When there are many ways to achieve goals—or outcomes are uncertain—decision theory gives you a principled selection rule.
Defining idea (decision-theoretic): choose actions to maximize expected utility under a probabilistic model.
- Utility can be defined over:
- States (e.g., “being at destination is good”),
- Outcomes (terminal results), or
- Trajectories (entire sequences, e.g., comfort over time).
- Mechanism: choose action (a) that maximizes (\mathbb[U \mid a])
- Risk note: expected utility is often treated as risk-neutral for simplicity; risk-averse behavior can be modeled via utility curvature or other risk-sensitive criteria.
Example (autonomous driving): separate hard constraints from soft preferences.
- Hard constraints: safety and legal compliance (must not violate; can be treated as constraints or extremely large penalties)
- Soft preferences: travel time, comfort, energy use (tradeoffs)
Common failure mode (utility-based): utility misspecification (the agent optimizes the wrong thing, sometimes very effectively).
Strengths
- Provides a principled way to choose among alternatives
- Naturally fits probabilistic reasoning and explicit tradeoffs
Limitations
- Designing a good utility function is hard
- Requires models (or learned predictors) for outcomes and probabilities
5) Learning agents
If the environment is unknown, changing, or too complex to hand-code, you add learning—often on top of any of the previous categories.
Defining idea: Improve performance over time by learning from experience.
A canonical decomposition (AIMA) includes:
- Performance element: selects actions (the “acting” part)
- Learning element: improves the performance element
- Critic: evaluates behavior relative to the performance measure (in RL terms: often tied to reward/returns)
- Problem generator: proposes exploratory actions (in RL terms: exploration)
Examples:
- A game-playing agent that improves by self-play and feedback
- An adaptive HVAC controller that learns occupancy/thermal dynamics to reduce energy while maintaining comfort
Common failure mode (learning agents): distribution shift / poor generalization (works in training conditions, fails in new ones) and unsafe exploration in real-world settings.
Strengths
- Adapts to unknown or changing environments
- Can reduce reliance on hand-crafted rules and models
Limitations
- Exploration can be costly or unsafe in the real world
- Learning can be data-hungry and sensitive to reward/feedback design
How the categories relate (and why they still matter)
These categories are often presented as a progression:
- Simple reflex (react to percept)
- Model-based reflex (react using inferred state)
- Goal-based (plan to reach goals)
- Utility-based (optimize tradeoffs under uncertainty)
- Learning (improve any of the above over time)
But this is not “more advanced is always better.” Simpler agents can be easier to verify, cheaper to run, and more reliable under tight constraints.
In modern AI systems, you’ll often see hybrids:
- A learned policy (learning agent) that behaves like a reflex agent at inference time.
- A planner (goal-based) guided by a learned value function (utility-based approximation).
- A model-based RL system that learns a world model (model-based) and uses it for planning (goal/utility-based).
- An LLM-era tool-using agent: an LLM produces reflex-like next actions (call a tool, ask a question), wraps them in a planner/executor loop (goal-based), and may use learned heuristics or scoring to pick among tool calls (utility-like).
Even if today’s implementations use neural networks, the taxonomy remains valuable because it clarifies:
- What information the agent uses (current percept vs. internal state/belief)
- Whether it reasons about the future (planning) or reacts
- Whether it optimizes a scalar objective (utility) or merely satisfies constraints (goals)
- Whether it can improve with experience (learning)
Choosing the right agent type for a task
A practical rule of thumb:
- If the environment is fully observable and the mapping is simple → simple reflex may suffice.
- If it’s partially observable → add internal state (model-based reflex / belief state).
- If you need long-horizon behavior → use goal-based planning.
- If you need tradeoffs and uncertainty handling → add utility (decision-theoretic selection).
- If the environment is unknown or changing → incorporate learning (with safety constraints where needed).
Summary
Classical AI theory (notably Russell & Norvig’s AIMA) categorizes agents by the structure of their decision-making: from immediate rule-based reactions, to belief-state tracking, to goal-directed planning, to decision-theoretic utility maximization, and finally to agents that learn and improve. The categories are simplified and often blended in modern systems, but they remain a crisp framework for designing, analyzing, and debugging intelligent behavior.
Next steps
If you want to go deeper, the usual conceptual path is:
- MDPs (fully observable decision-making) and POMDPs (partial observability / belief states)
- Classical planning (state-space search, STRIPS, heuristic planning)
- Reinforcement learning (policies, value functions, exploration)
- Modern agentic systems (planner–executor loops, tool use, model-based RL, and LLM agents)