AI Agent Fundamentals for Data Leaders

There’s a one-sentence definition that cuts through most of the AI noise: an agent is an LLM with tools, running in a loop.

I talk with data leaders about AI every day. Some are already building with agents. Many others nod along when words like “agent,” “context,” and “agentic loop” come up in everyday conversations: a team meeting, a planning session, a chat with a peer. Inside, they feel one step behind.

This post is here to close that gap: the fundamentals beyond the hype, explained in simple terms with real examples. If you’re already fluent in AI agent basics, skip ahead to post 2. If not, this is the foundation the rest of the series builds on.

The map:

• Post 1 (this one): The foundation. Generic AI agent fundamentals — what makes something an agent, regardless of domain.
• Post 2: Agents for data. The same anatomy, but specific to a company’s warehouse — the context and tools that actually ground an agent in data.
• Post 3: From agent to production. Access control, usage, evaluations, feedback loops — what it takes to take a data agent to production with confidence and trust.
• Post 4: Lynk. How Lynk puts this into practice.

Agents vs. Workflows

Before we get into what an agent is made of, let’s be precise about what an agent is. The test isn’t whether an LLM is involved — it’s who decides the next step.

For predefined scenarios, use workflow tools like n8n, Zapier, or Make: you define the steps ahead of time, and the path is fixed.

But many tasks have no fixed sequence. Take “why did revenue drop last week?” — the agent pulls a number, notices something odd, breaks it down by region, finds the drop is concentrated in one segment, and digs further. Nobody scripts that path in advance. The ability to reason, decide the next step, act on it, and keep going until the task is done — that’s what makes something an agent.

The Building Blocks

So what makes that autonomy work?

Imagine the best analyst in the world joins your team today. Brilliant analytical instincts, fluent in SQL, decades of experience under her belt. To do her job, she needs three things: a brain to reason, the ability to act on what she finds, and the discipline to keep going until the job is done. Those are the three parts of an agent: an LLM, tools, and an agentic loop. An agent is an LLM with tools, running in a loop.

But notice what’s still missing. She doesn’t know your business. She doesn’t know your data, or your best practices — which tables to use, when to use them, how you run a deep analysis. Everything she needs to know, someone has to put in front of her. That’s context: not a part of the agent, but the material it works with. The agent is the machinery; context is what you feed it. We’ll get to why that distinction matters so much — first, the machinery.

LLM

The analyst’s brain: the reasoning engine. A model trained to predict the next word, scaled up until those predictions become coherent answers, code, and analyses. At its core it does one simple thing: it takes a text input — its context — and produces an output as text. Each model has a fixed capacity for that input, called the context window, measured in tokens (roughly word fragments); different models ship with different sizes.

What matters for the rest of this post is what an LLM can’t do on its own. It doesn’t know your business, can’t take any action in the world, and forgets everything the moment the conversation ends.

Tools

The ability to act. Tools are how the agent’s thinking turns into action: query a database, search the web, run code, send a message, fetch a file — anything that lets the agent affect or observe the world beyond text generation. Without tools, it’s just a chatbot.

Say a user asks, “How many new customers did we sign last month?” The agent’s brain — the LLM — can reason that the answer lives in the database, and it can even write the SQL to get it. But that’s all it can do: generate text. It can’t reach into a database and run the query itself. That’s where tools come in. We give the agent a database-querying tool, and when the LLM decides a query is needed, it produces the SQL along with a request to call that tool. The tool executes the query, returns the results, and the LLM reasons over them on the next turn.

Acting on the world is only half of a tool’s job. The other half is bringing the world in: searching a filesystem, reading a document, listing what a database contains. A large share of an agent’s tool calls exist purely to retrieve context on demand — a role we’ll come back to in the context section.

Agentic Loop

The control flow that ties everything else together. A great analyst doesn’t just answer your question and walk away — she investigates: pulls some data, looks at the result, decides what to check next, and keeps going until the job is done. That’s the loop: reason, act, observe, repeat, until the agent judges the task complete or a stopping condition is met. Each pass changes what’s in the context window, so the next decision is more informed than the last. This is what makes it agentic rather than a single-turn call.

Context: Your Biggest Lever on Agents

If an agent is an LLM with tools running in a loop, where does the knowledge come from? Context: the information the LLM has in front of it at the moment it acts — everything loaded into the context window when a call is made. Same model, same question, different context, different answer.

All three parts of the machinery matter. But consider where the machinery comes from. The model is built by a handful of labs, and it improves whether you do anything or not. The loop and the tools are engineering: built once, then mostly stable. Context is the piece nobody can ship to you. What your metrics mean, which tables to trust, how your business defines churn — that knowledge exists only inside your company, and no model release will ever include it. It’s also the piece you can change today: improve the context, and the same machinery becomes a better agent. That’s what makes it your biggest lever.

And the lever is sensitive. LLMs reason over exactly what’s in front of them. Leave context out and the model is blind to the full picture. Give it stale or conflicting context and it’s confidently wrong, the failure mode you’ll catch last. Give it too much and it gets confused and expensive.

That last failure is the one worth reframing: the context window is a budget, not a bucket. Every token you load spends attention the model could have paid to something else. Managing context is an engineering job: deciding what gets spent, on what, when.

Types of context

Different types of context land in the window in different ways, for different reasons. Each gives you a different lever.

System prompt

The agent’s operating manual: who it is and how it should behave — the one piece of context written by whoever builds the agent, not drawn from your company’s knowledge. The system prompt is also where the tools registry and skills registry live: the list of capabilities the agent knows it has access to (skills are packaged playbooks — more on them below). It’s constant across the conversation, loaded into the window at the start of every call.

User input

The user’s request. The thing that kicks off the work.

Tool calls and conversation history

Every LLM call is blind to what came before — so without prior turns in the window, the agent has no idea what the conversation has been about or what its own tools have already returned. To give it continuity, conversation history and tool results are fed back into the window on every new call.

Dynamically loaded context

This is the big one — and the hardest to get right. It’s the second job of tools from earlier, seen from the context side.

Beyond the context that arrives by default, the agent can draw on far more information than any window could hold: a filesystem (common for coding agents), the web, vector databases, internal documents and wikis, Slack conversations, ticketing systems — anything you wire up. None of that lives in the context window by default. Instead, the agent is exposed to what exists, and decides, call by call across the loop, what to actually pull in. It’s the difference between handing the agent every book in the library and handing it a library card.

Three moving parts decide how well this works:

What’s available. The universe of context the agent can reach is determined by the tools we give it. A filesystem tool exposes a filesystem; a web search tool exposes the web; a connector exposes whatever system it wraps.

How that context is structured. The shape of the available context decides how efficiently it can be loaded. Take a filesystem of markdown files holding business knowledge. Put everything in one giant file and a standard read tool pulls in far more than the task called for — bloating the window with information the agent didn’t ask for. Go the other extreme and fragment every sentence into its own file, and the agent risks missing pieces it needed but didn’t know to look for. Granularity is a real design decision, with real tradeoffs.

How loading happens. Sometimes the agent knows exactly what’s available. Claude Code skills, for example, expose each skill’s name and description directly in the system prompt, so the agent can pick deliberately. Other times the agent doesn’t know what’s there at all — it just knows it can ask. Web search and filesystem search both work this way.

Memory

Long-term context that survives across conversations, so the agent doesn’t start from zero every time a new chat opens. The model itself still forgets everything the moment a conversation ends; what persists is notes the agent keeps outside itself, in files or a database, loaded back into the window the next time they’re relevant.

Skills

A skill is a playbook: a packaged set of instructions that captures how your team does a specific kind of job. It can look like a workflow written down, but the difference is the test from the top of this post — who decides. The agent chooses when to reach for a skill, and it keeps driving the loop while following it.

Mechanically, skills are just another type of dynamically loaded context. The agent sees the name and description of every skill it has access to in its system prompt — a catalog of what’s available. When it judges that a task calls for one, it loads the skill’s full instruction file — its SKILL.md — into the context window and follows it.

An Example Walkthrough

Concepts are easiest to grasp once you watch them work in sequence. So let’s walk through a single request, end to end, with every piece in play.

The setup:

• The LLM is the underlying model.
• The available context is everything the agent can draw on: all your schemas and tables described column by column, alongside a library of documents explaining the business and how to use the data correctly.
• The tools are a file search tool (to navigate that context), a SQL query tool, and a chart-drawing tool.
• The skills are two analytical playbooks — one for root cause analysis, one for churn analysis.

A user asks: “Analyze our churn rate month over month.”

What’s in the context window right now? The system prompt — with the tools and skills registries — plus the user’s question. That’s it. The agent knows it has a churn skill and a file search tool. It doesn’t yet know which tables exist, how churn is defined here, or what’s inside that skill.

Dynamic loading. The agent reaches for file search and pulls in the relevant pieces — the customer and subscription table definitions, the metrics and relationships that define churn, the business doc on how to use that data. It picks the churn skill from the catalog (not the root cause one) and loads its SKILL.md. The window now holds exactly what this task calls for, and nothing more.

The work. With the right context loaded, the agent figures out which tables to use, how churn is defined, and how to assemble the SQL — then calls the query tool. Results come back as conversation history, so on the next loop iteration the agent can see what it asked and what it found. It iterates — query, observe, reason, query again — following the churn skill.

The answer. When the analysis is done, the agent uses the chart-drawing tool to present the result: month-over-month churn, in the format your team expects.

One caveat before we move on. If you’re reading this as a data leader, the question on your mind probably isn’t “what happens when it works” — it’s “what happens when it doesn’t.” Real runs miss. The agent picks a near-miss table, joins on the wrong key, misreads the question, or returns a confidently wrong number. Context narrows those failures; it doesn’t eliminate them. That’s why the next two posts exist: post 2 on the structured semantic layer (predefined entities, metrics, joins) that keeps the agent from inventing the basics, and post 3 on the evaluations, access control, and production feedback loop that catch what still slips through.

Agentic Frameworks

Everything so far has been the anatomy of an agent. Frameworks like Claude Code and Cursor are where that anatomy gets assembled into something you can actually use. They pair a model with context management, a loop, and a built-in toolbox, so you don’t wire the plumbing yourself. And they’re built around the filesystem: a folder of files the agent reads, searches, and reasons over — a natural fit for coding, where the project is the filesystem.

For data, the anatomy is the same but the demands are different. What the agent works over isn’t a code project; it’s schemas, warehouse-scale data, and business tribal knowledge. You need tools that keep raw data out of the window (millions of rows don’t fit, and shouldn’t), and context management built for data. You also need access control over who can see what, visibility into how users actually interact with the agent in production, and data-specific evals.

Pulling It Together

An AI agent is that analyst from earlier in this post: something you can hand real work to. The machinery — model, tools, loop — you mostly get off the shelf; the data-specific parts you can’t are post 2’s subject. The lever in our hands is context: manage it well, and we get analytical capacity that scales far beyond what any human queue could support.

That promise only holds if we can trust the agent. For company data, trust isn’t a feature you ship — it’s a system you build. That’s what the rest of this series is about.