AI Systems
Most AI work stalls at the same place. The demo is impressive, the prompt is clever, and then someone asks "can we run this every week?" — and the whole thing falls apart.
The gap is not model quality. It is everything around the model: how instructions are packaged and reused, how the loop is structured, which tools the model can actually call, and how you find out when it is wrong.
This section covers that surrounding structure.
What You'll Find in This Section
| Guide | What It Covers |
|---|---|
| Building AI Skills | Packaging expertise into reusable instructions that load only when relevant |
| Designing an Agent Harness | The loop, context budget, guardrails, and failure modes of an agent that runs unattended |
| AI Tooling & MCP | Giving models real capabilities — tool design, MCP servers, and the practitioner stack |
| Evaluating AI Systems | Golden sets, LLM-as-judge, regression testing, and knowing when you actually improved |
The Four Layers
Almost every useful AI system decomposes the same way. Naming the layers makes it obvious which one is broken.
| Layer | Question it answers | Failure mode when missing |
|---|---|---|
| Instructions | What does "good" look like here? | Output drifts; every run needs hand-holding |
| Tools | What can it actually do? | Confident answers about data it never saw |
| Harness | How does the loop run and stop? | Runaway cost, infinite retries, no audit trail |
| Evaluation | Did it work? Did the change help? | You ship regressions and call them improvements |
A prompt is layer one only. That is why prompts do not survive contact with production.
Skill, Agent, or Workflow?
These get used interchangeably, and the confusion causes real architecture mistakes.
| Pattern | Decides what to do next? | Best for |
|---|---|---|
| Skill | No — it shapes how a task is done | Encoding a standard, a format, a house style, a checklist |
| Workflow | No — fixed sequence | Deterministic pipelines: exports, transforms, scheduled reports |
| Agent | Yes — chooses tools based on intermediate results | Open-ended investigation, triage, research with unknown depth |
Reach for the least powerful pattern that solves the problem. An agent where a workflow would do is slower, more expensive, and much harder to debug.
If the steps never change, do not use an agent. A workflow that calls a model at two fixed points is easier to reason about than a model deciding the sequence every run.
Principles That Hold Up
Ground it or expect it to guess. A model with no access to your data will produce something plausible about your data. Plausible and wrong is the worst possible output, because it survives review.
Make the intermediate steps visible. If you cannot see what the system retrieved before it concluded, you cannot debug it — you can only re-roll and hope.
Put the human gate where the cost is. Not everywhere, which trains people to rubber-stamp. Put it immediately before anything irreversible or externally visible.
Version the outputs, not just the code. When results change, the first question is always "did the prompt change, the model change, or the data change?" Only stored artifacts answer that.
Uncertainty is a feature. A system that says "the signal is weak here" is more useful than one that always sounds sure. Confidence you cannot calibrate is noise with good grammar.
Where to Start
New to this? Start with Building AI Skills — it is the lowest-cost, highest-return layer, and it works even if you never build an agent.
Already running agents that misbehave? Go straight to Designing an Agent Harness.
Cannot tell whether your changes are helping? Evaluating AI Systems.
How This Connects to the Rest of the Site
| Direction | Guide |
|---|---|
| Agents you build to run operations | Building Agents for SEO Tasks |
| Agents visiting your site | WebMCP Implementation |
| Live architecture example | OC MCP on omar-corral.com |
Building something in this space? Get in touch — I am always interested in how other people are structuring this.