I expected my transformer architecture talk for software engineers to get stuck on attention heads, matrix multiplication, and mixture of experts.

Instead, one question kept pulling the discussion back to engineering:

How should understanding this architecture change the way we build with LLMs?

Transformers make more sense when you stop treating them like minds and start treating them like resource managers.

They manage a finite context window. They decide which earlier tokens matter to the current token. They move working state through layers. Modern models route computation through selected experts. During generation, they cache work that would otherwise be repeated.

Thinking about LLMs this way changed how I debug prompts, design agents, and decide what belongs inside the model.

The loop is simpler than the experience

At the highest level, autoregressive generation is almost boring:

  1. Read the tokens available in the current context.
  2. Predict a distribution for the next token.
  3. Select a token.
  4. Append it to the sequence.
  5. Repeat.

That simple loop hides an absurd amount of computation. The model does not pause, reflect, and compose a finished answer. It predicts one token, adds it to the sequence, and does it again.

The state we give it shapes every pass.

Stateless does not mean state-free

People often say LLMs are stateless. That is true across independent requests: the model does not automatically carry a durable memory of what happened before.

During one inference call, though, state is everywhere.

The current context is working state. Each token’s representation evolves as it moves through the model. The attention mechanism reads prior positions. The residual stream carries updated information from layer to layer. The key-value cache preserves attention state for earlier tokens so generation does not need to rebuild all of it for every new token.

None of that is durable memory.

A chat transcript is not a knowledge base. A context window is not experience. A KV cache is not a record of what worked, what failed, or which constraint mattered.

An agent can reason over the state we provide. The surrounding system decides what to retain, retrieve, summarize, or discard.

Attention is a read operation

Picture people standing in a line.

Imagine that each person can see everyone ahead of them, but nobody behind them. The first person has almost no context. Each person later in the line can use more of what came before.

That is the causal structure of next-token generation. A token can attend to earlier tokens, not future ones.

Attention gives different weights to earlier tokens based on the current prediction. The residual stream carries that changing representation forward. MLP layers transform it. In mixture-of-experts models, selected expert networks handle each token.

The transformer has to allocate a few scarce resources:

  • The context window limits what information is available.
  • Attention weights the earlier tokens for the current prediction.
  • Dense or expert networks spend compute transforming the token’s state.

That is a much more useful engineering model than “the AI knows the answer.”

What this changes for software engineers

Once I started thinking in resource-allocation terms, I changed how I work.

1. Debug the context pipeline

When an agent gives a bad answer, “the model got confused” is not a diagnosis.

Inspect what the system actually assembled:

  • Which instructions were active?
  • Which documents were retrieved?
  • What order were they placed in?
  • Which constraint was missing?
  • Which stale details competed for attention?

Often the failure happened before the model generated its first token.

2. Treat tokens as a budget

More context is not automatically better context.

Every old log, duplicate instruction, and irrelevant document consumes bandwidth. A smaller prompt containing the right state can outperform a larger prompt filled with everything the system could find.

3. Separate working state from durable memory

The context window should contain what the model needs for the current decision. Durable goals, decisions, constraints, outcomes, and learned procedures belong in an external memory system that can retrieve them when relevant.

Otherwise, “memory” becomes an ever-growing transcript.

4. Use tools for precision

Models are good at interpretation and synthesis. Databases, compilers, calculators, and APIs are better at exact retrieval and deterministic operations.

I do not ask the model to impersonate every tool. I let it decide when to use one.

5. Design for reuse

Stable prompt prefixes, selective retrieval, and cached inference state all reduce repeated work. The exact mechanism varies by model and platform, but the systems principle is the same:

Do not recompute what you can safely reuse.

The model is only half the system

Understanding transformers did not make me less impressed by them. It made me more precise about what they do.

The model maps its current context to the next token. Prompt assembly, retrieval, tool choice, and memory all happen outside it.

A bad decision there can sink the answer before generation starts.

That raised another question for me. If an agent has thousands of prior facts, decisions, tools, constraints, and outcomes, how should it decide which experience is reusable?

I’m starting to dig into ontology to answer it.

Now, when an LLM-based system fails, I first inspect what the system asked it to read, retain, and reuse. The model may still be wrong. At least I am debugging the system I actually built.