Skip to content
SeedlingPlanted September 10, 2026Tended October 1, 202617 min read

Every Coding Agent Trend Is Program Synthesis

Prompt engineering, context engineering, loop engineering, harness design, evals. If you know the source material you can predict which one gets named next.

#program-synthesis#ai-agents#agentic-engineering#claude-code#context-engineering#research#open-source
On this page

Every coding agent trend is a shadow of a program synthesis primitive: named by people who can see the shape on the wall but not the object casting it. Watch the vocabulary arrive in order. “Prompt engineering” was circulating by July 2020 and pinned down in print by February 2021. “Evals” went from a repo OpenAI open-sourced the day GPT-4 shipped to a job title. “Context engineering” took over in June 2025, six days from Tobi Lütke’s tweet to Karpathy’s +1. This February, “harness engineering” was named and adopted inside a single week, and there are now separate eight-minute explainer videos for “loop engineering” and “agent harness”. And the topology itself is being named now. Agent work gets drawn as an explicit graph, nodes doing the work, edges carrying the results. LangGraph turned that framing into a framework in 2023-24, and Anthropic’s agent-patterns post cataloged the topologies: routing, parallelization, orchestrator-workers. Each term arrives as a new discipline, and each names a piece of machinery that program synthesis built decades ago.

I keep noticing this because I’ve spent 2026 running the classical loop at catalog scale with coding agents. The trends feel intuitive for a boring reason. They are one old loop, renamed a primitive at a time.

The loop, before it had product names

In 1969, Cordell Green’s QA3 system extracted working programs from resolution proofs. State what you want as a theorem, and the proof of it contains the program (Application of Theorem Proving to Problem Solving, IJCAI 1969). Manna and Waldinger built that into deductive synthesis, where the program falls out of a constructive proof of the specification (A Deductive Approach to Program Synthesis, TOPLAS 1980).

flowchart LR spec["specification<br/>(stated as a theorem)"] --> prover["theorem prover<br/>(QA3, resolution)"] --> proof["constructive proof"] --> prog["program<br/>(extracted from the proof)"] classDef stage fill:#cfe3f7,stroke:#0d4d8c,color:#111,stroke-width:1.5px; classDef out fill:#fff2a8,stroke:#a35d10,color:#111,stroke-width:1.5px; class spec,prover,proof stage; class prog out;

The version that matters most for agents showed up in 2006. Solar-Lezama and colleagues built a solver around “a counterexample-driven iteration over a synthesize-verify loop built from two communicating SAT solvers” (Combinatorial Sketching for Finite Programs, ASPLOS 2006). His 2008 thesis named the pattern: “a technique we have named counterexample guided inductive synthesis, or CEGIS” (Program Synthesis by Sketching, UC Berkeley 2008). Propose a candidate, verify it, and when verification fails, feed the counterexample back to sharpen the next proposal. If that sounds like every agent loop you’ve run this year, that’s the point.

flowchart LR spec["specification"] --> synth["inductive synthesizer<br/>(SAT solver 1)"] --> cand["candidate program"] --> ver["verifier<br/>(SAT solver 2)"] ver -->|passes| done["done"] ver -->|fails| cex["counterexample"] --> synth classDef stage fill:#cfe3f7,stroke:#0d4d8c,color:#111,stroke-width:1.5px; classDef check fill:#d5ecd7,stroke:#1e6a26,color:#111,stroke-width:1.5px; classDef out fill:#fff2a8,stroke:#a35d10,color:#111,stroke-width:1.5px; class spec,synth,cand stage; class ver,cex check; class done out;

Sumit Gulwani compressed the whole field into three dimensions in 2010: “expression of user intent, space of programs over which to search, and the search technique” (Dimensions in Program Synthesis, PPDP 2010). A year later he shipped the proof that this line of work ships products. FlashFill synthesizes string programs from input-output examples, and it’s in every copy of Excel (Automating String Processing in Spreadsheets Using Input-Output Examples, POPL 2011). And in 2013, SyGuS formalized the move everyone now performs daily: “allowing the user to supplement the logical specification with a syntactic template that constrains the space of allowed implementations” (Syntax-Guided Synthesis, FMCAD 2013). The 2017 survey by Gulwani, Polozov, and Singh is the single best on-ramp to all of it.

flowchart LR logic["logical spec"] --> intent["user intent<br/>(dimension 1)"] io["input-output examples<br/>(FlashFill)"] --> intent intent --> space["search space<br/>(dimension 2: the SyGuS grammar)"] --> tech["search technique<br/>(dimension 3)"] --> cand["candidate"] --> ver["verifier"] ver -->|fails| cex["counterexample"] --> tech ver -->|passes| prog["program"] classDef stage fill:#cfe3f7,stroke:#0d4d8c,color:#111,stroke-width:1.5px; classDef check fill:#d5ecd7,stroke:#1e6a26,color:#111,stroke-width:1.5px; classDef out fill:#fff2a8,stroke:#a35d10,color:#111,stroke-width:1.5px; class logic,io,intent,space,tech,cand stage; class ver,cex check; class prog out;

Intent, space, search, verify, feed the failure back.

The mapping

flowchart TD subgraph loop["autoresearch (2026): the whole loop, pointed at research"] intent["specification / user intent<br/>prompt engineering, 2020<br/>spec-driven development, 2025"] space["search space<br/>context engineering, 2025"] tech["search technique<br/>loop + harness engineering, 2025-26<br/>multi-agent orchestration, 2025<br/>agent graphs, 2023+"] cand["candidates<br/>code · proofs · experiments"] ver["verifier<br/>evals, 2023<br/>verifiable rewards, 2024<br/>decision models, 2026"] fb["counterexample feedback<br/>agent memory, 2023<br/>otherwise still unnamed"] end guard["oracle protection<br/>hooks + guardrails<br/>reward hacking, named 2016"] done["accepted result"] intent --> space --> tech --> cand --> ver ver -->|passes| done ver -->|fails| fb fb --> tech guard -.-> ver classDef stage fill:#cfe3f7,stroke:#0d4d8c,color:#111,stroke-width:1.5px; classDef check fill:#d5ecd7,stroke:#1e6a26,color:#111,stroke-width:1.5px; classDef out fill:#fff2a8,stroke:#a35d10,color:#111,stroke-width:1.5px; class intent,space,tech,cand,fb stage; class ver,guard check; class done out;
The trend termThe synthesis primitiveThe evidence
Prompt engineeringExpressing user intent (the specification)Reynolds & McDonell 2021: prompting “may be conceived as programming in natural language”
Context engineeringConstraining the search spaceKarpathy’s definition, “filling the context window with just the right information,” is SyGuS’s “syntactic template that constrains the space of allowed implementations,” twelve years on
Loop engineering / harness designThe search techniqueWillison: “An LLM agent runs tools in a loop to achieve a goal”
EvalsThe verifierOpenAI, Mar 2023: “a framework for evaluating OpenAI models and an open-source registry of benchmarks”
Hooks / guardrailsProtecting the verifier from the candidateDeepSeek-R1 skips a learned reward model because it “may suffer from reward hacking,” the problem named in 2016
AutoresearchThe full CEGIS loop aimed at experiments instead of functionsKarpathy’s own recipe: “the human iterates on the prompt (.md),” “the AI agent iterates on the training code (.py)“
Spec-driven developmentThe specification, named a second timeGitHub Spec Kit: “you start with a (you guessed it) spec”; Kiro ships the IDE around it
Verifiable rewardsThe verifier, named on the research trackTülu 3: “a novel method we call Reinforcement Learning with Verifiable Rewards”
Multi-agent orchestrationThe search technique, parallelizedAnthropic: “a lead agent coordinates the process while delegating to specialized subagents that operate in parallel”
Agent memoryThe state the loop carries between iterationsMemGPT: “virtual context management,” borrowed from OS memory hierarchies
Agent graphsThe search technique’s topology: candidate generation as a DAG, verifier gates on the edgesLangGraph plus Anthropic’s patterns: routing, parallelization, orchestrator-workers; named “Graph Engineering” in August 2026
Decision modelsThe verifier, as a model: yes or no and how sure, but not whyThe New Stack: they “return a set of predefined answers with confidence scores”; Jev lists “Verify everything” among its uses

Specification: prompt engineering, then spec-driven development

Both name the same step: say what you want precisely enough that a candidate can be checked against it.

Search space: context engineering

Context engineering is search-space design. The SyGuS insight was that a logical spec alone leaves the search hopeless; a grammar over allowed implementations makes it tractable. A CLAUDE.md is a grammar over behaviors. SutroYaro’s search_space.yaml is one literally: it enumerates what an agent may mutate, and the harness rejects everything outside it. When I wrote in coding-agents-are-the-base-agent that most agent errors are map problems, this is the theory underneath. A bad map is an unconstrained search space, and unconstrained search was the failure mode synthesis spent the 2000s escaping. By mid-2025 the failure even had its own name. Context rot, Chroma’s report on model performance growing unreliable as input length grows, is the search-space problem measured from the inside.

Search technique: loops, harnesses, multi-agent orchestration, agent graphs

The agent itself is the search technique. Chimera’s decomposition (provider, tools, loop, environment) is a menu of search strategies over the same space. Racing N different loops on one task, which is what Chimera’s multiplexer does, is a search-technique bake-off. In synthesis terms, none of it is exotic. Enumerative vs. stochastic vs. deductive search is a settled taxonomy in the 2017 survey, and which loop wins on which problem is an empirical question you can benchmark, not a matter of taste.

Verifier: evals, verifiable rewards and decision models

Evals are the verifier, and the verifier is the part that decides whether any of it was real. Synthesis learned early that the verifier defines the product. Your program is only as correct as the check it passed. My most expensive 2026 lesson was the same lesson. Bourbaki’s v0.2.1 claimed 94.3% on miniF2F because the Lean REPL reported success on tactics that failed standalone compilation; the audited number was 6.2%, and the release is retracted with the inflation preserved for the record. CEGIS treats verification failure as signal to feed back. Agent practice mostly treats it as a grade. The feedback edge is the part of the loop the industry hasn’t renamed yet, which is why I run audits per wave instead of evals at the end.

Decision models are the checking step, sold as a model. In the 2006 loop the checker was a solver. It said whether the candidate was right, and when it wasn’t, it handed back an input that broke it. Logic has a name for a checker that always finishes with a correct yes or no, a decision procedure, so the name is older than the models. This September a decision category of models showed up that does only the yes-or-no part. TypeSafe released Jev on September 15 and calls it a System One model, and The New Stack calls the category decision models. They don’t write text. You give them the state and a question with the answers you allow, and they pick one and say how sure they are. Sebastian Raschka called Jev “the ChatGPT moment for classification.” TypeSafe lists “Verify everything” among the uses, and its FAQ says it needed a new way to train, because rewarding a model with a check written in code only works when a simple check exists, and most real-world judgment calls don’t have one. Two weeks later OpenAI announced its own Decisions API, pitched at sorting content, routing requests and picking an agent’s next step. Neither hands back the input that broke anything. You get an answer and a confidence number.

The first studies already put numbers on it. One looked at the 2,170 public Jev projects on GitHub a week after launch, labeled by LLM agents, and found 77% use it to judge something about the input, 52% to score or rank, and 31% to pick the next action. Another tested Jev as a judge. It comes within three points of GPT-6 when the answer can be read straight off the text, at 0.36% of GPT-6’s price, and falls behind when the answer has to be worked out, as in math, code and logic. A third found that Jev goes by an answer’s name more than by what the answer is defined to mean. Call the two options yes and no, swap the definitions behind them, and it flips 24 times as many answers as asking the same thing twice does, while every answer still comes back in the right format.

Protecting the verifier: hooks and guardrails

Hooks and guardrails keep the candidate away from the check. An agent with write access can edit the test instead of the code, and a hook that blocks writes to the tests is oracle protection at the size of one repo.

State between iterations: agent memory

Agent memory is the state the loop carries between iterations. In CEGIS that state is the pile of counterexamples, and each failed check narrows the next proposal. MemGPT arrived in 2023, borrowing virtual memory from operating systems.

The whole loop: autoresearch

Autoresearch is all of it at once, pointed at experiments instead of functions. In Karpathy’s recipe the human edits the prompt file and the agent edits the training code, so the spec and the candidate sit in separate files, and the training run is the check.

Why the names keep coming

The charitable read is the correct one. Builders hit each wall in order and name it on impact. The spec came first (prompt engineering, 2020-21). Checking came next (evals, March 2023). The material around the spec followed (context engineering, June 2025), then the iteration structure itself (agentic loops and harnesses, late 2025 into 2026), and last the assembled loop pointed at research (autoresearch, March 2026). Read against Gulwani’s dimensions, the arrival order walks the primitives: user intent, verifier, search space, search technique, and finally the whole CEGIS loop.

The renaming has also started its second lap. Specification got named twice, first as prompt engineering in 2020, then as spec-driven development in 2025, when Kiro shipped an IDE around it and GitHub shipped a toolkit. The verifier got named on two tracks at once. Evals covered the product side, with eval-driven development as the discipline form. On the research side, Tülu 3 coined Reinforcement Learning with Verifiable Rewards in November 2024, and DeepSeek-R1 made rule-based verifiable rewards the headline recipe two months later. DeepSeek’s stated reason for skipping a learned reward model, that it “may suffer from reward hacking,” points at the one primitive named before any of this. Reward hacking has been on the books since 2016. Memory ran the pattern in reverse. MemGPT arrived in 2023, no “memory engineering” coinage ever landed, and the practice got absorbed into context engineering instead. The search technique is the station renamed fastest, from the graph framing (LangGraph, 2023) to multi-agent orchestration (June 2025) to agentic loops (September 2025) to harness engineering (this February) to graph engineering (this August). One station, five names in three years, and the newest drawings put verifier nodes directly on the graph’s edges. Cohorts rename the stations as they reach them; the stations don’t move.

You can catch the naming happen live. Mitchell Hashimoto, this February: “I don’t know if there is a broad industry-accepted term for this yet, but I’ve grown to calling this ‘harness engineering.’” Six days later, OpenAI published a post titled “Harness engineering.” That’s the whole mechanism inside one week. Names help adoption, and adoption is good.

It cuts the other way too. The trend terms are so young they lack provenance. “Prompt engineering” gets retro-attributed to Gwern, who actually wrote “prompt programming”, and its earliest well-documented definitional use in print is Reynolds & McDonell, February 2021. The synthesis primitives have DOIs.

The practical difference for anyone who reads the old material: you stop waiting for the name. The primitives that haven’t trended yet are sitting in the same papers, visible now.

Three predictions, on record

If this lens is right, it should predict. Dated 2026-09-12, checkable later:

  1. Software factories are next. They are the epitome of spec to implementation.
  2. The critics of software factories say they lack verification. So the next set: agents that are good at verification.
  3. Eventually we move to new categories of super apps rather than super agents.

If any of these arrives under some fresh name, this table gets a new row and the thesis gets a data point. If none arrives, the lens is weaker than I think and that goes in the log too. That’s the deal I’ve signed up for everywhere else. Claims you can check beat claims you can admire.

Update, 2026-09-30. The next set showed up three days after I dated these, as a decision category of models rather than agents. Within 24 hours of Jev landing on Vercel’s AI Gateway, nearly 13% of its paid teams were using it. But OpenAI pitched its version at sorting and routing rather than checking, and the first study of Jev as a judge puts it behind on code, which is what software factories need checked. So prediction 2 is half there, and the table gets a new row.

The seat I’m watching from

The catalogs ran as a synthesis pipeline with the roles made explicit: spec as a GitHub issue, one candidate generator per stub (53 for Hinton, 58 for Schmidhuber), a read-only verifier per wave, human acceptance at the gate. SutroYaro points the loop at experiments. Novalis points it at a terminal. The vocabulary around all of this will keep changing. The loop hasn’t changed since Green extracted a program from a proof in 1969, and I expect it to outlast the next six names too.

Subscribe to the garden

New notes as they sprout — no spam, unsubscribe anytime.

Quick Actions
Browse all notes

Knowledge Graph

20 notes · 85 connections