The skill caused the failure
A failed run may originate in routing, a tool adapter, missing evidence, the evaluator, or stochastic execution.
Adding instructions to compensate for a parser failure increases context without repairing execution.
Should this skill change?
First diagnose.
Then evolve.
A requirement-aware, regression-controlled framework for changing persistent LLM-agent skills only when the evidence supports the right intervention.
Existing skill-evolution systems are good at proposing edits, but often begin from an untested premise: that the user named the right problem and that a skill rewrite is the right remedy. GSE adds an epistemic gate before evolution and an empirical gate after it. The system may clarify, repair another component, retain the incumbent, or promote a candidate only after paired evidence.
DEFAULT TODAYRequest → rewrite
MISSING QUESTIONWho owns the failure?
VALID OUTCOMENo change
CORE UNITNet improvement
01 · Failure of the premise
An optimizer can efficiently improve the wrong thing.
Before changing persistent behavior, the system must test the request, the diagnosis, and the target objective—not merely the proposed wording.
The request states the real objective
“More detailed” may be a proxy for evidence linkage, decision usefulness, completeness, or simply confidence.
A longer report can score higher on surface detail and still become worse for the decision-maker.
One failure justifies persistence
A triggering example is selected evidence. It does not show that a global rule will generalize.
An exceptional case becomes a universal instruction, degrading the common path.
Evolution is selective intervention
The system chooses among clarify, revise the skill, repair another component, and retain the incumbent.
A no-change decision is a successful outcome when the evidence does not justify mutation.
02 · Two-sided guardrail
Guard the premise. Guard the promotion.
The first gate makes the intended change falsifiable. The second asks whether the candidate produces a meaningful held-out gain without breaking preserved behavior.
01 · INTAKE
Analyze the request
Extract objective, problem, evidence, constraints, invariants, and exclusions.
R = (O, P, E, C, I, N)02 · EPISTEMIC GATE
Clarify adaptively
Ask only when the answer can alter the intervention or acceptance criteria.
INFORMATION VALUE03 · ATTRIBUTION
Find the failure owner
Compare skill, routing, tool, data, model, evaluator, and execution causes.
WHERE TO INTERVENE04 · CONTRACT
Freeze the objective
Define target behavior, smallest meaningful gain, invariants, and scope.
TESTABLE HYPOTHESIS05 · MINIMAL EDIT
Generate a candidate
Every change must map to a diagnosed defect or contract requirement.
s′ = s + Δ06 · EMPIRICAL GATE
Compare versions
Run matched source, held-out, regression, and challenge cases.
ACCEPT · REJECT · RETESTpromote(s′) iff LCB1−α(U(s′) − U(s)) > δ
and CriticalRegressionRate(s′, s) ≤ ρ
and every hard invariant remains true
and CriticalRegressionRate(s′, s) ≤ ρ
and every hard invariant remains true
03 · Decision-frontier elicitation
Walk the design tree without asking premature questions.
Inspired by Matt Pocock's grilling pattern, GSE models requirement discovery as a dependency tree. Each round asks the current frontier: decisions whose prerequisites are settled. Facts are investigated by the agent; value judgments remain with the user.
INTERACTIVE DESIGN TREE
SOURCE-INSPIRED · GSE ADAPTATIONHow an ambiguous request becomes a change contract
Advance through the frontiers. Later questions appear only after upstream choices make them meaningful.
DECISION ROUNDS
Good integration
- The design tree makes hidden dependency structure explicit.
- Frontier rounds reduce both serial question fatigue and premature batch questioning.
- Facts-versus-decisions prevents asking users what execution traces can establish.
- The confirmation gate protects against silently converting an interview into authorization.
Do not copy it literally
- Stop when another answer cannot change the intervention or evaluation contract.
- Allow one-at-a-time questions when cognitive load or language preference makes rounds worse.
- Reopen branches when new trace evidence invalidates an upstream assumption.
- Represent the frontier explicitly enough to evaluate question ordering and coverage.
Fₜ = { d ∈ unresolved decisions : prerequisites(d) ⊆ resolvedₜ }
ask(Fₜ) while ExpectedDecisionValue(Fₜ) > interaction cost
ask(Fₜ) while ExpectedDecisionValue(Fₜ) > interaction cost
Mechanism adapted from Matt Pocock's grilling design tree. The cost-aware stopping rule, failure-owner branches, evidence updates, and promotion contract are GSE extensions.
04 · Interactive diagnosis
The same user request can imply a different repair.
Select a hidden root cause. The visible complaint remains similar, but the correct intervention changes.
05 · Change contract
Turn an opinion into a falsifiable change.
The contract is the stable interface between what the user means, what the editor changes, and what the evaluator measures.
WHY THE CONTRACT MATTERS
“More analytical” is not a test.
Without an explicit target, an editor can optimize surface proxies such as length, section count, or confident tone.
- Separate the observed symptom from the suspected cause.
- Name hard invariants before generating a candidate.
- Constrain the edit to the smallest justified scope.
- Define a meaningful gain, not merely any positive score.
problem:
observed: evidence exists; decision synthesis is weak
suspected_cause: final synthesis procedure
objective:
target: concise conclusions linked to evidence
success: higher usefulness; accuracy preserved
invariants:
- citations remain valid
- missing evidence is never invented
- export format is unchanged
change_hypothesis:
evidence → reason → action improves decisions
minimal_scope: final synthesis only
06 · Research position
The novelty is not version comparison alone.
Recent systems already use traces, held-out gates, and multi-objective search. GSE adds an earlier decision boundary: whether skill mutation is the correct intervention and what objective should govern it.
DOMINANT EVOLUTION QUESTIONHow should the skill change?
GSE QUESTIONShould it change—and why?
Failure premise
Skill is usually assumed causal
Root cause is explicitly compared
User objective
Request or benchmark reward
Elicited, contracted, testable
Action space
Generate/revise/select
Clarify · revise · repair other · retain
Successful outcome
A higher-scoring revision
The best justified intervention
Safety signal
Held-out candidate performance
Alignment + held-out gain + regression + cost
Negative evidence
Reject a weak candidate
Learn that evolution itself was unnecessary
07 · Experiment intuition
See where each guardrail earns its cost.
Change the hidden request type and switch the two guards on or off. The projection illustrates the hypotheses the experiment will test; it is not a reported result.
INTERACTIVE ABLATION
ILLUSTRATIVE · NOT EMPIRICALGuardrail contribution simulator
A conceptual model of expected behavior across request categories.
The hidden oracle state, not visible to the method.
Adaptive elicitation, root-cause analysis, and change contract.
Incumbent comparison, held-out cases, and regression threshold.
C4 · FULL GSE
CLARIFY → REVISEAmbiguous true defect
INTERPRETATION
Elicitation recovers the actual analytical objective; paired evaluation checks that the resulting revision generalizes without damaging citations or brevity.
Projected values visualize directional hypotheses only. The preregistered experiment must estimate real effects.
08 · Evaluation design
A factorial study isolates what actually works.
The design separates requirement understanding from version control instead of comparing one large framework against a weak baseline.
DATASET
300 evolution episodes
Thirty base skills across coding, documents, research, and operational tool use; balanced across twelve request types.
EVIDENCE
Four evaluation slices
Trigger repair, target generalization, preserved behavior, and challenge cases with matched seeds and environments.
OUTCOME
Intervention quality
Reward the correct decision—even when that means clarification, a non-skill repair, or retaining the incumbent.
| CONDITION | REQUIREMENT GUARD | VERSION GUARD | ROLE |
|---|---|---|---|
| No change | — | Incumbent only | Lower-bound reference |
| Direct evolution | No | No | Conventional rewrite baseline |
| Elicitation only | Yes | No | Isolate requirement contribution |
| Evaluation only | No | Yes | Isolate version-gate contribution |
| Full GSE | Yes | Yes | Test joint and interaction effects |
| Oracle specification | Complete oracle | Yes | Upper-bound requirement condition |
PRIMARY 01
Evolution Decision Accuracy
Macro-F1 over clarify, revise skill, repair another component, and retain.
EDA = MacroF1(action, oracle_action)PRIMARY 02
Net Evolution Utility
Held-out gain adjusted for regression severity, compute, and interaction burden.
NEU = ΔQ − λrR − λcC − λiIPRIMARY 03
Harmful Evolution Rate
Accepted revisions that perform meaningfully worse than the incumbent on locked tests.
HER = harmful_promotions / promotionsPRIMARY 04
Unnecessary Evolution Rate
Skill edits made when the oracle action is retention or repair outside the skill layer.
UER = wrong_skill_edits / oracle_no_editSkill evolution is a selective intervention problem—not an automatic rewrite operation.
A reliable system must infer the true objective, identify the correct failure owner, and require comparative evidence before changing persistent agent behavior.