Extending Gadfly
The core is deliberately small. These extensions widen who can use the method, each answering one need and keeping its contracts.
| Capability | Use it | What it keeps |
|---|---|---|
| Command line and report | gadfly plan, run, report | records as JSON Lines, an explicit budget, evidence apart from obligations |
| Frame schema and authoring check | midwife/frame.schema.json, gadfly check-frame | the parser as the authority |
| Portfolio summary | gadfly portfolio | no verdict across frames |
placement intervention | "placement": {"span": id, "after": id} | old frames' hashes and categories |
local-json battery | "battery": {"kind": "local-json", "url": ...} | loopback, no credentials, a whole-call deadline |
| Settled trials | gadfly run --settle | every level, category and evidence line of a whole run |
| Calls at once | gadfly run --parallel N | the budget, and every call already made |
| Dead ends before a call | gadfly plan | the verdict rules, read over every answer each trial could give |
| Shams of another kind | "sham": {"draws": n, "kind": {anchor: kind}} | old frames' hashes and plans |
| Stopping at unresolved | gadfly run --stop-unresolved | the category a whole run reads |
| Records through a pipe | gadfly run --out -, gadfly report FRAME - | each record written as it exists, never over a file |
| Errors and warnings as classes | midwife.errors, --json on every command | one exit status a class, and every message tabled |
Why each exists
Each extension answers a need the core left to the reader. Its page has the details.
- Command line: the essay requires kept evidence and visible obligations. Without it, a reader writes Python to keep records, then joins evidence and obligations in a print loop of their own.
- Frame schema and authoring check: declaration errors should show in an editor before a run. The parser stays the authority, because rules across fields are not all expressible in JSON Schema.
- Portfolio summary:
overall()reads one frame over its inputs. No fixed rule combines different relations, promises and recipients, so a portfolio verdict would be a category error, andgadfly portfolioonly counts. - Placement: presentation invariance does not test whether a clause's place matters.
- Local JSON battery: a local evaluator gets the shipped batteries' privacy and deadline treatment without an adapter of its own.
- Settled trials: a trial whose level was already fixed still spent its remaining calls; with four of five to agree, four alike decide it.
- Calls at once: a battery that can serve several calls, such as a router, waited on one at a time.
- Dead ends before a call: a frame that no answers could make carried, such as one with no bypass under the default policy, spent every call before its verdict said so.
- Shams of another kind: an anchor alone of its kind got no sham, and the only way round was to give it a kind it does not have.
- Stopping at unresolved: an input that could read nothing else still spent every trial, such as one whose ablation and control both held.
- Records through a pipe: a run could only write a file, so a script that wanted the verdict kept one it did not need.
-stands for standard output and input, as clig.dev asks. - Errors and warnings as classes: a script or another model could tell a bad frame from a bad battery only by reading the words, and openings and early stops were lines with no class. Each is now a Python error or warning, with its own exit status.
What comes next
The examples point at these, most useful first. None is built yet.
- Expectations per input: an optional expected result for each input, so a frame whose inputs differ on purpose reads as promised. Today several real frames read unresolved when each input behaved as promised (design calls A and B in
examples/FINDINGS.md). - Regression tests for AI deployments: keep a set of frames and run them again when the model, prompt or policy changes, then compare verdicts. Does the new model still enforce the rule because of the rule?
gadfly portfoliois a start. - Comparing readers: run one frame on Gemma, Claude or a panel of human reviewers, which makes the essay's handoff measurable.
- Changes beyond cutting: change a value (400 ms becomes 100 ms) or add contrary evidence, which tests direction, not only dependence.
- Better statistics: report counts with ranges, and keep the temperature with the answers.
- Help writing frames: beyond the dead ends
plannames, a warning when a control leaves a line that still points to the answer, as in A frame built wrong. It flags problems and never writes a frame from a result.
Deliberately left out
These stay out on purpose, each because it would undo a boundary the method keeps.
- A single score across categories: the method returns distinctions, not a quality metric, and collapsing them would hide what the essay asks to keep.
- Automatic consent to training-tier providers: consent stays an explicit, exact model slug in the frame.
- A hosted record store: keeping no state is a boundary, and how long records are kept is a separate decision.