Groundplane

Where it came from

I first built this shape inside a production AI platform at TikTok. Some summaries had to rank a table of numbers. A senior stakeholder pointed out that naming the wrong winner was unacceptable, even if the model usually got it right. That led me to move the ranking into code.

Code computed the winner, and the model described the result. Groundplane is that idea taken out of the platform, generalised past argmax, and published under MIT.

rows = warehouse.query("select campaign, ctr from campaign_daily")summary = llm(f"Which campaign performed best?\n{rows}")# says "north". It was "harbour".

from groundplane import FactRegistry, boundary, superlative reg = FactRegistry()reg.record_ranking(    "campaign_ctr",    {"north": 0.0412, "harbour": 0.0455, "delta": 0.0301},    key="ctr", tool="warehouse.query",    args={"table": "campaign_daily", "window": "7d"},) with boundary(reg, facts=["campaign_ctr"],              checks=[superlative(fact="campaign_ctr")]) as b:    b.submit(llm_structured(facts=b.facts()))    # {"winner": "north", ...}

UnsupportedClaim: unsupported claim in field 'winner':model said 'north', registered facts support 'harbour'| fact='campaign_ctr'| provenance=warehouse.query(table='campaign_daily', window='7d')| computed argmax on 'ctr' is 'harbour' (0.0455);  'north' ranked #2 at 0.0412

Scope

It validates declared fields against declared facts. If the model names the right winner and editorialises misleadingly around it, that passes. I kept the scope that narrow on purpose. I compute rankings in code and check the model's structured output against them. A judge model would make this check probabilistic. Groundplane's check needs to return the same result for the same facts.

Recording facts and checking fields

The check runs in code after the model submits its fields. Tool results go into a FactRegistry as typed facts, each carrying the tool call and arguments that produced it. Facts are write-once: a later call that changes the value is a new fact with a new name, preserving the earlier value and its provenance.

A boundary wraps a block of model output and names the facts it may use. The model emits structured fields, never prose. Submitting a plain string is a TypeError. Leaving the block without submitting also raises, because a caller needs to know when validation never happened.

Six checks, one question each

Each check asks how the recorded facts relate to each other, which is what a per-field validator cannot see. For example, two valid values can still be assigned to the wrong rows. A field-level type check would accept both.

superlative checks that the named winner is the computed argmax and any quoted score is the computed score. ranking_prefix checks a top-k list against the computed order, including a cut inside a block of tied scores. aggregate_reconciles recomputes a stated sum, mean, count, min, max or median over the recorded rows and refuses a truncated table. entities_recorded checks that every name the model used came from a recorded set. row_integrity resolves the named row first and reads every other field off that row, which catches a neighbour's value in the wrong column. comparison recomputes “A beat B by 12%” in code and, when the number is wrong, says which convention it does match: percentage points, a ratio, or the two entities reversed.

Numeric comparisons are exact by default. Each check accepts a tolerance, starting at zero, so the caller has to choose how much difference is acceptable.

Making errors useful

I wanted the error message to explain the mismatch on its own. It includes the field, the model's answer, the recorded result and the tool call that produced it. For a ranking, it also shows where the model's choice actually placed.

Inside a LangGraph graph the same failure can become a state update instead of a crash. The graph routes back to the model with the checker's message as the correction. A misconfigured check still propagates: a developer bug is not something to reask the model about. The MCP adapter records a tool result as a fact with the call as provenance. It prefers the structured payload over the text blocks, because text is a rendering and reading it is parsing prose again.

Testing the checks

The core has no dependencies and the adapters import neither framework they adapt, so a plain interpreter can read and test all of it. 169 tests run on five Python versions in CI, many of them adversarial cases where the plausible model answer is provably wrong.