datumwise

Learn · Manual

The Frame-QL Manual

The formal query language of Columna, by datumwise

Frame-QL First Edition — a renamed, lightly-revised continuation of the Coframe-QL Manual, Third Edition (which it supersedes). The Third Edition’s content is preserved; this edition renames it to Columna/Frame-QL/Manifold and reconciles it to the column-foundation redesign (ADR-031), the Core/Pro split, and the shipped envelope grammar (ADR-035). It inherits the v4 operator model (ADR-009…015), the V/M/B anchor vocabulary (ADR-024), the anchor-ascription rule, family-aware scans, macro bindings, EXPLAIN, and the (anchor, universe) grain (ADR-025…028). The envelope — SELECT <series> [AS <alias>], … AT {anchor} with its optional FROM, WHERE, HAVING, ORDER BY, LIMIT … PER, and WITH clauses, where @ {…} is the input-anchor marker universally and AT {…} is the sole output-grain declaration — is the language, and this edition documents it as implemented (columna-core 0.9.0). The pre-launch shipped surface was a fragment of it — the terse cols @ anchor form, where a trailing @ spelled the output anchor; that fragment is retired (its two @s meant opposite things) and preserved for lineage in Appendix D. The fourth outcome, named inform under ADR-020’s inform-and-serve doctrine, ships on the wire as the refuse mood; this edition uses refuse throughout (Chapter 7). The §2c universe law (structural universe resolution; one expression, one universe; cross_universe), the shipped B3 basis/absence rule, and the shipped reason codes are reconciled into the semantics they govern.

About the examples (stated once): every FrameQL example in this manual is checked against the shipped parser (columna_core.envelope.parse_statement, columna-core 0.9.0) by docs/tools/check_manual_frameql.py, which rides the manuals merge as a standing test — the manual can never document syntax its own parser rejects. A code block fenced frameql-illformed is deliberately ill-formed: it shows a syntax the parser rejects (a [SCHEDULED]/[ROADMAP] surface not yet shipped, or a teaching non-example), and the check asserts it does not parse. Every other FrameQL block parses clean (it may still refuse or clarify at plan time — that is semantics, which the four moods report, not grammar).

Preface

The Frame is the query, not operational instruction. Frame-QL takes its name from what remains when you take the Data out of a DataFrame: the Frame — a collection of column expressions on one anchor, and still everything. In Columna a column is an expression of what a thing is, not a step in a process, and a Frame-QL statement is a declared assembly of such columns at one stated grain — never a sequence of operations to perform. The writer says what the answer is — which series, at which anchor, under which conditions — and the framework decides how to realize it. This is the origin law of the language, and everything below is its consequence: the statement names a frame; the planner assembles the frame and owns the envelope’s clauses; the engine computes the series, envelope-blind. Nothing in the surface syntax describes an operation to run.

Frame-QL is the formal query language of Columna. It is the language compilers compile, agents emit, integrity certificates are stated against, and external systems integrate with. Every query against a Manifold — whether typed by a user, generated by the Columna agent, or produced by sugar-expansion from a higher-level surface — ultimately becomes a Frame-QL expression that the framework parses, typechecks, plans, and executes.

This manual is the reference for that language. It is intended for analytics engineers writing queries directly, developers building tools that integrate with Columna, and anyone who wants to understand precisely what the framework will and will not accept.

A note on scope. This manual specifies the language: its grammar, its semantics, its type rules, and the outcomes it produces — every query returns a (result, annotation) pair, and this manual specifies when the result is served, when it is served with disclosures, and in which narrow cases no result can be produced. It is not a tutorial on Columna concepts — for that, see the Columna Framework manual and the companion Columna Reference Manual. This manual presumes familiarity with the framework’s concepts: columns as functions from anchors to values; the three anchors a column carries — the V-anchor (its domain), the M-anchor (its missingness mechanism), and, per family, the B-anchor (the axes its reducer is blocked from rolling up across); the family set, default family, and the fertile/mule distinction; degeneracy and coverage; defining boundaries; reducers and the three anchor-actions crossed with the order axis; and the four-state integrity certificate. Terms are used here as they are defined there.

The grammar is given formally in the companion Frame-QL BNF Grammar; this manual covers semantics and usage. Where this manual and the BNF differ, the BNF is authoritative on syntax and this manual on meaning.

Revision note (Frame-QL First Edition)

This edition is the vetted Third-Edition text, renamed and reconciled. The reconciliations are small and are flagged in place with a ▸ Frame-QL revision note. The ledger:

  1. Many-to-many aggregate-across (fan-out) is a clarification, not a served disclosure (Chapters 5.6, 7.3, 7.4). Under the column-foundation (ADR-031) the engine transports along functional edges and never joins; a non-functional (M:N) edge is therefore not a traversable path, and the membership aggregation the Third Edition served is not expressible without a declared resolution. The unmet precondition is reported, statically and before execution, as a clarification naming the same three remedies. This replaces “serve membership with overlap disclosure” for the undeclared case.
  2. The inform-and-serve boundary is the planner/engine boundary (Chapters 5, 7.2). Everything statically catchable — ambiguity, unknown operator, unknown column, type mismatch, non-traversable edge, governance/access — is decided by the planner, before resolution, and may yield a clarify/refuse with no result. Analytical risk that survives to the column engine is served-and-disclosed, never withheld. A B-anchor crossing is detected at the planner (it attaches the critical disclosure) and may be downgraded to a route-around at the engine (Chapter 5.3); the number is always served.
  3. Mule recompute is recompute-from-base at the output grain where no fertile decomposition exists (Chapter 7.4, Appendix A) — stated explicitly, consistent with the Third Edition.
  4. Universe depth. The (anchor, universe) grain and the basis-driven empty-bucket rule are unchanged at the language surface (Chapter 1.5); beneath the surface, Columna enriches universe with a population-consistency layer (support reconciliation across a universe’s columns; path-independence of a reduced total across anchorings). This is semantics under the syntax, not new syntax.
  5. The envelope is the language; the terse @-fragment is retired (Preface; Chapters 1–4; Appendix D). The pre-launch surface spelled the output anchor with a trailing @ (aov @ cal.month); the ratified envelope (ADR-035) makes @ {…} the input-anchor marker universally and AT {…} the sole output-grain declaration, and adds the FROM/WHERE/HAVING/ORDER BY/LIMIT … PER/WITH clauses. This edition documents the envelope as implemented; the terse form moves to Appendix D (lineage). FROM is optional — it defaults to the bound Manifold and is required only where a surface holds more than one (Chapter 1.3).
  6. The fourth mood is refuse, not inform (Chapter 7). The shipped wire’s four moods are serve · disclose · clarify · refuse. The withholding outcome this manual’s Third-Edition lineage called inform (ADR-020’s “inform-and-serve” doctrine) is the wire’s refuse; the doctrine’s spirit — withhold only structurally, disclose analytical risk — is unchanged, and this edition uses refuse throughout. (The separate cross_universe category error rides the query-error channel, not a mood — Chapters 2, 7.)

Core and Pro

Columna ships in two tiers, and this one manual covers both. Constructs available only in Frame-QL Pro are tagged inline [Pro]; everything untagged is in Frame-QL Core. Core is a single-backend, transport-based engine with the full query surface, the reducer/scan/map operator umbrella, V/M/B anchors, universes, families, B-anchor disclosure, and EXPLAIN. Pro adds the missingness behavior map and MISSING_POLICY, coverage states, WITH allocation, the sketch reducers and sketch types beyond the Core set, multi-backend/federation, and custom operators/types. The tagging is a serialization view over one source of truth, exactly as disclosure_default is (Chapter 7.1): it narrows what a tier exposes, never what the language means.

Chapter 1. Basic Structure

1.1 What a query is

A Frame-QL query produces a single output frame. A frame is an assembly of co-anchored value columns at a single anchor; the anchor’s dimensions appear automatically as the leading columns of the frame, and the value columns are exactly the series the query computes. This is the framework’s frame model from the conceptual manual, expressed at the language level: the query says what Manifold to address, what series to compute, at what anchor the result lives, with what filters and ordering. The result is a frame.

1.2 The skeleton

The complete skeleton of a Frame-QL query — the envelope (ADR-035), given here in its fixed clause order:

[EXPLAIN]
[FROM   <manifold>]
[WITH   <name> = <expression> [, ...]]
SELECT  <series_1> [AS <alias_1>], <series_2> [AS <alias_2>], ...
AT { <output_anchor> }
[WHERE  <per-series predicate> [AND ...]]
[HAVING <output-frame predicate> [AND ...]]
[ORDER BY <output-frame column> [ASC|DESC] [, ...]]
[LIMIT  n [PER { <anchor coordinates> }]]

SELECT and AT are required; every other clause is optional. FROM is optional and defaults to the bound Manifold — the surface the session or tool is attached to — and is required only when a surface holds more than one Manifold and the statement must name which (Chapter 1.3). ON is not a query clause — a population is pinned in definitions, and universe is resolved structurally at query time (Chapter 1.5, the §2c law); an ON in a query is a syntax error. Clause order is fixed: the clauses are optional but may not be reordered, and a misordered clause is reported as a syntax error naming the expected order. A statement may prefix the whole query with EXPLAIN (Chapter 1.7), which validates and plans without executing. Each clause has a precise role developed in the following chapters; the important thing here is the overall shape — a query declares a frame, one structural aspect per clause.

The query produces a frame whose anchor is the output anchor named in AT, whose value columns are the series listed in SELECT, and whose rows are determined by the data, the filters, the ordering, and the limit.

1.3 The FROM clause

A query may name its Manifold explicitly (this is the FROM clause shown alone, not a whole query):

FROM finance_manifold

FROM is optional, and defaults to the bound Manifold. The demo, the MCP query tool, and the agent are each attached to exactly one Manifold, and against a single-Manifold surface a bare SELECT … AT {…} resolves against that bound Manifold — no ceremony on the common case. FROM earns its keep on a multi-Manifold surface: where a surface holds more than one Manifold, a statement that names none is not underdetermined-by-guess, it refuses, naming the available Manifolds and asking which. The clause exists so that any surface — a query console, an agent, an MCP client, a planner shared across several Manifolds — can address different Manifolds by naming the target in each statement, without carrying Manifold identity as ambient session state. A statement that names its Manifold is self-contained: it means the same thing regardless of who submits it, from where, with what session history — and is therefore reproducible, cacheable, and auditable on its face; and an unnamed one is reproducible all the same, because the resolved identity always rides in the annotation (below).

Resolved identity is never silent. A Manifold is a versioned artifact, and the result’s annotation always records the resolved identity — (Manifold, version, refresh, certificate state) — so that even a statement that named neither the Manifold nor a version is exactly reproducible after the fact, from its own annotation. An EXPLAIN shows the resolved identity before anything runs. (▸ Frame-QL revision, shipped-law — [SCHEDULED]: explicit version pinning — a VERSION n on FROM — is designed but not yet in the shipped envelope grammar; it enters by ruling (ADR-035 D1), not by default. [SCHEDULED] (not [ROADMAP]) because the surface is designed and committed, only unshipped — the resolved version is always disclosed today, so nothing is lost by pinning arriving later.)

Surfaces may supply FROM on the user’s behalf — licensed by disclosure. A user or client working against a single Manifold should not have to type its name in every statement, and the formal language does not bend to spare them: the resolution is the surface’s job. A surface whose context makes the target Manifold unambiguous injects the FROM clause before submission (the sanctioned sugar of Chapter 8.3). What licenses the injection is the disclosure machinery: the annotation’s canonical form carries the statement as executed, FROM and resolved version included, on every round trip — so the convenience can never become hidden state. The user need not know the clause exists; the record always shows it.

Injection is convenience, never authorization. A surface that exposes one Manifold makes others unaddressable through that surface — which can feel like access control, and is not. A programmatic client against the same planner may write any FROM it likes. Entitlement is enforced at the governance layer — the Manifold author’s access rules (the reserved ACCESS construct; an unentitled statement is the refuse outcome of Chapter 7.3) — and the surface’s injection sits strictly on the ergonomics side of that line.

Cross-Manifold analysis happens through composition, not through cross-Manifold queries: two Manifolds are composed into a third (by inheritance or joining, per the framework manual’s Chapter 8), and the composed Manifold is then queried like any other. There is no inline cross-Manifold join construct in the language — and a shared planner does not change this: it routes each statement to exactly one Manifold; the multi-Manifold question is answered by composing first and querying the composite.

1.4 The SELECT clause and the output anchor

The SELECT clause lists the series expressions that will become the value columns of the output frame, and ends with AT <output_anchor> declaring the output anchor of the whole query:

SELECT sum(revenue @ {transaction}),
       sum(cost @ {transaction})
       AT {customer}

Several structural points are worth being explicit about, because they govern how the language reads:

Series expressions are output columns. Each series in the list becomes one value column of the output frame. There is no other way for a series to enter the result — the SELECT list literally names the value columns of the frame.

The output anchor is a single declaration for the whole query. All series in a query share the output anchor, by structural necessity: the result is one frame, a frame’s columns are co-anchored, so all the value columns in the result are at the one declared anchor. The AT clause is therefore a query-level declaration, not a per-series one. There is no syntax for series at different output anchors in one query — different anchors require different queries.

The output anchor’s dimensions appear automatically as leading columns. A query at AT {customer, region} produces a frame whose first columns are customer and region — the anchor coordinates that identify the rows — followed by the value columns computed by the series expressions. The writer does not list anchor dimensions in SELECT; they are implied by AT and produced automatically.

The output anchor is required. A query without an AT clause is ill-formed and refused. The framework does not infer an output anchor from context, does not default to {}, and does not guess. (A specific tool or surface may choose to apply an implicit default at its own discretion — typically by interpreting “no anchor” as AT {} for ad-hoc exploration — but this is tool-level sugar, not part of the language. The formal language requires the anchor to be named.)

1.5 AT and @: output anchors versus input anchors

Every anchor specification in the language is one construct with one meaning, written with one of two keywords depending on position. The construct is anchor ascription:

expr @ a means the value of expr, materialized at anchor a.

The writer states where the value is wanted; the framework supplies whichever anchor-action realizes the ascription — aggregation through the relevant family when the expression’s grain is finer than a, broadcast (replication) when it is coarser, identity when they are equal. The legitimacy of the realization is checked by the ordinary machinery (the family’s B-anchor, the verified hierarchy edges, the order requirements); the ascription itself never names the mechanism, only the destination.

The destination, stated in full, is a pair. An anchor names a coordinate signature; it does not by itself determine which points inhabit it — two columns addressed by the same coordinates can live on different declared universes (the expected hourly readings and the alerts that happened to fire are both at {sensor, hour}, and are not about the same points). Every ascription is therefore implicitly expr @ (a, U): the anchor names the coordinates, the universe defines the points. The universe is resolved STRUCTURALLY, never pinned at query time (the §2c universe law): every column expression evaluates inside exactly one universe, and a single expression that spans two populations is a cross_universe category error, not a hedged answer. Which universe a column belongs to is fixed in its definition — ON is a definition-language clause, not query syntax — so a query never re-declares a population; it is disclosed: the resolved (anchor, universe) of every series rides in the result’s annotation. A frame that genuinely needs several populations juxtaposes them — separate columns, each evaluating in its own universe, standing together at a shared anchor (Chapter 2.4).

The choice of universe is consequential because a universe declares its basis, and the basis fixes what a missing row means (the shipped B3 law): in an events population, absence is a lawful zero — a SUM/COUNT over nothing is a true zero, immaterial; in a spine or product population, absence is a gap — the figure covers what is present, and the frame carries an incomplete_data caveat, never a silent zero; in a registry, membership is a checkable fact (present vs absent). Serving follows the declared basis; the resolved basis and any gap ride in the annotation. (The full universe model — bases, declarations, co-universality — is the framework manual’s; this manual specifies only what the query language exposes: structural universe resolution, the one-expression-one-universe law, juxtaposition, and the disclosure of the resolved (anchor, universe) and basis.)

This single rule covers every appearance of an anchor in the language. revenue @ {transaction} ascribes a column to its grain. sum(revenue @ {transaction}) @ {customer, month} ascribes a reduction’s result to an intermediate grain. revenue @ {} in a share-of-total ascribes a column to the Manifold scalar, realized by aggregation, then consumed at a finer grain, realized by broadcast. These are not three constructs; they are one construct in three places.

The two keywords mark the one structural distinction that remains:

@ writes an ascription in input position — the expression’s value at the named grain feeds a larger expression.

AT writes the one ascription that is nobody’s input — the query’s final output anchor. It appears exactly once per query, at the end of the SELECT list, and applies to all series.

The two are not interchangeable, and the framework refuses queries that use @ where AT belongs or vice versa — not because they mean different things (both are ascriptions) but because the query’s final output is structurally unique, and the language keeps that uniqueness lexically visible.

1.6 Series names and the AS alias

Each series in a query’s SELECT list becomes a value column of the output frame, and the column needs a name. A series expression may declare its name explicitly with AS:

SELECT revenue AS total_revenue,
       (revenue - cost) AS profit
       AT {customer}

When AS is not provided, the framework names the column by its canonical expression — verbatim, not a mechanical rewrite. A column’s identity is what it is: either the name you give it with AS, or the expression itself. No name is invented, and none is mangled.

Bare column reference. The column’s own name. revenue → revenue.

Member access. The dotted reference, verbatim. revenue.sum → revenue.sum (not revenue_sum — the framework does not rewrite the dot; the key is the expression).

Single reducer applied to a column. The reduction, verbatim, input anchor and all. avg(revenue @ {day}) → avg(revenue @ {day}); max(revenue @ {customer*month}) → max(revenue @ {customer*month}). The input anchor is part of the column’s identity (per the Two Anchors law), so it appears in the key — two reductions that differ only in their pin are different columns with different keys, never a collision on an invented name.

Any other expression. No derivable identity. The writer must supply AS. This includes:

For these the canonical expression is not a single-atom identity the framework will key on, so the query is ill-formed without AS.

Name collisions are refused, not silently resolved. If two series in the same query resolve to the same key — the same canonical expression with no AS on either — the query is refused with a structured error naming the collision. The framework does not append disambiguating suffixes or otherwise guess; the writer must declare an alias for at least one of the colliding series. The same rule applies if a series’s key (canonical or aliased) collides with an anchor dimension’s name (anchor dimensions appear automatically as leading columns of the output frame and own their names).

Keying on names across versions. Because an unaliased column’s key is its canonical expression, it will change if you rewrite the expression — and the mechanical defaults of earlier versions (sum_revenue, revenue_sum) are gone (contract_version "2", 0.14.0). A consumer that needs a stable handle should key on an AS alias: an alias is author-owned and will never change under any future rule.

Alias visibility. An alias declared in SELECT is visible in:

An alias is not visible in:

Valid alias names. An alias is an identifier subject to the same rules as a column name: alphanumeric with underscores, not starting with a digit, not a reserved keyword. The alias must not collide with anchor dimension names, with another series’s alias or default name, or with reserved identifiers in the Manifold’s namespace.

The aliasing rules require explicitness exactly where the canonical expression is not a single-atom identity (composite expressions, bracket filters, name collisions) and key by the expression itself everywhere else (bare columns, member access, single reductions). The writer is asked to name a series only when the framework cannot identify it by its own expression.

(One mechanism deliberately complements the alias rules: the macro binding of Chapter 4.5, WITH name = expression, which provides the expression reuse that aliases do not — an alias names an output column after reduction; a macro names an expression template before everything.)

1.7 Statements: queries and EXPLAIN

A statement is a query, optionally prefixed with EXPLAIN:

EXPLAIN
FROM finance_manifold
SELECT max( sum(revenue @ {transaction}) @ {customer, month} ) AS peak_month
       AT {customer}

EXPLAIN performs everything short of execution — parse, macro expansion, desugar to canonical form, type-check, plan, certificate consultation — and returns, instead of a result frame:

EXPLAIN never touches data. It is the validation surface of the propose-validate-refine loop: for an agent, the cheap inner loop that turns the substrate’s checking into a feedback signal before any execution cost; for a human, the audit of exactly how the framework read the query. A statement that EXPLAINs clean executes identically when resubmitted without the prefix — same canonical form, same plan — modulo certificate changes between the two calls.

Chapter 2. Series Expressions: The Canonical Form

This chapter specifies the series — one output column’s expression — in its fully-explicit canonical form: the form the planner type-checks against, where every reduction carries its input anchor @ {a_in} and the statement carries its output anchor AT {A}. The sugars of Chapter 3 let the writer omit an input anchor when it is determinable, and desugar to this form; the canonical form is the bedrock. Under the envelope (ADR-035, Chapter 1), @ {…} is the input-anchor marker universally and AT {…} is the sole output-grain declaration — one meaning for each marker, everywhere.

2.1 The shape of the canonical form

Canonically, a series is a reduction (or a map, or a composite of both) over column references, each reference carrying its input anchor, and the whole statement carrying one output anchor:

SELECT op( col_1 @ {a_1}, col_2 @ {a_2}, ... ) AS name
AT { A }

Every grain is named: every reduction reads an explicit input grain and produces the output grain A; a nested subexpression carries its own input anchor. The framework parses this form directly, type-checks it, and plans it. The sugars of Chapter 3 are normalizations to it.

Two facts give the canonical form its standing, and they are two views of one thing.

Semantic: under the anchor-ascription rule (Chapter 1.5), every sub-expression of a canonical-form statement has a local, self-contained meaning — this value, at this anchor — so the meaning of the whole is the composition of the meanings of the parts, with no ambient context to consult.

Architectural: the canonical form is the planner’s decomposition target. The planner receives a statement, expands and desugars it, and decomposes the plan into atoms — single-output metric requests of exactly the canonical shape, op(input @ {a_in}) @ {a_out}, each carried to the metric engine as a typed retrieval specification (the operator name, the input, the input and output anchors, any order specification, the resolved missing-behavior) — plus the map expressions the planner evaluates itself over the atoms’ co-anchored results. The metric engine answers each atom from its cache or by issuing the few specific backend calls; the engine never sees the statement, only atoms, and is envelope-blind — the planner owns assembly and the envelope’s clauses. A canonical-form statement is therefore not merely a fully-explicit notation: it is its own execution decomposition, readable off the text. This is why the canonical form is the bedrock — it is simultaneously the type-checked meaning of the statement and the unit of planning — and why EXPLAIN (Chapter 1.7) can return the atom decomposition as a faithful account of what execution will do.

2.2 Bare names: inputs versus complete series

A name like revenue may be one of two things, distinguished by where it appears:

As an input to an outer expression — for example, revenue inside sum(revenue) AT {A} — a bare name names the column’s family. It refers to the column under its default family’s reducer, sourced from the column’s root. The outer expression consumes the family; the framework brings the column from its root to wherever the outer expression needs it, applying the default-family reduction as part of getting there. The input position is well-defined for bare names because the outer expression’s output anchor and the column’s root together determine what is being consumed.

As a complete series with a matching grain — a bare measure standing alone in SELECT is a series iff its grain matches the output anchor AT {A} (or reaches it by a verified edge, carrying an implicit identity reduction). SELECT revenue AT {region} is well-posed when revenue reaches region. A bare measure that would require a reduction to reach AT but names none does not serve: it clarifies input_anchor_ambiguous, which must say at what grain, and how, it collapses. A measure family member is addressed with a dot — level.last, level.sum — the same suffix addressing the anchor grammar uses (Chapter 1); SELECT level.last AT {store} reads the end-of-period inventory snapshot per store.

2.3 Single-series reducers and the input pin

A reducer applied to a column, with an explicit input pin and the statement’s output anchor:

SELECT sum(revenue @ {transaction})  AS gross    AT {customer}
SELECT max(revenue @ {transaction})  AS peak_txn AT {customer}
SELECT avg(aov @ {day})              AS typical  AT {customer}

The first sums transaction-grain revenue up to customer — the natural extensive aggregation. The second takes each customer’s maximum transaction. The third averages the customer’s daily average-order-value.

The third’s choice of input anchor matters, and this is the central rule the canonical form enforces: when the answer depends on the input grain, the input anchor must be explicit. “Average per customer” is not well-defined without saying what is averaged — daily values? weekly? per-transaction? — and each gives a different number. For fertile reducers on a column with a unique natural grain, the input grain is determined and the sugar of Chapter 3 supplies it; for mules (avg, mean, a rate, a snapshot) and for genuinely underdetermined cases, the pin must be named. When it is omitted and the grain is underdetermined —

SELECT avg(aov) AT {customer}

— the framework does not guess. It clarifies on the shipped reason input_anchor_ambiguous, naming the candidate input grains as substitutable alternatives. Its sibling co_anchor_ambiguous is the multi-input case: a reduction over several inputs whose common input grain is not determined (one reason per contested dimension). An average with no declared input grain is a question, not an answer.

2.4 Map expressions

A map combines co-anchored columns point by point:

SELECT (revenue @ {customer, day}) - (cost @ {customer, day}) AS profit AT {customer, day}
SELECT (revenue @ {transaction}) / (orders @ {transaction})   AS aov    AT {transaction}

The operands of a map must be co-anchored — the same input anchor for all. Operands natively at different grains must first be brought to a common grain (by reduction, or by broadcast of a coarse value down a functional edge — §2.6). A map preserves the anchor: co-anchored inputs give a co-anchored result. A map may also appear inside a reducer — sum( (revenue @ {transaction}) - (cost @ {transaction}) ) AT {customer} computes per-transaction profit, then sums to customer grain.

A common anchor is, in full, a common (anchor, universe): columns combined at a shared anchor are checked for co-universality. Combining measures from different universes in one expression is not a hedged answer — it is the cross_universe category error (§2.5), on the query-error channel, not a map failure. To place several populations in one view, juxtapose them (separate series, each in its own universe — §2.5), never combine them in one expression.

2.5 One expression, one universe

Every column expression evaluates inside exactly one universe and never crosses the boundary — the §2c expression law (Chapter 1.5). A single expression that combines measures from more than one population is a category error, returned on the query-error channel as cross_universe (not a mood), naming the two legal paths: ask each population separately, or reconcile them into one. The retired cross-universe “wedge” — a single rate whose numerator and denominator live in different populations — lands here.

A frame in one universe needs no qualification at all: the population is implicit and the frame resolves (single-universe sugar). When several populations genuinely belong in one view, the frame juxtaposes them — separate series, each evaluating in its own universe, standing together at a shared anchor:

SELECT revenue AS revenue,
       level.last AS inv
AT {region}

Here revenue evaluates in its population and inv in its own; the frame carries both, each honest about where it comes from. ON is not part of the query grammar — a population is pinned in definitions, never re-declared at query time; universe is resolved structurally from each series’ measures. The resolved (anchor, universe) of every series rides in the annotation (Chapter 7.1).

2.6 Broadcast

When a column is brought to a finer anchor than the grain it arrives at, the anchor-action is a broadcast (the prior edition’s fillup):

SELECT (revenue @ {customer}) / (revenue @ {}) AS share_of_total AT {customer}

The denominator revenue @ {} is the Manifold-wide scalar — {} being the defining boundaries collapsed to one point — broadcast unchanged to every customer for the division. Broadcast has a single semantics — replicate: every finer point receives the same coarse value, whether intensive or extensive. Its purpose is to make a coarse value available at a finer grain for a downstream comparison (share-of-total, deviation-from-group-mean) — never to distribute a total into a finer-grained quantity. Proportional down-allocation is not an operation in the language.

The double-count hazard of replicating an extensive value — spreading a total and summing it back up — is foreclosed structurally: a broadcast value carries, in the B-anchor of any reducer that would sum it, the dimension family it was broadcast along, so an attempt to re-aggregate it back across that family is caught by the ordinary B-anchor check (Chapter 5.3) and served, if at all, only with the corresponding critical disclosure. There is no allocation rule to supply and none to get wrong.

2.7 Composite reductions

Two reducers compose when one applies to the result of another, with an explicit intermediate anchor:

SELECT max( sum(revenue @ {transaction}) @ {customer, month} ) AS peak_month AT {customer}

For each customer, the maximum monthly revenue. The inner sum(revenue @ {transaction}) @ {customer, month} produces monthly revenues per customer; the outer max reduces those monthlies to one peak. The intermediate anchor @ {customer, month} is essential — “max revenue per customer” is ambiguous without saying what kind of revenue (daily? monthly? transactional?), and omitting it clarifies input_anchor_ambiguous. A composite reduction is not a separate construct; it is the anchor-ascription rule applied recursively — nested ascriptions, each local, read by the planner as two atoms, the inner feeding the outer (§2.1). Any composite reduction requires an AS alias, since no defensible default name can be derived from a nested expression (Chapter 1.6).

2.8 Subsetting and scans [ROADMAP]

Two further expression forms are documented as the language the envelope grows into by ruling (ADR-035 D1); they are not part of the shipped 0.9.0 envelope, and the ratified grammar spec keeps them roadmap (they are grammar the engine does not yet have). They enter the language when ruled in, not by accretion.

The bracket filter restricts a column reference to a subset of its domain — revenue[region = "east"] — producing a degenerate column that carries its restriction as part of its identity (framework manual, Chapter 6). It differs from WHERE (Chapter 4.1): WHERE restricts the whole statement’s input; a bracket filter restricts an individual reference within one expression. Whether an emptied bucket appears with a zero, an absent value, or not at all is decided by the destination universe’s basis (Chapter 1.5, the B3 law): an events population makes an emptied SUM a true zero; a spine/product population makes it a gap carrying incomplete_data; the resolved basis rides in the annotation.

Scans are order-dependent, anchor-preserving operations — cumsum(revenue @ {customer, day}), lag(...), rolling_mean(..., window=7d) — whose output at each point depends on an order and on neighbors. The order is derived from an orderable axis in the anchor (day) or named explicitly; an order-dependent operation whose order is neither derivable nor named is a clarification. The family-aware parameters reset / within / step lean on the verified dimension-family structure to express calendar and hierarchy intelligence directly (year-to-date via reset = year; same-period-last-year via step = year); each names a coarser level in the order axis’s family, reached by a verified climb, and a parameter riding a CONTRADICTED edge carries that finding. (The full scan reference is Appendix A.)

2.9 The grammar grows by ruling

The envelope is deliberately small, and it grows by ruling (ADR-035 D1), not by accretion: a construct enters the language when it is ruled in, and until then it does not exist. This is why Chapter 6’s examples carry a shipping mark — an executable example runs against the shipped package or it is marked pending/roadmap; the manual is structurally incapable of documenting a query it did not run. What ships in 0.9.0 is the envelope of Chapter 1 with the series forms of §§2.1–2.7: reductions with explicit input pins, maps, composite reductions, the anchor product, one universe per expression, every grain either determined or pinned — and every answer one of the four moods (serve · disclose · clarify · refuse).

Chapter 3. Sugars

The canonical form of Chapter 2 is precise but verbose. Two sugars permit common cases to be written more compactly, with the framework expanding them mechanically to the canonical form before type-checking. Each sugar has a precise applicability condition; outside that condition, the canonical form is required.

3.1 Sugar: default-family reduction implicit

When a column reference appears in a position that requires a reduction to land at the outer expression’s anchor, and the column’s default family’s reducer is the intended reduction, the reducer may be omitted. The column reference col @ a (or just col as an input naming the family) is read as defaultReducer(col) applied appropriately.

In practice this means a writer can usually write revenue instead of sum(revenue) when revenue’s default family is sum:

revenue AT {customer}               -- sugared
sum(revenue @ {transaction}) AT {customer}   -- canonical (assuming revenue's root is {transaction})

The sugar applies in two combined ways. First, the default reducer is inferred from the column’s declared default family — the framework looks up the column-spec, finds the default family, and uses its reducer. Second, when the column reference has no explicit input anchor and the path from the column’s root to the output anchor is determined (a single fertile path), the input anchor is taken to be the column’s root.

Both inference steps are mechanical and deterministic. The framework refuses to apply the sugar when either inference fails: when the column has no default family (so there is no default reducer), or when the path from root to output anchor is ambiguous (so the input anchor cannot be inferred).

Some specific cases where the sugar does not apply, and the canonical form is required:

3.2 Sugar: omitting @ root(col) for direct fertile reduction

A specific common case deserves separate notice because it is the most-used form. When a column reference appears as an input to a fertile reducer, and the column’s root is the deepest grain involved in the reduction, the input anchor @ <root> may be omitted:

sum(revenue) AT {customer}    -- sugared
sum(revenue @ {transaction}) AT {customer}   -- canonical, when revenue's root is {transaction}

This is technically a corollary of the path-determinism case of Sugar 3.1, but it is so common that it is worth stating directly. When the writer omits @ a and the framework infers it as the column’s root, that inference is silent and automatic, and the resulting form is what almost all simple queries look like.

3.3 What sugar does not do

Two things the sugar mechanism explicitly does not do, because both would invite ambiguity the framework refuses to resolve silently:

No implicit output anchor. The output anchor (AT A) is always required and never inferred. A query without AT is ill-formed and refused, regardless of what could be guessed. This is the strict rule from Chapter 1.4.

No implicit reducer for non-default families. A bare column name is read as its default family; if the writer wants a non-default reducer, the reducer must be named. The framework does not infer “you probably meant max” from context.

The combination is: the language gives the writer one defensible default (the column’s declared default family with the column’s root as the input anchor, when the path is determined), and otherwise requires explicit specification. The agent and any sugar layer may apply additional defaults as tool-level conveniences, but the formal language does not.

Chapter 4. Clauses

4.1 The WHERE clause: pre-query input filtering

WHERE restricts the input to the query — applied before reductions, narrowing what data the reducers see. Its predicate references dimensions and column values at the input grain (the data being reduced), not the output grain (the result of the reduction).

FROM finance_manifold
SELECT sum(revenue) AT {customer}
WHERE region = "east" AND date >= "2024-01-01"

This computes per-customer revenue over transactions where region is east and date is in 2024 or later. The framework restricts the input data to the predicate’s satisfying rows, then performs the reduction over the restricted input. The output is per-customer sums of east-region 2024+ revenue.

WHERE predicates may reference any dimension or column at the input grain of any series in the query. They are evaluated before reduction, so they see the raw data; references to series expressions (which are output columns) are not permitted in WHERE — those belong in HAVING.

WHERE is a query-time expression-level restriction, not a Manifold-level state change. The Manifold’s columns retain their declared coverage; the query’s result is restricted to what the predicate permits.

When series have different input grains, the predicate applies per series, at each series’ own input grain. A query whose series draw from {transaction} and from {warehouse, day} under WHERE region = "east" restricts each series’ input independently: each predicate dimension must be reachable from that series’ input anchor (directly as a coordinate, or through a verified hierarchy edge — region reached from transaction via transaction → customer → region). If a predicate dimension is not reachable from some series’ input anchor, the query is underdetermined for that series — there is no defensible way to apply the restriction — and the framework asks for clarification rather than silently applying the predicate to some series and not others.

4.2 The HAVING clause: post-query output filtering

HAVING filters the output frame after the series have been computed. Its predicate references the output columns — the series expressions and the anchor dimensions — and is applied after reduction:

FROM finance_manifold
SELECT sum(revenue) AS total_revenue,
       count(*) AS transaction_count
       AT {customer}
HAVING total_revenue > 10000

This computes per-customer total revenue and transaction count, then keeps only the rows where total_revenue exceeds 10,000. The framework applies the reduction first, then evaluates the HAVING predicate against each row of the output frame.

HAVING references output columns by name — a series’s alias or default name, or an anchor dimension’s name. It cannot reference the input grain (that information has been reduced away by the time HAVING evaluates). The prior edition also permitted referencing a series by repeating its expression; that form is removed — deciding when two expressions are “the same” (before or after desugaring? with or without inferred anchors?) raised more questions than the convenience answered — and the reuse it served is covered cleanly by macro bindings (Chapter 4.5), which expand identically at every site by construction.

The distinction between WHERE and HAVING is precise: WHERE filters input rows (before reduction, referencing input-grain columns); HAVING filters output rows (after reduction, referencing output-grain columns). Some predicates make sense as both — “region = east” can be a WHERE on input rows or a HAVING on an output that includes region — but the meaning differs: WHERE restricts what is reduced; HAVING restricts what is shown.

4.3 The ORDER BY clause

ORDER BY specifies the sort order of the output frame:

ORDER BY total_revenue DESC, customer ASC

Multiple sort columns are evaluated in left-to-right priority. The framework permits sort by any output column — anchor dimensions or series expressions (referenced by alias or expression). ASC is the default direction; DESC reverses.

ORDER BY is also consulted by LIMIT n PER (next section) to determine within-group ordering when partitioned truncation is requested. The same ORDER BY clause serves both purposes: it is the canonical ordering of the result.

4.4 The LIMIT n clause and LIMIT n PER {dims}

LIMIT n keeps the first n rows of the ordered result:

ORDER BY total_revenue DESC
LIMIT 100

This keeps the 100 customers with the highest total revenue — a flat top-N truncation.

LIMIT n PER {dims} keeps the top n rows within each group defined by the named dimensions:

ORDER BY region, total_revenue DESC
LIMIT 5 PER {region}

This keeps the top 5 customers (by total_revenue) within each region — partitioned top-N. The result has 5 rows per region (or fewer if a region has fewer than 5 customers): 5 from the east, 5 from the west, and so on.

The clause has precise requirements:

The PER dimensions must be coordinates of the output anchor. Partitioning by something not in the output frame is meaningless — there is nothing in the result to partition by. The framework refuses PER {region} if region is not part of the query’s output anchor.

The PER dimensions must be a subset of the ORDER BY columns. This makes the within-group order deterministic: the columns in PER group the result, and the remaining ORDER BY columns rank within each group. Without PER ⊆ ORDER BY, the within-group ranking would be unspecified.

Ties are broken by the remaining ORDER BY columns. If 5 customers tie for the 5th-highest revenue within a region, the framework consults the next ORDER BY column to disambiguate. If ORDER BY does not fully disambiguate within PER groups, the framework discloses the under-determination at plan time and asks for clarification at execution time only if the ambiguity actually arises and matters for the truncation.

The empty PER set is permitted and reduces to flat LIMIT n. LIMIT 5 PER {} partitions the result into one group (the whole result) and takes the top 5 — equivalent to plain LIMIT 5. The form is uniform; the empty case is the degenerate flat case.

The LIMIT n PER form is genuinely common in analytics — top products per category, top customers per region, top events per session — and is awkward in conventional SQL. Having it as a first-class clause makes the language match the natural shape of the questions analysts ask.

4.5 The WITH clause: named bindings

The WITH clause declares named bindings used by the statement. There are two kinds, sharing a clause but doing different jobs.

Allocation bindings [Pro] — WITH allocation <name> = <rule> — declare the partition-of-unity weighting an aggregate-across a many-to-many relationship uses (Chapter 5.6, Example 6.12). They are semantic declarations consumed by the planner. (In Core, the undeclared aggregate-across is a clarification, per Chapter 5.6; allocation, which resolves it, is a Pro construct.)

Macro bindings — WITH <name> = <expression> — are a purely syntactic device: name an expression once, reference it by name. Expansion is textual and happens before desugaring and type-checking — each reference is replaced by the bound expression, and then every ordinary rule applies at the site of use. Everything about macros follows from “textual, pre-everything”:

A worked example:

FROM finance_manifold
WITH profit = (revenue - cost)
SELECT profit AT {customer, month}

desugars (assuming co-anchorable defaults) to SELECT (revenue - cost) AS profit AT {customer, month} — and the same profit binding may simultaneously appear in HAVING profit > 0, expanding identically there.

In the shipped envelope’s fixed clause order, WITH precedes SELECT (it binds the names the series then reference); its lexical position carries no scoping meaning beyond that ordering — bindings are visible throughout the statement. (The distinction from AS is worth restating because SQL offers nothing like it: an alias names an output column after reduction and is visible only where output columns exist; a macro names an expression template before everything and is visible wherever expressions are.)

Chapter 5. Type Rules

This chapter specifies the rules the framework applies when type-checking and planning a query. The rules divide into two kinds, and the division matters (Chapter 7 develops it fully). Well-formedness rules are about whether the query is an executable instruction at all — whether it determines one answer. A query failing a well-formedness rule cannot be executed, and the framework responds with a structured clarification naming what is underdetermined. Soundness checks are about the analytical risk of an executable query — B-anchor crossings, coverage gaps, missingness mechanisms, contradicted preconditions. Per the inform-and-serve doctrine, a query that fails a soundness check is not withheld on that account: it is executed, and the risk is returned as a disclosure bound to the result, with severity, quantification, and remedy. The framework withholds a result only when it cannot execute (well-formedness), when the data is not there, or when a declared governance rule forbids it — never on its own analytical judgment.

5.1 Output anchor required

Every query must specify an output anchor via AT. Queries without AT are ill-formed and refused. This is the strict rule from Chapter 1.4.

5.2 Anchor compatibility

The output anchor of every series in a query must be the same (the query’s output anchor). Since series expressions are checked individually and the language requires shared output anchor by structural necessity (Chapter 1.4), this rule is enforced by the AT being a query-level declaration rather than per-series.

For inputs within an expression, anchors must compose correctly:

5.3 B-anchor checking

For every reduction in a query, the framework checks whether the reducer’s B-anchor for the column includes any of the dimension families being eliminated by the reduction:

A B-anchor crossing is a soundness finding, not a well-formedness failure: the query determines an answer (the arithmetic is computable), so per the inform-and-serve doctrine the framework serves the result, bound to a critical disclosure naming the column, the reducer, and the crossed family, quantifying the hazard where it can (the replication or double-count factor), and — when the column’s family set contains an alternative reducer applicable along the crossed axis — naming that alternative as the remedy (typically last for the time axis of a stock). Where a sound route exists that dissolves the crossing rather than flagging it — recomputing from a fertile source at the column’s root instead of re-aggregating a cached image — the planner routes around and discloses the routing as an informational finding instead. A Manifold author may additionally declare a hard stop (WITHHOLD) on a (column, reducer) where governance requires that the crossing never be served; that withholding is the author’s rule, reported as such, not the engine’s analytical judgment.

▸ Frame-QL revision (locus). The B-anchor crossing is detected statically by the planner — the B-anchor and the path’s eliminated families are knowable from the spec, before any data is touched — and it is the planner that attaches the critical disclosure. What is not static is whether a route-around dissolves the crossing; that is a resolution fact, so the column engine may, during execution, recompute from a fertile root and downgrade the finding to an informational route-around (RECOMPUTED_FROM_DETAIL), or leave it critical when no sound route exists. The disclosure thus originates at the planner and has its final severity settled at the engine; in every case the number is served. This is the inform-and-serve boundary in miniature: the planner names the risk, the engine never withholds on it.

5.4 Family selection

When a column reference appears with a non-default reducer named, the reducer must be in the column’s family set. Calling a reducer on a column whose family set does not declare it is refused; the framework lists the available reducers in the family set.

When the default-family sugar applies (Chapter 3.1), the framework uses the column’s declared default family’s reducer.

5.5 Order specification for order-dependent operations

Every order-dependent operation (scan, ordered reducer) must have its order determinable:

If neither applies — the operation is order-dependent but no order is determined — the framework refuses, naming the operation and the missing order specification.

For the family-aware scan parameters (reset, within, step — Chapter 2.8): the named level must be a member of the same dimension family as the scan’s order axis, reachable from the order’s grain by declared roll-up edges. A level outside the order’s family, or unreachable from its grain, leaves the parameter underdetermined — a clarification. The climb the parameter rides is checked like any climb: its edge’s integrity verdict enters the query’s dependency cone, and a CONTRADICTED edge surfaces as a disclosure on the served result.

5.6 Many-to-many

Aggregating a measure across a many-to-many relationship has a precondition the relationship does not supply — functionality, the partition property that makes per-bucket totals reconcile to the grand total. The check applies specifically to aggregate-across operations; queries that use the many-to-many in other ways (filtering, listing memberships, querying either side independently) invoke no such precondition and are unaffected.

When a query aggregates across a many-to-many without a declared resolution, the framework serves the membership aggregation — each fine entity contributing its full value to every bucket it belongs to — bound to a disclosure stating that the precondition is unmet: per-bucket totals overlap, their sum exceeds the grand total by the overlap amount (quantified in the finding), and the result answers a membership question (“revenue of products in each category”), not a partition question. The disclosure names the three resolutions as remedies: a membership filter (accept the overlap deliberately), a primary designation (convert the relationship to functional), or an allocation declared via WITH allocation (Example 6.12), which supplies the partition-of-unity that makes totals reconcile.

A query that declares one of the three resolutions is computed under it with no precondition finding.

▸ Frame-QL revision (column-foundation). The paragraph above describes a join-based engine, where the membership aggregation is always computable and so can be served with an overlap disclosure. Frame-QL’s engine (ADR-031) transports a measure along functional edges and never joins; a non-functional (M:N) edge is not a traversable path, so the membership aggregation is not expressible at all without one of the three resolutions. The undeclared aggregate-across is therefore caught by the planner as a well-formedness clarification — “this aggregate-across rides a non-functional edge and is underdetermined; supply a membership filter, a primary designation, or WITH allocation” — returned statically, before execution, with no result. The three remedies are unchanged; only the outcome class moves, from serve-with-disclosure to clarify, because the engine cannot fabricate a number it has no path to. A declared resolution makes the query well-formed exactly as before. (This is the one place the inform-and-serve doctrine yields to well-formedness: the engine still never withholds a number it can produce — here it genuinely cannot.)

▸ Frame-QL revision (RELATE faces, v0.11). The clarification above is now a menu. A relationship may declare faces — named crossing dispositions — and the undeclared-aggregate-across clarify lists the declared faces with their folklore as its alternatives, so the human re-issues by naming one. A face names the value’s disposition on the trip, never the selection criterion; the three are a self-teaching verb triad, mutually exclusive and jointly covering the sensible crossings of a many-to-many by an additive quantity:

  • touch — the value reaches every match. Per-bucket totals overlap and their sum exceeds the grand total (and, where the fine side has members in no bucket, can also fall short of it — both skews disclosed). This is the membership question (“revenue of products in each category”), now executed, not just described.
  • assign — the value goes to exactly one match (a declared canonical pick — by a primary flag, first-listed, max-weight — adjudicated as “exactly one per fine entity” regardless of the criterion). Collapses the M:N into a functional edge.
  • alloc — the value splits across matches (equal shares, or a declared weight); per-bucket totals reconcile to the grand total. The WITH allocation construct of §6.12 is this face’s parameter.

A face is declared on the relationship (not parameterized per query — EXPLAIN stays static and the cache keys the name), described per the folklore rule, and adjudicated at publish: a face is closed by default, and its license opens the crossing. Faces are addressed as a qualified coordinate, {<coordinate>.<face>}:

RELATE product <-> category VIA product_categories(product_id, category_id)
    FACES { touch = TOUCH -- "revenue reaches every category a product sits in — deliberately multi-counted; totals exceed the grand total" }
    NOTE "a product belongs to up to 3 categories"

SELECT revenue AT {category.touch}      -- disclose: over_count (caution) + coverage (info)

The bare {category} stays uniformly barred and now clarifies with the face menu; {category.touch} executes the join-multiply and serves in disclose, carrying the over-count as a material caveat and the coverage (the covered/uncovered count) as an informational one. Naming honesty holds: category.touch ≠ category, marked in the grain name. Per-basis licensing, v1 = events only — on an events population the expansion is honest arithmetic; on a spine/grid, replication would corrupt the grid’s own completeness claim, so the crossing is refused there until that thinking lands.

v1 executes touch only. assign and alloc are declared-but-deferred: the grammar knows them, but declaring one is an honest, fail-closed parse refusal — “FACE <name>: scheme assign is declared-but-deferred — v1 executes TOUCH only (assign/alloc are post-launch ledger items)” — so no manifold can carry an inert face that silently serves nothing.

5.7 Coverage compatibility [Pro]

When a query combines columns at a shared anchor, the framework consults the coverage states of the columns:

Coverage in the marginal-mismatch range (between the Manifold’s configured thresholds) is served with the coverage state disclosed; combinations across a coverage gap carry the gap, its asymmetry, and the resulting bias direction in the annotation. The coverage machinery is part of the framework’s universe-integrity layer (per the framework manual’s Chapter 10), and its findings carry the certificate state of the preconditions they derive from.

Chapter 6. Examples

This chapter shows worked examples that exercise the language across its common shapes. Each example shows the query, the canonical desugared form where it differs, and a brief note on what the query computes.

6.1 Simple aggregation

FROM finance_manifold
SELECT revenue AT {customer}

Canonical: FROM finance_manifold SELECT sum(revenue @ {transaction}) AT {customer} (assuming sum is revenue’s default family and {transaction} is revenue’s root).

Computes per-customer revenue total.

6.2 Multiple metrics at a shared anchor

FROM finance_manifold
SELECT revenue,
       cost,
       count(*) AS transaction_count
       AT {customer, month}

Computes per-customer-month revenue, cost, and transaction count. All three series share the output anchor {customer, month}.

6.3 Composite reduction with explicit intermediate anchor

FROM finance_manifold
SELECT max( sum(revenue @ {transaction}) @ {customer, month} ) AS max_monthly_revenue
       AT {customer}

Computes, for each customer, the maximum monthly revenue. The intermediate anchor @ {customer, month} is required — it specifies that the question is about monthly maxima, not daily or transaction-level maxima. Without the inner anchor, the question would be ambiguous and the framework would ask for the grain (a clarification, per Chapter 7). The AS alias is also required for any composite reduction, since the framework cannot derive a default name from a nested expression.

6.4 Mean with explicit input anchor

FROM product_manifold
SELECT mean( engagement_score @ {customer, week} ) AT {customer}

Computes the per-customer mean of weekly engagement scores. The mule reducer mean requires the input anchor to be explicit — “average engagement per customer” is meaningless without saying what kind of engagement is being averaged.

6.5 Map of co-anchored columns

FROM finance_manifold
SELECT (revenue - cost) AS profit AT {customer, day}

Computes per-customer-day profit by subtracting cost from revenue at the daily grain. Revenue and cost must be co-anchorable at {customer, day}; the map combines them point by point.

6.6 Ratio across grains

FROM finance_manifold
SELECT ( revenue @ {customer} ) / ( revenue @ {} ) AS share_of_total
       AT {customer}

Computes each customer’s share of total revenue. The numerator is per-customer revenue; the denominator is the Manifold-wide grand total ({} being the Manifold’s defining boundaries collapsed to one point). The denominator is broadcast — replicated unchanged — to every customer for the division.

6.7 Bracket filter on a column

FROM finance_manifold
SELECT revenue[region = "east"] AS east_revenue,
       revenue AS total_revenue
       AT {customer}

Computes per-customer east-region revenue alongside total revenue. The bracket filter restricts the revenue column to its east-region subset; the result column appears alongside unrestricted revenue.

6.8 WHERE: pre-query filtering

FROM finance_manifold
SELECT revenue AT {customer}
WHERE date >= "2024-01-01" AND region IN ("east", "west")

Restricts the input to transactions in 2024+ and east or west regions before computing per-customer revenue. The output is per-customer revenue over the restricted input.

6.9 HAVING: post-query filtering

FROM finance_manifold
SELECT revenue AS total_revenue AT {customer}
HAVING total_revenue > 50000

Computes per-customer revenue then filters out customers with revenue at or below 50,000.

6.10 Top N per group

FROM product_manifold
SELECT product_revenue AT {category, product}
ORDER BY category, product_revenue DESC
LIMIT 5 PER {category}

Computes per-category-product revenue, sorts by category then by revenue descending, then keeps the top 5 products per category. The result has up to 5 rows per category, with each category’s rows ordered by revenue.

6.11 Scan for running total

FROM finance_manifold
SELECT cumsum( revenue @ {customer, day} ) AS revenue_to_date
       AT {customer, day}

Computes the running cumulative revenue per customer, ordered by day. The scan partitions by customer and accumulates by day; the result is co-anchored with the input.

6.12 Many-to-many with allocation [ROADMAP]

The WITH allocation <name> = … form is a grow-by-ruling surface (ADR-035 D1): it is not part of the shipped envelope grammar, so the block below is marked frameql-illformed — the shipped parser rejects its WITH allocation head, and the manual’s self-check asserts exactly that until the form ships.

FROM product_manifold
WITH allocation product_to_category = proportional_to(category_weight)
SELECT sum( revenue @ {product} ) AT {category}

Computes per-category revenue when products belong to multiple categories. The WITH allocation clause declares the allocation rule the framework uses to apportion each product’s revenue across its categories; without this declaration, the aggregation across the many-to-many relationship would be served with membership semantics and an overlap disclosure (Chapter 5.6 — or, under the column-foundation, raised as a clarification; see the revision note there); the allocation supplies the partition-of-unity that makes per-category totals reconcile, with no precondition finding.

6.13 Time intelligence: year-to-date and year-over-year

FROM finance_manifold
SELECT cumsum( revenue @ {customer, day}, reset = year ) AS revenue_ytd
       AT {customer, day}

Computes per-customer year-to-date revenue: the running sum restarts at each year boundary, the year level reached from day through the verified time family.

FROM finance_manifold
SELECT ( revenue - lag(revenue, 1, step = year) ) / lag(revenue, 1, step = year) AS yoy_growth
       AT {customer, month}

Computes per-customer year-over-year revenue growth at month grain: lag(…, 1, step = year) is the same month one year earlier, and the growth is an ordinary map over the co-anchored pair.

6.14 Macro bindings for reuse

FROM finance_manifold
WITH profit = (revenue - cost)
SELECT profit AT {customer, month}
HAVING profit > 0
ORDER BY profit DESC
LIMIT 10

The profit macro expands textually at every reference site before type-checking; as a bare-macro series it takes profit as its default name. The statement desugars to one canonical query with (revenue - cost) AS profit selected, filtered, and sorted — visible in full via EXPLAIN.

6.15 The envelope, end to end

The top-3 stores per region by gross, with each region’s typical order value — the ratified acceptance target for the envelope (ADR-035), exercising anchor products, an auto-leading coarser coordinate, a WITH macro, two different input grains juxtaposed in one frame, and PER ⊆ ORDER BY:

FROM retail
WITH line = revenue @ {transaction}
SELECT sum(line)        AS gross,
       avg(aov @ {day}) AS typical
AT { region * store }
ORDER BY region, gross DESC
LIMIT 3 PER { region }

Reading it clause by clause: AT {region * store} stands at the (region, store) grid — region auto-leads as the coarser coordinate (the edge region ← store carries it), an anchor coordinate rather than a selected series, and so nameable by ORDER BY/PER. WITH line = revenue @ {transaction} binds the transaction-grain line once; sum(line) reduces it to (region, store) — the reduction is edge-carried (transaction → store → region), the grammar admitting it while the kernel’s edge verdict decides serve vs disclose. typical = avg(aov @ {day}) reads aov at day grain and averages to (region, store) — two different input grains (transaction, day) collapsing into one frame, legal because each series names its own @. ORDER BY region, gross DESC references the frame’s own columns; region leads so groups are contiguous, gross DESC ranks within. LIMIT 3 PER {region} keeps the top 3 per region — well-posed because region is both an anchor coordinate AND an ORDER BY key (the conjoined PER law: PER keys are anchor coordinates only, and PER ⊆ ORDER BY). PER {typical} would refuse (an alias, not a coordinate); PER {store} would refuse unless store is added to ORDER BY.

Chapter 7. Outcomes, Disclosures, and Clarifications

This chapter specifies what a query returns and what it returns it with. The prior edition specified a refusal taxonomy; this edition replaces it, per the inform-and-serve doctrine: analytical risk is disclosed, never used as grounds to withhold a result the query determines. The chapter specifies the universal return contract, the complete outcome taxonomy, the structure of disclosures, and the control model that governs who may cause a result to be withheld.

7.1 The universal return contract: (result, annotation)

Every query returns a pair:

An annotation is a list of targeted findings plus a frame-level severity rollup. Each finding carries: a code (the machine-readable category); a severity (none < info < caution < critical); the originating four-state certificate_state (VERIFIED / CORROBORATED / CONTRADICTED / UNTESTABLE); a targeted subject (the specific metric, column, anchor, or cell it concerns — never smeared frame-wide); a plain-language headline and detail; a quantification where one is computable (a caveat with a number is honest; “may be inaccurate” is noise); the bias direction and bound where determinable; an actionable remedy; a requires_acknowledgment flag; and a provenance_ref linking the finding to the certificate entry or precondition it derives from, so the finding travels bound to the frame and remains auditable.

Beyond its findings, the annotation always carries the canonical desugared form of the statement as executed — every macro expanded, every sugar normalized, every inferred anchor and reducer explicit, the FROM clause and its resolved Manifold identity (Manifold, version, refresh, certificate state) included, and the resolved (anchor, universe) of every series — together with a reference to the plan’s atom decomposition (Chapter 2.1). Every result is thereby auditable against the exact reading the framework gave the query: a human checks the interpretation at a glance, and an agent learns the canonical language from every exchange rather than only from documentation. (EXPLAIN, Chapter 1.7, returns the same material without execution.)

The annotation is computed in full for every query. A Manifold’s disclosure_default (with optional per-call override) controls how much of it is returned — MINIMAL / STANDARD / FULL — as a serialization-time view, never as a control on what is assessed, and subject to a non-negotiable safety floor: the overall severity, every critical finding, and every requires_acknowledgment flag are present at every level. Verbosity may trim noise; it may never silence risk. The canonical form rides at every disclosure level — interpretation is never noise.

7.2 The outcome taxonomy

Four outcomes exhaust the contract — the shipped wire’s four moods, serve · disclose · clarify · refuse (the cross_universe category error rides the separate query-error channel, not a mood):

Serve, clean. Every precondition in the query’s dependency cone is VERIFIED or CORROBORATED, no soundness finding fires. The annotation is the single OK finding.

Serve, with disclosures. The query is executable and determinate, and one or more soundness findings apply. The result is served; the findings ride with it. This outcome has a first-class sub-case, route-around: where the planner can dissolve a risk rather than flag it — recomputing a mule from its fertile sources instead of re-aggregating a cached image; serving from transaction detail when a CONTRADICTED or stale pre-aggregate would otherwise be used — it does so, and discloses the routing as an info finding (RECOMPUTED_FROM_DETAIL, SERVED_FROM_DETAIL) rather than disclosing a risk that no longer exists. Only when no sound route exists does the risk surface as a caution or critical finding on the served number.

Clarify. The query is not an executable instruction: it does not determine one answer. The framework returns no result and opens a structured clarification naming exactly what is underdetermined and what would determine it. How the clarification is conducted is a surface decision — a console may show the message, an agent may ask the question in the requester’s vocabulary, an MCP client may receive the structured payload and resubmit.

Refuse. The query is well-formed but no result can or may be produced for a non-analytical reason: the data is not there, or a declared governance rule withholds it. The framework returns no result and states the reason and the remedy. (▸ shipped-law: this is the wire’s fourth mood, refuse. The Third-Edition lineage called it inform, under ADR-020’s “inform-and-serve” doctrine — same outcome, same spirit; the shipped name is refuse.)

There is no fifth outcome. In particular there is no analytical-risk withholding: any result that is determinate and producible for an entitled caller is produced, with its risk on its face.

▸ Frame-QL revision (the doctrine’s locus). The four outcomes map cleanly onto the two-projection architecture (Chapter 2.1). Clarify and the governance/no-data half of refuse are planner outcomes: they are decided statically, from vocabulary and shape, before the column engine resolves anything — ambiguity, a missing anchor, an unknown operator, a type mismatch, a non-traversable edge, a WITHHOLD or access rule, an empty realized support. Serve, clean and serve, with disclosures are engine outcomes: once a query is well-formed and permitted, the engine resolves it and any surviving analytical risk rides out as a disclosure. The boundary between “may withhold” and “must serve” is therefore the boundary between the planner and the engine — the planner may refuse what is not an executable, permitted instruction; the engine, having an executable one, never withholds on its own analytical judgment. This is why the canonical form is both the type-checked meaning and the unit of planning: the same decomposition that the planner validates is the decomposition the engine serves.

7.3 The withholding outcomes, enumerated

A query yields no result in exactly these cases:

Clarifications — the query cannot be executed as posed:

Refusals — the query executes against nothing, or a rule withholds it:

Everything formerly in this chapter’s refusal taxonomy that does not appear above — B-anchor violations, mule re-aggregation, many-to-many aggregate-across, coverage mismatch, MNAR, contradicted integrity preconditions — has moved to 7.4: those queries are served, with disclosures.

7.4 The disclosure catalog

The principal soundness findings, each carrying the fields of 7.1:

The disclosure severities are findings about the data and the operation; they are never the engine’s grounds for withholding. The safety asymmetry of the certificate is preserved verbatim in the findings: a finding’s certificate_state retains the state a pre-ADR-020 refusal would have minted — CONTRADICTED or UNTESTABLE — demotable at most to CORROBORATED by a passing data check, never silently to VERIFIED.

7.5 The four-party control model

Withholding and handling are governed by four parties, each with a distinct, legitimate authority. The engine’s neutrality among them is the design’s load-bearing property.

The Manifold author exercises corporate governance. The author already controls the Manifold’s entire shape — scope, sources, columns, operators, the V/M/B declarations — and the same authority extends to declared withholding rules: per-(column, reducer) hard stops (WITHHOLD in the definition language), access and entitlement rules, and policy customization of operator behavior. When an author rule fires, the engine reports it as the author’s rule: the refuse outcome names the declaration, its rationale where given, and the permitted alternatives.

The engine withholds only what it cannot do: an ambiguous query is not an executable instruction and must be clarified first; an empty data support cannot be served. These are the engine’s only inherent non-serves — soft, structural, and free of judgment. In every other case the engine’s stance is fixed: serve, and disclose. The engine executes governance rules it did not author and produces disclosures it does not act on; it never withholds on its own analytical opinion.

The surface — a query console, the agent, an MCP server, an embedding application — decides how outcomes are handled: how a clarification dialogue is conducted, how a refuse is presented, and what to do with a served result given its disclosure content. Severity-driven presentation gates (blur-until-acknowledged on requires_acknowledgment, confirmation steps on critical findings) are surface decisions, applied to presentation, never API-level withholding — entitled programmatic callers receive the bytes with the annotation, or composition breaks.

The user may declare their own suppression preferences — “do not return results whose disclosures match these conditions” — implemented at the surface, or communicated to the engine as a per-call policy. This is the user governing their own consumption, the symmetric counterpart of the author governing the corpus.

One sentence holds the model together: the author may forbid, the engine may only clarify or report emptiness, the surface decides presentation, and the user decides their own threshold — and every one of these is visible in the annotation, so no withholding and no risk is ever silent.

7.6 Message conventions

Every clarification, refuse, and finding includes: its category code; the specific column, expression, or precondition concerned; the data version and certificate state at the time; and a remediation where one is determinable. The conventions hold across surfaces because they are properties of the structured payload, not of any one presentation: a clarification like “the inner grain of this mean is underdetermined — name the input anchor (@ {customer, day} or @ {customer, transaction} give different answers)” and a finding like “the roll-up edge customer → region is CONTRADICTED (customer 19847 maps to both ‘east’ and ‘west’); served from transaction detail, routing disclosed” are designed to be acted on — by a person reading them or by an agent in a propose-validate-refine loop.

Chapter 8. Sugar and Extensions

8.1 The role of sugar

Frame-QL is a small language: a query has a fixed clause structure, expressions follow the canonical form, and the sugars of Chapter 3 cover the common-case shortcuts. The agent and various surfaces may apply additional sugar layers — domain-specific shortcuts, natural-language-to-query expansion, embedded query fragments in other languages — that desugar to Frame-QL before reaching the framework.

This is by design. The formal language stays small and analyzable; surface sugars handle ergonomics. The framework type-checks the desugared Frame-QL, so any sugar that produces well-formed Frame-QL is admissible; sugar that produces ill-formed Frame-QL is rejected at the framework boundary, no matter how natural it looked at the surface.

8.2 Name aliases

A Manifold may declare operator aliases through its operator registry — total for sum, unique_visitors for count_distinct(visitor_id), etc. — that are valid in queries against that Manifold:

FROM retail_manifold
SELECT total(revenue), unique_visitors AT {store}

The aliases are Manifold-level (declared in the operator registry), not universal. The same alias may mean different things in different Manifolds, or be undeclared in some; the framework resolves aliases against the named Manifold’s registry.

Surfaces building on Frame-QL may apply patterns like:

The framework places no constraints on what surfaces do, as long as what they hand to the framework is well-formed Frame-QL.

Appendix A: Operator Reference

The registry is one umbrella, and it is the planner’s contract with the engine. Reducers, scans, and map functions are three kinds of operator in one installation-level registry. The registry holds, for each operator, its name, its kind, and its type signature; the planner reads exactly this to typecheck an operator against its inputs and to route it — a reducer is decomposed into an atom op(input @ a_in) @ a_out, a scan is handed to the column engine to run against a derived order, a map is evaluated by the planner over co-anchored results. The mechanics — how sum combines, how an HLL sketch merges, how rolling_mean walks a window — live in the engine, never in the registry. This is the same logical/physical split the type system uses: the registry is vocabulary (names, kinds, signatures, fertility, the {mechanism → behavior} map), the engine is implementation. Custom operators and custom datatypes (e.g. an HLLSketch-typed column with an approx_distinct fertile reducer) are added here, at the registry, which is why a new operator needs no change to the planner’s machinery — only a new registry entry the planner can read. [Pro] custom operators/types and the sketch reducers below beyond a basic set.

Reducers

ReducerTypeNotes
sumfertileextensive monoid; carrier is the value
count, count(*)fertilecounts present (non-missing) values
max, minfertileidempotent monoids; overlap-robust
productfertilewatch for overflow / zero
any, allfertilelogical monoids on Boolean
meanmulecomputed at presentation from sum and count
weighted_meanmulecomputed from weighted sum and weight sum
variance, stddevmulecomputed from count, sum, sum-of-squares
median (exact)muleno fertile carrier; recomputed from base at the output grain, not re-aggregable past it
modemuleno fertile carrier; recomputed from base at the output grain, not re-aggregable past it
approx_distinctfertile via HLLHLL sketch is fertile; estimate at presentation. [Pro] beyond Core’s basic HLL
approx_quantilefertile via t-digestt-digest sketch is fertile. [Pro]
approx_frequencyfertile via count-mincount-min sketch is fertile. [Pro]
lastordered, fertile-along-its-orderrequires an order; path-invariant along the order
firstordered, fertile-along-its-orderrequires an order; path-invariant along the order
value_at_max, value_at_minordered, muleneeds the full set

A note on last and first as family founders: an ordered reducer may appear in a column’s family set (a stock measure’s LAST family is the canonical case — the closing balance over time). Because order is never part of a cache key, what such a family caches is the un-ordered slice of its source; the ordered reduction is applied (or recomputed) at serve time against the declared order. The family is path-invariant along its order — last-of-lasts is the global last — which is exactly the property the serve-time application relies on.

Map functions

Arithmetic: +, -, *, /, %, unary -. Comparison: =, !=, <, <=, >, >=, between, in. Logical: and, or, not. Conditional: if(predicate, then, else), case ... when ... then ... else ... end. Null/missing: is_null, is_missing, coalesce. Numeric: log, exp, sqrt, abs, sign, ceil, floor, round. String: concat, substring, lower, upper, trim, length. Temporal: year, month, day, week, quarter, date_diff, date_add.

Scan functions

cumsum, cumprod, cummin, cummax — running aggregates (prefix of the fertile reducer). rolling_sum, rolling_mean, rolling_min, rolling_max, rolling_count — windowed (require a window). lag(col, n), lead(col, n) — shifted values. rank, dense_rank, row_number — ordinal positions. pct_change — relative change from previous. ewm_mean — exponentially-weighted mean.

Scan parameters, passed by keyword: window (rolling extent), n (shift offset), by (explicit order column), and the family-aware trio reset / within / step (Chapter 2.8), each naming a coarser level in the order axis’s dimension family.

Type predicates and casts

is_<type> — predicate for value type (e.g., is_integer, is_string). cast(col, <type>) — explicit type conversion.

Appendix B: Reserved Keywords

The shipped envelope statement keywords (columna-core 0.9.0; case-insensitive, whole-word): EXPLAIN, FROM, WITH, SELECT, AS, AT, WHERE, HAVING, ORDER, BY, LIMIT, PER, ASC, DESC, AND. These are the clauses of the envelope (Chapter 1.2), in fixed order; SELECT and AT are required, the rest optional (FROM defaults to the bound Manifold).

Structural markers. @ {…} — the input anchor, on a column reference or reduction, the input-anchor marker universally; AT {…} — the output grain, the sole output-anchor declaration; * — the anchor product (store * cal.month), the same operator as UNIVERSE u = store * day (comma accepted on input, * canonical); {…} — an anchor set (a product of levels). The trailing-@ output form is retired — a top-level @ no longer spells an output anchor (Appendix D).

ON is a DEFINITION-language clause, not a query keyword. A population is pinned in definitions; the §2c universe law resolves universe structurally at query time (Chapters 1.5, 2.5). An ON in a query is a syntax error.

Grow-by-ruling keywords — documented, not yet in the shipped envelope statement grammar (they enter by ruling, ADR-035 D1): VERSION (the FROM … VERSION n pin, Chapter 1.3, [SCHEDULED]); the scan-parameter keywords window, n, by, reset, within, step (recognized in scan-argument position, Chapter 2.8 [ROADMAP]); the bracket-filter [...] syntax on column references (Chapter 2.8 [ROADMAP]); and the expression-level OR, NOT, IF, CASE, WHEN, THEN, ELSE, END, BETWEEN, IN (series-internal expression text is captured verbatim and delegated to the expression parser at plan time, so its dialect grows independently of the envelope). WITH allocation is [Pro] (Chapter 4.5).

Reserved for future use: INHERIT, JOIN, COMPOSE, ASSERT, ACCESS.

Operator names from Appendix A are reserved when used in operator position; the same identifiers may be used as column names in Manifolds that declare them.

▸ shipped-law reconciliation (2026-07-17), RESOLVED. The prior edition of this appendix flagged FROM/SELECT/AT/{…} as “Coframe canonical form, not the shipped grammar,” pending the Coframe→envelope rewrite. That rewrite has landed: the envelope SELECT … AT {…} is the shipped grammar (Chapter 1), so these are the shipped query keywords, reconciled above. The retired terse @-fragment (its : label and trailing-@ output) moves to Appendix D.

Appendix C: For SQL Speakers — Five Intuitions That Transfer Wrong

Frame-QL deliberately borrows SQL’s surface so that the syntax costs a SQL-literate reader nothing. The price of familiarity is that a handful of SQL intuitions transfer incorrectly, and they are worth naming once, plainly.

1. There is no GROUP BY, because the output anchor is the grouping. AT {customer, month} declares the grain of the result; grouping is implied by the ascription, and the anchor’s dimensions appear automatically as the frame’s leading columns. SELECT lists only the value series — listing a dimension in SELECT is the error, not the habit.

2. WHERE is not one table scan; it applies per series, at each series’ own input grain. In a multi-series query whose series draw from different roots, the predicate restricts each series’ input independently, its dimensions reached through verified hierarchy edges (Chapter 4.1). A predicate dimension unreachable from some series’ input grain is a clarification, not a silent partial filter.

3. There are no joins, and nothing is missing. Combining columns means bringing them to a common anchor — aggregate one side, broadcast the other — and mapping at that anchor. The relationship a SQL writer would join on was declared once, as a verified edge in a dimension family, and compiled into the Manifold; the fan trap is not avoided by care but unrepresentable by construction.

4. SUM may not equal the sum of the stored rows — on purpose, and it says so. Under the defaults (extensive reducer, MCAR missingness), the framework serves skip-and-rescale: an unbiased estimate of the population total, with the rescale factor disclosed in the annotation. Reconciling against a source system’s raw SUM? Pin the family’s missing-policy to skip (or read the factor out of the finding) — the difference is the declared missingness, made visible instead of silently undercounted.

5. Aliases are not variables — reuse is WITH, not SELECT-list chaining. An AS alias names an output column after reduction; it is invisible in WHERE and to sibling series. The reuse SQL writers reach for is the macro binding (WITH name = expression, Chapter 4.5): textual, expanded before everything, valid wherever the expanded expression would be.

Appendix D: Lineage — the retired terse @-fragment

Before the envelope shipped, Columna’s public query surface was a fragment of the language: a single output column, its output anchor spelled by a trailing @.

aov @ cal.month                 # the fragment: "aov, output-anchored at cal.month"
revenue, orders @ region        # a short list, one shared trailing-@ output anchor
name: expr @ anchor             # optional `:` label on a column

This form shipped in the pre-launch demo, the What-is-Columna glossary, the site query strings, and early manual drafts. It is retired, and it is not kept as a sugar. The reason is the one collision the envelope exists to kill: the fragment spelled two different things with @ — the outer trailing @ meant the output anchor (aov @ cal.month), while the inner @ of an inline reduction meant the input anchor (avg(aov@day)). Under the envelope’s ruling, @ {…} is the input-anchor marker universally and AT {…} is the sole output-grain declaration, so a top-level aov @ cal.month would now read as “aov input-anchored at cal.month, output anchor unstated” — an underdetermined frame. Keeping the fragment as sugar would mean @ spelled the output anchor in one position and the input anchor in every other; the envelope removes the ambiguity by removing the fragment.

Migration is mechanical. Every terse statement rewrites to the envelope by moving the trailing anchor into AT {…} and any : label into AS:

Retired fragmentEnvelope
aov @ cal.monthSELECT aov AT {cal.month}
revenue, orders @ regionSELECT revenue, orders AT {region}
inv: level.last @ regionSELECT level.last AS inv AT {region}
avg(aov@day) @ cal.monthSELECT avg(aov @ {day}) AT {cal.month}

What the envelope absorbed from the fragment: the per-subexpression input anchor (@ {…}), the anchor product *, dotted member addressing (level.last), and juxtaposition of several readings at a shared anchor. What it declined: the trailing-@ output spelling and the : label — both replaced by the unambiguous AT {…} / AS. (This is the Coframe→envelope lineage note of ADR-035 D2: the ideas the shipped grammar kept, and the container ceremony it left behind.)

End of the Frame-QL Manual, First Edition (a renamed, lightly-revised continuation of the Coframe-QL Manual, Third Edition; reconciled to the shipped envelope grammar, ADR-035).