Can each semantic layer express and run the same questions?

Output check: all 26 answers the layers executed match the answer key.

• Semantic Rails, q19 and q20 need a model change: rolling and prior_period are query-time expressions, but they run over a dense calendar, and this package declares no calendar entity. The engine refuses the query ("time.fill requires a calendar entity in the package"; the refused query and its validation errors are in the Semantic Rails results), so the package needs a calendar entity first. (documentation)
• MetricFlow, q17 needs a model change: the conversion window is part of the conversion metric's definition (window: 7 days). No query option changes it, so a 14-day window needs a new conversion metric. (documentation)
• MetricFlow, q18 needs a model change: the window and the same-store constant property are part of the conversion metric's definition. No query option changes the window, so a 50-minute window needs a new conversion metric. (documentation)
• MetricFlow, q19 needs a model change: a trailing window is a cumulative metric's window, set in its definition. The query interface has no window option, so it needs a new cumulative metric. (documentation)
• MetricFlow, q20 needs a model change: a prior-period value is a derived metric with an offset_window, set in its definition. The query interface has no offset option, so it needs a new derived metric. (documentation)
• MetricFlow, q21 needs a model change: a query's --where filters every metric in the query, so one query can't return total revenue beside the revenue of large orders; that needs a metric with its own filter:. (documentation)
• MetricFlow, q22 needs a model change: a simple metric's aggregation is part of its definition (agg: sum for item_revenue_usd). The query interface can't choose another aggregation, so the average and the maximum each need a new simple metric. (documentation)
• Cube, q17 needs a model change: the model's session-to-order join carries the 7-day window, and a query can't change a join's condition. The SQL API joins cubes only along the model's declared joins (__cubeJoinField or CROSS JOIN), so it can narrow that join (q18) but can't reach orders more than 7 days after a session. A 14-day window needs a new join in the model. (documentation)
• Cube, q18–q21 and q23 are workarounds: an SQL API query answers it by wrapping a Cube query in SQL of its own, a derived table: with SUM OVER for q19, LAG for q20, a filter on each customer-month's order count for q23, a CASE over each order's revenue (an ungrouped query of the revenue measure) for q21, and for q18 a 50-minute flag over the declared 7-day same-store join, weighted by each group's session count. (Cube's join excludes an order at the session's start minute, which the rule counts, and Cube post-processes q18 over a Cube query result it caps at 50,000 rows; no order in this data falls on the start minute, and that result holds only the 10 sessions' same-store orders within 7 days. It has no ORDER BY, because Cube refuses to post-process the sorted query.) The rubric counts SQL written around a layer's members as hand-written, as it does SQL outside SEMANTIC_VIEW(...), so the label is workaround rather than native. A REST query has no such option: its filters apply to the whole query, and compareDateRange returns each date range as a separate result set, not the prior month beside each month. In the model these would be a rolling_window or time_shift measure, a measure with filters, or a multi-stage measure with the new threshold. (documentation)
• KtX, q17 and q18 need a model change: the 7-day window is inside the pack's conversion SQL sources. ktx-sl joins are equality-only, and a semantic query has no join or window option, so a different window needs a new SQL source. (documentation)
• KtX, q19 and q20 need a model change: ktx-sl rejects window functions in measures, inline query measures included ("Window functions (OVER clause) are not supported in measures"; the refused queries are in the KtX results), so a trailing or prior-period value needs a SQL source. (documentation)
• KtX, q23 needs a model change: the more-than-10 threshold is inside the pack's customer-month SQL source. A query filter is row-level or applies at the query's own grain, so it can't count each customer-month's orders; a new threshold needs a new SQL source. (documentation)

• The label says how, not only whether. Malloy answers q17 and q18 by declaring a windowed join in the query itself, which is Malloy syntax rather than SQL, so it is native. Cube's SQL API answers to q18–q21 and q23 wrap a Cube query in SQL of its own, so they are workarounds, as SQL outside SEMANTIC_VIEW(...) is; its q22 query only aggregates a measure, so it is native. Malloy's and Cube's q19 and q20 step over month rows, which equals the calendar rule here only because every month has orders.
• Some variants don't discriminate on this data. q17 equals q09, q18's same-store condition removes nothing (every session and candidate order is at one store), and q22's maxima are each product type's top price in every month, so a layer returning the base metric would still match there. The labels don't depend on it: each layer's query for a variant is published with the pack.
• We chose the set, knowing which parameters Semantic Rails composes at query time and which some other layers set in the model. It probes where each layer's frozen-model boundary lies. It isn't a ranking, and it doesn't weigh what a model change costs in each layer.

The output checks, the question split, the rubric, this table, the scoreboards, the model-change reasons and the label tables are generated from the comparison pack's capability_matrix.json at engine commit 9059fba. The rest of the page is written by hand.

Output check: 7 of 7 match the answer key across the 5 layers checked on the current data. The stale Snowflake Semantic Views capture differs on q07.

Output check: 9 of 9 match the answer key across the 5 layers checked on the current data. The stale Snowflake Semantic Views capture differs on q16.

How each layer is modeled

• Semantic Rails: every question runs through the Semantic Rails runtime without helper views or hand-written SQL; q11 and q12 read the precomputed customer lifetime columns. Its q09 and q15 window runs from the session start up to, not including, 7 days later, where the rule counts orders after the session start, up to and including 7 days later; no order falls on either boundary.
• MetricFlow: validity-windowed semantic models, conversion metrics, and metric filters; q10, q13, and q14 group their metric filters by surrogate entities defined with expr. Its conversion attribution is last-touch, and its 7-day window runs from the session's minute up to, not including, 7 days later. Neither difference shows on this data: no customer has two sessions, and no order falls on a window boundary.
• Cube: declared joins carrying validity and event-window conditions, subquery dimensions, and multi-stage measures, and q05 as one multi-fact query. q11 counts a duplicate orders.order_count measure, because Cube 1.7.45's Tesseract planner renders invalid SQL when a query selects a measure and filters on a subquery dimension over that same measure.
• Malloy: arbitrary-condition joins and query-derived sources joined back to orders.
• Snowflake Semantic Views: q01–q07 through SEMANTIC_VIEW(...), and q08–q16 as SQL on the same tables; range joins haven't been modeled. Our q09 and q15 SQL counts orders from the session start up to, not including, 7 days later; its earlier dataset wasn't checked for orders on those boundaries.
• KtX: q01–q07 through its Python semantic layer (ktx-sl), and q08–q16 through SQL-backed sources or query-level filters; the ktx-sl bundled in @kaelio/ktx 0.16.0 has equality-only joins and no query-derived sources. The SQL sources are standalone per-question fact tables rather than bridge sources joined to orders, a known gap in our KtX model. Their q09 and q15 window runs from the session start up to, not including, 7 days later; no order falls on either boundary.

The outputs column checks the 5 layers on the current data against the answer key; on a frozen-model question, the layers that answered it.
† The stale Snowflake Semantic Views capture differs from the answer key on q07 and q16.

• Performance: latency, token use, and cost.
• Compiler-surface controls. Other layers have real advantages here, for example MetricFlow's metric-time-only planning, distinct-values planning, and duplicate-alias rejection.
• Warehouse coverage, maturity, BI integrations, and agent-workflow fit. The warehouse table below comes from each product's documentation, not from runs.
• Whether a model is idiomatic. The rubric checks where an answer's logic lives, not whether an expert in that layer would model it the same way.
• KtX's broader context product: this comparison runs its Python semantic layer (ktx-sl) only.