A query builder that returns the right answer in Rust and the wrong answer in Python is worse than no query builder at all, because the bug report will blame the database, the database will be fine, and the actual divergence will sit quietly in a translation layer that nobody thought to test. We wrote about the Kit’s architecture in July, one Rust core with three thin language surfaces, and that design removes most of the opportunity for drift, but it does not remove all of it, because the surfaces still translate errors into idiomatic exceptions, still normalize result shapes, and still marshal values across the N-API and PyO3 seams, and any one of those boundaries is a place where TypeScript can end up believing something Python does not.

So the architecture post was only half the story, and the other half is the conformance suite at tests/conformance/, which is the thing that actually proves the three surfaces agree instead of merely assuming they do because they share a core.

The fixture is the spec

The suite is built around a simple inversion: instead of writing tests in each language that happen to check the same behaviors, the behaviors live in language-neutral JSON under tests/conformance/fixtures/, and each language ships a small runner that loads the fixtures, executes them against a real database, and asserts the outcome. There are two dozen fixture files covering the boring places where SDKs actually diverge: schema definition, inserts with valid and invalid rows, updates, deletes through every foreign-key action, queries with filters and ordering and projection, migrations and migration failures, key encoding, aggregates, joins, subqueries, CTEs, views, stored procedures, and the nullable-unique-constraint semantics that have bitten every ORM I have ever used.

Every named scenario has an expected outcome, and the outcome is one of exactly two shapes: either the exact row that must come back, compared field by field, or a typed error code, because the interesting guarantee is not just that an invalid insert fails everywhere, it is that it fails with the same error everywhere.

{
  "insert_user_invalid_age": {
    "error": "VALIDATION"
  },
  "insert_post_missing_user": {
    "error": "FOREIGN_KEY"
  }
}

The error codes are stable identifiers like VALIDATION, FOREIGN_KEY, RESTRICT, and DUPLICATE, and each runner maps its language’s idiomatic exceptions back onto those codes before asserting, so a Python ForeignKeyError and a Rust error variant and a TypeScript thrown object all have to land on the same string, and a surface that invents its own error taxonomy fails the suite the day it ships.

Three runners, one verdict

The TypeScript runner is a vitest file that gets picked up by the normal npm test in packages/kit, the Rust runner is a workspace member called conformance-runner that also runs as a standalone binary, and the Python runner is a pytest module that drives the PyO3 binding, and all three open a real on-disk database in a temp directory per scenario, no mocks, no fakes, because a mock that agrees with itself proves nothing about the storage engine. The delete scenarios are my favorite part, because after a cascade or a set-null or a restrict, the runner does not just check that the operation returned the right thing, it reads back the full state of the users, posts, and comments tables and compares every remaining row against the fixture, which means a cascade that silently over-deletes or under-deletes in one language is caught by name, with the scenario and the table in the assertion message.

There is a small amount of per-runner honesty baked in, and it is worth admitting: the TypeScript runner normalizes the storage representation of NULL for nullable columns so the logical result matches what Rust and Python produce, and the Rust runner commits each mutating scenario before verifying state so the transactional isolation matches the other two, and these are exactly the kinds of seams I mentioned at the top, the places where a shared core still leaves room for a surface to see the world slightly differently, and the suite exists to make those differences visible and deliberate instead of accidental and discovered by a user.

Writing the test once

When we add a behavior, the workflow is: add the JSON fixture describing the input and the expected outcome, update the three runners if the fixture exercises a new operation shape, then run all three suites before committing. The fixture is the part that takes thought, because you are writing the specification of the behavior in its most compressed form, and the runner updates are usually mechanical, and the thing you never do is write three separate tests that each try to express the same expectation in three different assertion styles, because that is how you end up with a TypeScript test that asserts on the message text and a Python test that asserts on the exception class and a Rust test that asserts on nothing at all because someone got tired.

This is not a new idea, and I want to give credit where it is due, because CockroachDB built their whole correctness story on sqllogictest files that any engine can execute, and SQLite has spent decades proving the complementary half, that one heavily tested core wrapped thinly by every language is worth more than any number of per-language reimplementations, and the pattern is always the same: a test corpus that does not care what language runs it is worth more than three times as many tests written three times, because the corpus is the contract and the contract cannot drift against itself.

What it does not catch

The honest tradeoff is that conformance fixtures prove behavioral equivalence, not idiomatic goodness, and a surface can pass every fixture while still feeling awkward to write against, because the suite has no opinion about whether the Python API reads like Python. It also cannot catch bugs in the seams the fixtures do not model, and every fixture file is a deliberate choice about what matters, which means the suite is only as complete as the scenarios someone bothered to encode. What it does catch, reliably and cheaply, is the failure mode that actually ships: the same query, run through three languages, returning three different answers, and in twenty years of watching multi-language database tools drift apart, that is the failure that ends up in the bug tracker at the worst possible time, and now it ends up in CI instead.