Every database you have ever used comes with a client library, and that library is a toll booth between you and your own data, because the moment the bytes live inside a .db3 or behind a wire protocol, the only way to read them is the driver’s way, at the driver’s pace, with the driver’s idea of what a query is. We got tired of paying that toll on data that was fundamentally just text, so grexa-db inverts the arrangement: records are plain Markdown files with YAML frontmatter sitting in a directory tree, and the relational layer materializes on top of them as directories of symlinks, which means rg, grep, find, your editor, your file manager, and git are all database clients already, and none of them know the database exists.

This is not a new idea dressed up; it is the oldest idea in computing taken seriously. In the mid-nineties the answer to “how do I share this data with the accounting department” was “export it to CSV and put it on the share,” and the reason that worked is the same reason grexa-db works, which is that a plain file imposes no reader and no writer and no version negotiation, it just sits there being bytes until something opens it. The modern equivalent of that shared-drive CSV is a grexa-db collection synced with syncthing or versioned with git, except now the directory tree carries a typed schema, a query builder, and materialized joins, and the replication story is still “copy the files,” because replication was never the hard part once you refuse to own the write path.

Records are files, and the path is the identity

A grexa-db database is any directory, a collection is any subdirectory containing a schema.md, and a record is any other file in that directory, so schema discovery is implicit and cp -r remains a perfectly valid way to clone a database, which is a sentence you cannot say about Postgres. The schema itself is YAML frontmatter at the top of schema.md with free-form human documentation below it, fields typed as string, integer, float, boolean, date, array<T>, enum<a|b|c>, or ref<collection>, and the record files use the same frontmatter convention, so the format you query is the format you would have written by hand anyway.

The decision that shapes everything else is that the path is the record identity, full stop, there is no surrogate id column hiding in the frontmatter. Renaming a record file is therefore delete-plus-insert, not an update, and references are deliberately weak: a ref<bookmarks> value is just a DB-root-relative path like bookmarks/rust-lang.org.md, validated for shape and containment (no absolute paths, no .. segments, no symlink escapes past the root) but never enforced, so a dangling ref is a normal observable state that the engine reports rather than repairs. That sounds like a weakness until you remember who the writer is, because grexa-db’s engine is read-only by design; your editor is the write path, the engine cannot rewrite your records to fix a ref any more than grep can, and integrity workflows like rename-with-ref-repair are caller operations, not engine magic.

The relational part shows up in views/, where a materialized query becomes a directory of symlinks grouped by whatever field you grouped on, so “notes by tag” is a real folder you can open in a file manager, cd through in a shell, or hand to any tool that walks directories. The API side is a typed Rust builder, deliberately not SQL:

let db = grexa_db::Db::open("my-db")?;
let notes = db.collection("notes")?;

// Streaming filter: O(1) memory, record content read lazily
for record in notes.query().filter("rating").ge(4)? {
    let r = record?;
    println!("{}", r.field("title")?.as_str().unwrap());
}

// Materialize a view: one subdirectory per tag, symlinks back to records
db.materialize_view(
    "notes-by-tag",
    notes.query().filter("rating").ge(4)?,
    Some("tags"),
)?;

The publishing mechanism is the piece I am most fond of, because it is the kind of bug you only find by shipping: the naive design builds the new view in a temp directory and renames it over the old one, but rename(2) over a non-empty directory fails with ENOTEMPTY on Linux, so the shipped design builds each view in a generation directory under views/.generations/gen-<random>/, points a temp symlink at it, and renames the symlink over the published name, which is atomic on Linux and macOS because rename(2) swaps one directory entry for another atomically, and a symlink is a single directory entry. Readers see either the complete old generation or the complete new one, never a half-built view, stale generations get garbage-collected once nothing references them, and staleness is the documented default rather than an accident, since symlinks do not auto-update and re-materialization is a caller decision, which is exactly the honesty you want from a cache that happens to be a directory tree.

Why not SQLite, honestly

The obvious alternative was SQLite, and we wrote about that comparison in more detail in the filesystem-as-a-database post, but the short version is that SQLite is the right answer for ad-hoc relational reads and the wrong answer for human-legible data, because a .db3 file is the database in a literal sense while being the data in no usable sense: you cannot open it in an editor, you cannot grep it, you cannot diff two snapshots without the sqlite3 CLI, and even backing it up is not the rsync operation people assume, since copying a live database can tear a page mid-write and the safe paths are .dump, VACUUM INTO, or the online backup API. grexa-db’s bet pays off when the access pattern is relationship-first traversal by humans and shell tools, when writes are rare and arrive through an editor, and when the team would rather version a directory than run a migration.

The trade is just as honest in the other direction, because the moment you need a filtered aggregate across every record you are back to a full directory scan, and the moment you need multi-column indexing you have reinvented a database badly, and the spec says so in its own non-goals: no high write throughput, no multi-writer concurrency control, no network layer, no SQL parser, and a documented comfort zone that ends around a quarter million records per collection before filesystems get cranky and a real database wins.

The measured version of the ceiling

We did not leave that ceiling as a vibes-based warning, because the scaling R&D doc profiles a 200k-record collection on a tmpfs (so the numbers are a conservative floor; real disk makes the I/O wins larger) and finds the time goes exactly where you would hope it does not: the directory walk is 76 ms, only 9 percent, while reading and parsing records is the other 91 percent at roughly 4 microseconds per record, mostly serde_yaml building a full Value for fields the query never touches. The shipped fixes stack on each other without changing the on-disk format or the public API: a hand-rolled frontmatter fast-path that parses only the fields a filter references (2.37x on full parses, 5.5x on single-field filters, differential-tested against serde for identical output), a std::thread work-stealing parallel scan worth 2 to 5x end-to-end, a bounded top-K so order_by with a .limit() holds O(k) memory instead of buffering every match, and a caller-held in-memory index that a long-lived process loads once and keeps fresh with filesystem notifications and reconciliation (Linux gets inotify; everywhere else the index is a rebuildable sidecar that reconciles on demand), which is worth 297x on selective queries and turns a roughly 190 ms lookup into roughly 0.6 ms while every record remains a plain file you can open in Vim.

The honest part of that doc is the failure it kept, because the first index design auto-loaded a 30 MB JSON sidecar plus a freshness stat per record on every query, which made a cold CLI query slower than the parallel scan it was supposed to replace (523 ms against 192 ms), so it was reverted and cold one-shot queries deliberately scan instead. The ceiling that does not move is inode amplification, around 18x disk usage and 270x delete cost at half a million records, because transparency charges rent in inodes, and that rent is the reason grexa-db widens the sweet spot instead of pretending there is no ceiling at all.

When the filesystem is the schema

The rule we ended up with is the same rule the mid-nineties CSV export followed, restated with better tooling: if the humans and the shell are the primary readers, put the data where they already are, which is the filesystem, and let the database layer be a lens over it rather than a vault around it. If your reads are ad-hoc predicates you cannot predict, reach for SQLite, and if your reads are column sweeps over immutable history, reach for Parquet, and if none of those fit, that is what server databases are for; grexa-db owns exactly one corner, records as files and joins as symlink directories, and it is Apache-2.0 specifically so that corner stays embeddable anywhere, including the proprietary apps, because a storage idea this simple only earns its keep if it can go everywhere the files can.