Transactions
An advanced page: what the transaction option does, and three ways to store data with and without one — plain files, a database, and a file journal written by hand. Read the server first.
What transaction is for
At the end of every turn, Twinstate hands the turn's writes to your store adapters: a create here, an update there, perhaps across several kinds, and then the record of which mutations it applied (in Twinstate's own records). Without a transaction those writes are made one after another. If the process dies between two of them, some are on disk and some are not.
Twinstate does not know what a transaction is, and never opens one. The transaction option is a function you write that answers one question: how do I run this batch of work inside one transaction of my store? Twinstate supplies the batch; your function supplies the transaction:
ts
transaction: (run: (dbTx: DbTx) => Promise<void>) => db.transaction(run),- When a turn ends, Twinstate calls your function and passes it
run: the turn's writes, not yet executed. - Your function hands
runto your store, which opens a transaction and callsrunwith its own handle — here calleddbTx. runcalls each adapter write with that handle as its last argument —create(id, stored, dbTx)— and saves the applied mutation ids through thetwinstateadapter with the same handle. So every write of the turn is part of one transaction.- Your store commits if
runresolved, and rolls back if a write threw — then Twinstate stops (fail-stop) and a restart reloads from disk.
Two handles, two names. In a handler, tx is always Twinstate's turn handle (tx.update, tx.create). In an adapter, dbTx is always your store's transaction handle (dbTx.insert, dbTx.write). They never meet: a handler never sees dbTx, and an adapter never sees tx. The type of dbTx is written once, on run's parameter, and inferred into every adapter from there; an adapter expecting anything else does not compile.
Plain files, no transaction
The simplest store: leave transaction out. Each adapter writes its own file directly — atomically, one file at a time — and gets no third argument.
ts
import { flat, twinstateFiles } from 'twinstate/server';
createTwinServer(twin, {
// no `transaction`: adapters get no dbTx
store: {
conversation: {
load: async (id) => flat.read(await readJsonIfExists(metaPath(id))), // undefined → absent
list: () => listConversationIds(), // the folders under conversations/ that hold a meta.json
create: (id, stored) => writeFileAtomic(metaPath(id), flat.write(stored)),
update: (id, stored) => writeFileAtomic(metaPath(id), flat.write(stored)),
// the whole folder: its composer and transcript go with it, which is this store's layout
remove: (id) => removeConversationFolder(id),
},
composer: {
// defaulted: a missing file means "still empty", never absent
load: async (id) => {
const json = await readJsonIfExists(composerPath(id));
return json && flat.read(json);
},
// ops ['update']: its first update writes the file (an upsert); no remove — it goes with
// its conversation's folder
update: (id, stored) => writeFileAtomic(composerPath(id), flat.write(stored)),
},
message: {
// a child of the transcript, ops: ['create'] — written once, by appending
load: (id) => readMessage(id), // the id is `<conversation>.<seq>`
children: (conversationId) => readMessageIds(conversationId),
window: (q) => readMessagePage(q), // the latest page first
create: (id, stored) => appendLine(transcriptPath(parentOf(id)), stored),
// no remove: messages only ever go with their conversation, whose folder holds the transcript
// (a torn last line is not a record — its turn never committed — and the file is trimmed on open)
},
twinstate: twinstateFiles(join(DATA_DIR, 'twinstate')), // a small file per record
},
handlers: {/* … */},
});
// the one helper every whole-file write goes through
async function writeFileAtomic(path: string, data: unknown) {
await writeDurably(`${path}.tmp`, JSON.stringify(data)); // write and fsync the new version
await rename(`${path}.tmp`, path); // swap it in: a reader sees old or new, never half
}- Each whole-file write is atomic: no reader and no crash ever sees half a
meta.json. Appends are not: a crash mid-append can leave a partial last line intranscript.jsonl. So the append helper first cuts the file back to its last newline — dropping any torn fragment — and then appends a whole line; the reader, too, skips any line that does not parse. The lost append is redone when the turn's mutation is resent. - A turn is not. A turn that writes two files can be cut between them. That case is repaired, not prevented: the server stops, restarts and reloads from disk; the mutation got no reply, so the client resends it, and the handler — safe to re-run over its own partial result — finishes the job.
- Most turns write one file, and those are fully atomic anyway. Declaring a per-parent kind
defaulted(a composer that exists for every conversation) is one way to keep it so: creating a conversation writes only its own file. - A client's record of applied mutations is a small file, written after the turn's data. A crash between the two lets one resend run again — harmless, except that a create can bring back a record another device removed before the resend arrived (see the server). This is the one residual case files without a transaction leave.
A database
With a database that has transactions, one line makes every turn atomic. Here it is with Drizzle over SQLite through an async driver (libSQL, say), storing each record's { value, meta } as one JSON document. (A driver whose transactions are synchronous, such as better-sqlite3's, cannot run an async run, so it does not fit this option as written.) Every write is awaited, so it returns Promise<void> as the adapter's type requires:
ts
createTwinServer(twin, {
transaction: (run: (dbTx: DrizzleTx) => Promise<void>) => db.transaction(run),
store: {
note: {
load: async (id) => (await db.select().from(notes).where(eq(notes.id, id)))[0]?.doc,
list: async () => (await db.select({ id: notes.id }).from(notes)).map((r) => r.id),
create: async (id, stored, dbTx) => {
await dbTx.insert(notes).values({ id, doc: stored });
},
update: async (id, stored, dbTx) => {
await dbTx.update(notes).set({ doc: stored }).where(eq(notes.id, id));
},
remove: async (id, dbTx) => {
// Twinstate calls only the parent's remove: the note_bodies row goes by the
// database's ON DELETE CASCADE on its note_id foreign key
await dbTx.delete(notes).where(eq(notes.id, id));
},
},
// …
twinstate: {
// Twinstate's own records, one row each: a mutation rewrites only its own client's row
load: async (id) =>
(await db.select().from(twinstateTable).where(eq(twinstateTable.id, id)))[0]?.value,
save: async (id, value, dbTx) => {
await dbTx
.insert(twinstateTable)
.values({ id, value })
.onConflictDoUpdate({ target: twinstateTable.id, set: { value } });
},
remove: async (id, dbTx) => {
await dbTx.delete(twinstateTable).where(eq(twinstateTable.id, id));
},
list: async () =>
(await db.select({ id: twinstateTable.id }).from(twinstateTable)).map((r) => r.id),
},
},
handlers: {
createNote: async ({ id, title }, tx) => {
// tx: Twinstate's turn handle — both creates land in one database transaction
await tx.create(note, id, { title, updatedAt: now(), pinned: false });
await tx.create(noteBody, id, { text: '', images: [], related: null });
},
},
});Reads (load, list) use the database directly: Twinstate reads outside transactions, and only its writes are batched. A turn, its mutation ids included, now commits whole or not at all, so the residual case above does not exist.
A file journal, by hand
Files can have transactions too — it is what a database does under the hood. A redo journal: write every new version beside its original, then write a journal naming the swaps (the commit point), then perform them. On boot, a journal that exists means "finish this"; none means "nothing to finish".
ts
import { open, rename, rm, readFile, mkdir } from 'node:fs/promises';
import { dirname, join } from 'node:path';
// what adapters receive as dbTx: writes are only staged here
type FileTx = {
write(path: string, data: unknown): void;
remove(path: string): void; // a file or a whole directory
};
const JOURNAL = join(DATA_DIR, 'turn.journal');
async function writeDurably(path: string, text: string) {
await mkdir(dirname(path), { recursive: true });
const fh = await open(path, 'w');
await fh.writeFile(text);
await fh.sync(); // on disk, not just in the OS cache
await fh.close();
}
async function fileTransaction(run: (dbTx: FileTx) => Promise<void>) {
const writes = new Map<string, string>();
const removes = new Set<string>();
// 1. Twinstate runs the turn's writes; they only collect here
await run({
write: (path, data) => void writes.set(path, JSON.stringify(data)),
remove: (path) => void removes.add(path),
});
// 2. every new version beside its original — nothing real has changed yet
for (const [path, text] of writes) await writeDurably(`${path}.tmp`, text);
// 3. the commit point: once the journal is on disk, the turn has happened
await writeDurably(
JOURNAL,
JSON.stringify({ renames: [...writes.keys()], removes: [...removes] }),
);
// 4. carry it out
await applyJournal();
}
// also run at boot, before the server loads anything
async function applyJournal() {
let journal: { renames: string[]; removes: string[] };
try {
journal = JSON.parse(await readFile(JOURNAL, 'utf8'));
} catch {
return; // no journal: nothing committed was left unfinished
}
for (const path of journal.renames) {
// redoing after a crash: a .tmp already gone was already renamed
await rename(`${path}.tmp`, path).catch((e) => {
if (e.code !== 'ENOENT') throw e;
});
}
for (const path of journal.removes) await rm(path, { recursive: true, force: true });
await rm(JOURNAL); // done
}Wiring it in — the same adapters as for plain files, writing through dbTx:
ts
await applyJournal(); // crashed after the commit point → finish that turn
await removeStrayTmpFiles(); // crashed before it → the originals were never touched
createTwinServer(twin, {
transaction: fileTransaction,
store: {
conversation: {
load: async (id) => flat.read(await readJsonIfExists(metaPath(id))),
create: (id, stored, dbTx) => dbTx.write(metaPath(id), flat.write(stored)),
update: (id, stored, dbTx) => dbTx.write(metaPath(id), flat.write(stored)),
remove: (id, dbTx) => dbTx.remove(conversationDir(id)), // the whole folder, as with plain files
},
// …
twinstate: {
// ids always fit the id grammar, so they are safe in a path
load: (id) => readJsonIfExists(twinstatePath(id)), // a small file per record
save: (id, value, dbTx) => dbTx.write(twinstatePath(id), value), // in the same turn: atomic with the data
remove: (id, dbTx) => dbTx.remove(twinstatePath(id)),
list: () => listFileNames(TWINSTATE_DIR),
},
},
// …
});Why it holds at every crash point:
| the process dies | on restart |
|---|---|
| during steps 1–2 | no journal: the stray .tmp files are deleted and the originals are untouched — as if the turn never ran. The client got no reply, so it resends |
| after step 3 (journal written) | applyJournal finishes the renames: the turn is fully on disk, its mutation ids included, so the resend is answered ok without running again |
| during step 4 (some renames done) | the same — renames are redone, and those already done are skipped |
One turn runs at a time (the queue), so one journal is enough; a rename is atomic on one filesystem, so a reader sees a file's old version or its new one, never half of each.
What the snippet glosses over:
- Directory syncs. For a rename to survive a power cut, the directory itself must be flushed too (open it and
fsyncit). Andfsyncis only as durable as the system and drive behind it: some do not force the drive's own cache to disk onfsync(macOS needsF_FULLFSYNC, which Node does not expose; some drives have a volatile write cache), so losing power at the wrong instant can lose the last turn. A crash or a kill is fine. - Appends. An append-only file (a transcript) is not rewritten through
.tmp: the journal records{ append: path, expectedLength, bytes }, and recovery checks the file's length — truncate back toexpectedLengthand re-append, or skip if the bytes are already there. That is the fiddly part. - New directories. A turn that creates a record may create its folder;
removeStrayTmpFilesmust also remove folders left holding only.tmpfiles. - Tests. Crash injection — killing the process at every step and checking that a restart always yields all or nothing — is what makes it trustworthy, not the code. It is roughly sixty lines plus those edge cases, which is why it is on the roadmap as an optional
fileTransaction()utility rather than something every app writes.
Which to use
| store | atomic per | what a crash can leave | use it when |
|---|---|---|---|
| plain files | file | a turn cut between files — repaired by the resend; one create residual | most turns write one file, and idempotent handlers are the rule |
| a database | turn | nothing | you already have one, or turns routinely write several records |
| files with a journal | turn | nothing (short of power loss where fsync does not reach the disk) | you want files and whole turns, and will test crash recovery |