diff --git a/DECISIONS.md b/DECISIONS.md index 4a3263e..3502c4d 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -1352,3 +1352,11 @@ Implementation decisions not spelled out in `memory-bank/implementation-plan.md` - **Version 1 needs no migration**: it is the first local format and the CR allows resetting dev data. The CR's "identity migration for version 1" is covered by the framework itself (a version-1 save passes through unchanged and validates); the chaining is tested with made-up versions 1 → 2 → 3. - **The stored contract, dispatch record and resolution are checked against the sim's types at compile time**, so a sim change that the save doesn't follow fails the build instead of a player's load. - **`migrateSave` never throws**: unknown shapes, missing or future versions and invalid fields all come back as a readable message (Zod's pretty error for field problems), for the storage fallback (14.3) and import (14.10). + +## Step 14.3 + +- **Rotation and fallback are written once, over text slots**: each backend only reads, writes and removes whole slots atomically. A save is a few slot writes (backups shift, then the new main), each all-or-nothing, so an interruption at any point leaves a loadable main save, at worst with a duplicated backup. +- **Backups are the previous saves, not snapshots in time**: with a save after every command, the three backups are the last three states before the current one. That's what recovers from a corrupted write; it isn't a history feature. +- **Platform objects are handed in** (`IDBFactory`, the Capacitor Filesystem plugin through a five-method interface), so the engine has no browser or Capacitor imports and the backends are tested with `fake-indexeddb` and a fake plugin. The real plugin is wired up in the web app (14.7). +- **The engine's TypeScript config adds the DOM library** for the IndexedDB and Web Locks types; it still only touches them through what it's given. +- **Pruning old resolved dispatches belongs to the save document**, not storage; it happens when a command saves (14.4). diff --git a/memory-bank/architecture.md b/memory-bank/architecture.md index f52ef8a..589f631 100644 --- a/memory-bank/architecture.md +++ b/memory-bank/architecture.md @@ -80,6 +80,8 @@ The `routes` catalog is the single source for both server and client. Each entry One document per company on the device, validated on every load and save: `saveVersion` (currently 1), `saveId`, `lastTimeSeen` (the clock never goes below it), `player` (world seed, tutorial state, deal accepted), `company` (name, credits, loan), `ships` (with `upgrades` as a level map and shop fields), `pilots`, `bays`, `discoveredLocations`, `contractBoards` (stored boards), `dispatches` (sim record, status, resolution, acknowledged), `acceptedContracts`, `claimedGoals`. It mirrors the database tables below without player ids; the stored contract, dispatch record and resolution are type-checked against the sim types. `migrateSave(raw)` steps an older save through `SAVE_MIGRATIONS` (`applyMigrations`) and validates it, returning a readable error instead of throwing. +Storage (`packages/engine/src/storage.ts`): `SaveStorage` keeps the save in slot `main` and the previous three in `backup-1..3` (rotated on every save), falling back to the newest valid backup if the main save is unreadable. Backends store text per slot, all-or-nothing: `MemoryBackend` (tests), `IndexedDbBackend` (web, database `deadhead`, store `saves`), `FilesystemBackend` (native, `deadhead/.json` in the Capacitor data directory, temp file then rename). + ## Database (`apps/server/src/db`) Postgres via Drizzle (`pg` driver). Migrations are SQL files in `apps/server/drizzle/`, generated with `pnpm --filter @deadhead/server db:generate` and applied with `db:migrate` (uses `DATABASE_URL`; rerunning is a no-op). Contracts are JSONB snapshots (`contract_boards.contracts`, `dispatches.record`); `0005_cr007_goods` backfills pre-CR-007 contracts with a default good of their class, the origin's first shipper and no Priority. Every game table hangs off `players.id`; times are bigint Unix ms. diff --git a/memory-bank/progress.md b/memory-bank/progress.md index 284ebd6..b06358f 100644 --- a/memory-bank/progress.md +++ b/memory-bank/progress.md @@ -343,7 +343,7 @@ Spec: `memory-bank/change-requests/CR-014-local-first.md` (received as "CR-011 - **Not started:** the Fixed Map and Fuel Tanks request (recorded after this phase as CR-015). - [x] **Step 14.1 — Engine package and clock.** New `packages/engine` (depends on content, schema, sim; picked up as a Vitest project). `Clock` interface (`now()`) and `DeviceClock`: device time (injected, `Date.now` by default) plus a developer offset, clamped to the last time it handed out; `lastTimeSeen` for the save, `restore(lastTimeSeen)` after loading one, `advance(ms)` for the developer tools (forward only). Tests (`packages/engine/test/clock.test.ts`): offset applied; device clock set back returns the last time seen; a forward move shows at once; a save's last time seen wins over a device that is behind; the offset can't go backward. - [x] **Step 14.2 — Save document schema and migrations.** `packages/schema/src/save.ts`: the save document (every database table for one company, without player ids, plus `saveVersion`, `saveId`, `lastTimeSeen`), with compile-time checks that its contract, dispatch record and resolution match the sim types. Migration framework: `SAVE_MIGRATIONS` (ordered version-to-version steps; none yet, version 1 is the first format), `applyMigrations` (steps a save up one version at a time), `migrateSave` (migrate, then validate; never throws, gives a readable error). Tests (`packages/schema/test/save.test.ts`, fixture `save-fixture.ts` reused later): a fully populated save validates; a wrong field is rejected naming `company.credits`; non-saves and saves from a newer build are refused; a version-1 save round-trips through JSON and migration unchanged; migrations chain in order and a missing step is reported. -- [ ] Step 14.3 — Storage layer +- [x] **Step 14.3 — Storage layer.** `packages/engine/src/storage.ts`: `SaveStorage` (load, save with rotation, list backups, load a backup, clear) over a `SaveBackend` that stores text in slots `main`, `backup-1..3`, each write all-or-nothing. Saving moves the previous main save to backup 1 (and 1→2, 2→3), so the last three saves are kept. Loading: no main save → empty; an unreadable or invalid one → the newest backup that validates, reported as `restored-from-backup` with the problem; nothing valid → `unreadable`. Backends: `MemoryBackend`, `IndexedDbBackend` (one object store, handed an `IDBFactory`), `FilesystemBackend` (one JSON file per slot in the app data directory, written to a `.tmp` file then renamed; handed the Capacitor Filesystem plugin through a small interface). Tests (`packages/engine/test/storage.test.ts`): one suite run on all three (IndexedDB via `fake-indexeddb`, the filesystem via a fake plugin): empty, round trip, exactly three backups newest first, corrupted main save falls back past a broken backup to the next valid one, unreadable, clear; and a native write interrupted mid-file or before the rename leaves the previous save loading. - [ ] Step 14.4 — Mutation queue and tab lock - [ ] Step 14.5 — Port the game logic - [ ] Step 14.6 — Resolution loop diff --git a/packages/engine/package.json b/packages/engine/package.json index 5824abb..e1cbd7a 100644 --- a/packages/engine/package.json +++ b/packages/engine/package.json @@ -13,5 +13,8 @@ "@deadhead/schema": "workspace:*", "@deadhead/sim": "workspace:*", "zod": "^4.6.5" + }, + "devDependencies": { + "fake-indexeddb": "^6.2.5" } } diff --git a/packages/engine/src/index.ts b/packages/engine/src/index.ts index 0432901..654ad8a 100644 --- a/packages/engine/src/index.ts +++ b/packages/engine/src/index.ts @@ -1,3 +1,15 @@ // The game engine (CR-014): the game's commands running on the device, using // storage, the clock and notifications only through injected interfaces. export { DeviceClock, type Clock } from "./clock"; +export { + BACKUP_SLOTS, + FilesystemBackend, + IndexedDbBackend, + MemoryBackend, + SaveStorage, + type BackupInfo, + type FilesystemPlugin, + type LoadResult, + type SaveBackend, + type Slot, +} from "./storage"; diff --git a/packages/engine/src/storage.ts b/packages/engine/src/storage.ts new file mode 100644 index 0000000..90afa36 --- /dev/null +++ b/packages/engine/src/storage.ts @@ -0,0 +1,260 @@ +import { migrateSave, type SaveDocument } from "@deadhead/schema"; + +/** + * Where a save lives (CR-014 §2): a main slot plus three rotating backups. + * Each backend only stores text in named slots, writing a slot all-or-nothing; + * `SaveStorage` does the rotation, validation and fallback once for all of them. + */ +export type Slot = "main" | "backup-1" | "backup-2" | "backup-3"; +export const BACKUP_SLOTS = ["backup-1", "backup-2", "backup-3"] as const; + +export interface SaveBackend { + /** For debug info: "memory", "indexeddb" or "filesystem". */ + readonly kind: string; + read(slot: Slot): Promise; + /** Replaces the slot's text completely, or leaves it as it was. */ + write(slot: Slot, text: string): Promise; + remove(slot: Slot): Promise; +} + +export type LoadResult = + | { status: "empty" } + | { status: "loaded"; save: SaveDocument } + /** The main save was unreadable; the newest valid backup was loaded instead. */ + | { status: "restored-from-backup"; save: SaveDocument; backup: number; problem: string } + /** Nothing readable: the main save and every backup failed. */ + | { status: "unreadable"; problem: string }; + +export interface BackupInfo { + /** 1 is the newest. */ + index: number; + lastTimeSeen: number | null; + valid: boolean; +} + +/** Text to a save, or the reason it isn't one. */ +function parse(text: string) { + let raw: unknown; + try { + raw = JSON.parse(text); + } catch { + return { ok: false as const, error: "The save isn't valid JSON." }; + } + return migrateSave(raw); +} + +export class SaveStorage { + constructor(private readonly backend: SaveBackend) {} + + get kind(): string { + return this.backend.kind; + } + + /** + * The current save. Missing main save: empty (a first launch). Unreadable + * main save: the newest backup that validates, with the problem reported. + */ + async load(): Promise { + const text = await this.backend.read("main"); + if (text === null) return { status: "empty" }; + const main = parse(text); + if (main.ok) return { status: "loaded", save: main.save }; + for (const [i, slot] of BACKUP_SLOTS.entries()) { + const backup = await this.backend.read(slot); + if (backup === null) continue; + const parsed = parse(backup); + if (parsed.ok) { + return { + status: "restored-from-backup", + save: parsed.save, + backup: i + 1, + problem: main.error, + }; + } + } + return { status: "unreadable", problem: main.error }; + } + + /** Writes the save; the previous main save becomes backup 1, and so on, keeping three. */ + async save(save: SaveDocument): Promise { + const previous = await this.backend.read("main"); + if (previous !== null) { + for (let i = BACKUP_SLOTS.length - 1; i > 0; i--) { + const older = await this.backend.read(BACKUP_SLOTS[i - 1]!); + if (older !== null) await this.backend.write(BACKUP_SLOTS[i]!, older); + } + await this.backend.write("backup-1", previous); + } + await this.backend.write("main", JSON.stringify(save)); + } + + async listBackups(): Promise { + const found: BackupInfo[] = []; + for (const [i, slot] of BACKUP_SLOTS.entries()) { + const text = await this.backend.read(slot); + if (text === null) continue; + const parsed = parse(text); + found.push({ + index: i + 1, + valid: parsed.ok, + lastTimeSeen: parsed.ok ? parsed.save.lastTimeSeen : null, + }); + } + return found; + } + + async loadBackup(index: number) { + const slot = BACKUP_SLOTS[index - 1]; + if (!slot) return { ok: false as const, error: `There is no backup ${index}.` }; + const text = await this.backend.read(slot); + return text === null ? { ok: false as const, error: `Backup ${index} is empty.` } : parse(text); + } + + /** Deletes the save and its backups (a new company or an import starts clean). */ + async clear(): Promise { + for (const slot of ["main", ...BACKUP_SLOTS] as const) await this.backend.remove(slot); + } +} + +/** For tests and as a fallback when the device offers no storage. */ +export class MemoryBackend implements SaveBackend { + readonly kind = "memory"; + readonly slots = new Map(); + async read(slot: Slot) { + return this.slots.get(slot) ?? null; + } + async write(slot: Slot, text: string) { + this.slots.set(slot, text); + } + async remove(slot: Slot) { + this.slots.delete(slot); + } +} + +const DB_NAME = "deadhead"; +const STORE = "saves"; + +/** Web storage: one IndexedDB object store, one record per slot (each put is atomic). */ +export class IndexedDbBackend implements SaveBackend { + readonly kind = "indexeddb"; + #db: Promise | null = null; + + constructor(private readonly factory: IDBFactory) {} + + #open(): Promise { + this.#db ??= new Promise((resolve, reject) => { + const request = this.factory.open(DB_NAME, 1); + request.onupgradeneeded = () => request.result.createObjectStore(STORE); + request.onsuccess = () => resolve(request.result); + request.onerror = () => reject(request.error); + }); + return this.#db; + } + + async #run( + mode: IDBTransactionMode, + op: (store: IDBObjectStore) => IDBRequest, + ): Promise { + const db = await this.#open(); + return new Promise((resolve, reject) => { + const tx = db.transaction(STORE, mode); + const request = op(tx.objectStore(STORE)); + tx.oncomplete = () => resolve(request.result); + tx.onerror = () => reject(tx.error); + tx.onabort = () => reject(tx.error); + }); + } + + async read(slot: Slot) { + const value = await this.#run("readonly", (s) => s.get(slot) as IDBRequest); + return value ?? null; + } + async write(slot: Slot, text: string) { + await this.#run("readwrite", (s) => s.put(text, slot)); + } + async remove(slot: Slot) { + await this.#run("readwrite", (s) => s.delete(slot)); + } +} + +/** + * The parts of the Capacitor Filesystem plugin this uses (`@capacitor/filesystem`), + * so the engine doesn't depend on Capacitor and tests can pass a fake. + */ +export interface FilesystemPlugin { + readFile(options: { + path: string; + directory: "DATA"; + encoding: "utf8"; + }): Promise<{ data: string | Blob }>; + writeFile(options: { + path: string; + data: string; + directory: "DATA"; + encoding: "utf8"; + recursive?: boolean; + }): Promise; + rename(options: { + from: string; + to: string; + directory: "DATA"; + toDirectory: "DATA"; + }): Promise; + deleteFile(options: { path: string; directory: "DATA" }): Promise; +} + +/** + * Native storage: one JSON file per slot in the app's data directory (included + * in iCloud and Android backups). A write goes to a temporary file that is then + * renamed over the slot, so the slot is never half-written. + */ +export class FilesystemBackend implements SaveBackend { + readonly kind = "filesystem"; + constructor( + private readonly fs: FilesystemPlugin, + private readonly folder = "deadhead", + ) {} + + #path(name: string) { + return `${this.folder}/${name}.json`; + } + + async read(slot: Slot) { + try { + const { data } = await this.fs.readFile({ + path: this.#path(slot), + directory: "DATA", + encoding: "utf8", + }); + return typeof data === "string" ? data : await data.text(); + } catch { + // Capacitor rejects for a missing file. + return null; + } + } + + async write(slot: Slot, text: string) { + const temp = this.#path(`${slot}.tmp`); + await this.fs.writeFile({ + path: temp, + data: text, + directory: "DATA", + encoding: "utf8", + recursive: true, + }); + await this.fs.rename({ + from: temp, + to: this.#path(slot), + directory: "DATA", + toDirectory: "DATA", + }); + } + + async remove(slot: Slot) { + try { + await this.fs.deleteFile({ path: this.#path(slot), directory: "DATA" }); + } catch { + // Already gone. + } + } +} diff --git a/packages/engine/test/storage.test.ts b/packages/engine/test/storage.test.ts new file mode 100644 index 0000000..8c27578 --- /dev/null +++ b/packages/engine/test/storage.test.ts @@ -0,0 +1,125 @@ +import { IDBFactory } from "fake-indexeddb"; +import type { SaveDocument } from "@deadhead/schema"; +import { describe, expect, it } from "vitest"; +import { fullSave } from "../../schema/test/save-fixture"; +import { + FilesystemBackend, + IndexedDbBackend, + MemoryBackend, + SaveStorage, + type FilesystemPlugin, + type SaveBackend, +} from "../src/index"; + +/** A Capacitor Filesystem stand-in: files in a map, with failures on demand. */ +class FakeFilesystem implements FilesystemPlugin { + files = new Map(); + /** Fail the next writeFile after writing this many characters of it. */ + failWriteAfter: number | null = null; + failRename = false; + async readFile({ path }: { path: string }) { + const data = this.files.get(path); + if (data === undefined) throw new Error("File does not exist"); + return { data }; + } + async writeFile({ path, data }: { path: string; data: string }) { + if (this.failWriteAfter !== null) { + this.files.set(path, data.slice(0, this.failWriteAfter)); + this.failWriteAfter = null; + throw new Error("Disk full"); + } + this.files.set(path, data); + } + async rename({ from, to }: { from: string; to: string }) { + if (this.failRename) throw new Error("Interrupted"); + const data = this.files.get(from); + if (data === undefined) throw new Error("File does not exist"); + this.files.set(to, data); + this.files.delete(from); + } + async deleteFile({ path }: { path: string }) { + if (!this.files.delete(path)) throw new Error("File does not exist"); + } +} + +const version = (n: number): SaveDocument => ({ ...fullSave(), lastTimeSeen: 1_000 + n }); + +const backends: [string, () => SaveBackend][] = [ + ["memory", () => new MemoryBackend()], + ["IndexedDB", () => new IndexedDbBackend(new IDBFactory())], + ["filesystem", () => new FilesystemBackend(new FakeFilesystem())], +]; + +describe.each(backends)("save storage on %s (Step 14.3)", (_name, makeBackend) => { + it("is empty before the first save", async () => { + expect(await new SaveStorage(makeBackend()).load()).toEqual({ status: "empty" }); + }); + + it("loads what it saved", async () => { + const storage = new SaveStorage(makeBackend()); + await storage.save(version(1)); + expect(await storage.load()).toEqual({ status: "loaded", save: version(1) }); + }); + + it("keeps exactly the last three saves as backups, newest first", async () => { + const storage = new SaveStorage(makeBackend()); + for (let n = 1; n <= 5; n++) await storage.save(version(n)); + expect(await storage.listBackups()).toEqual([ + { index: 1, valid: true, lastTimeSeen: 1_004 }, + { index: 2, valid: true, lastTimeSeen: 1_003 }, + { index: 3, valid: true, lastTimeSeen: 1_002 }, + ]); + const second = await storage.loadBackup(2); + expect(second.ok && second.save.lastTimeSeen).toBe(1_003); + }); + + it("falls back to the newest valid backup when the main save is corrupted", async () => { + const backend = makeBackend(); + const storage = new SaveStorage(backend); + await storage.save(version(1)); + await storage.save(version(2)); + await storage.save(version(3)); + await backend.write("main", '{"saveVersion": 1, "company": '); + await backend.write("backup-1", JSON.stringify({ ...version(2), company: null })); + const result = await storage.load(); + expect(result).toMatchObject({ status: "restored-from-backup", backup: 2, save: version(1) }); + }); + + it("reports an unreadable save when nothing validates", async () => { + const backend = makeBackend(); + await backend.write("main", "not a save"); + expect(await new SaveStorage(backend).load()).toEqual({ + status: "unreadable", + problem: "The save isn't valid JSON.", + }); + }); + + it("clears the save and its backups", async () => { + const storage = new SaveStorage(makeBackend()); + await storage.save(version(1)); + await storage.save(version(2)); + await storage.clear(); + expect(await storage.load()).toEqual({ status: "empty" }); + expect(await storage.listBackups()).toEqual([]); + }); +}); + +describe("native writes are atomic (Step 14.3)", () => { + it("never leaves a half-written main file when a write is interrupted", async () => { + const fs = new FakeFilesystem(); + const storage = new SaveStorage(new FilesystemBackend(fs)); + await storage.save(version(1)); + + fs.failWriteAfter = 20; + await expect(storage.save(version(2))).rejects.toThrow("Disk full"); + expect(await storage.load()).toEqual({ status: "loaded", save: version(1) }); + + fs.failRename = true; + await expect(storage.save(version(3))).rejects.toThrow("Interrupted"); + expect(await storage.load()).toEqual({ status: "loaded", save: version(1) }); + + fs.failRename = false; + await storage.save(version(4)); + expect(await storage.load()).toEqual({ status: "loaded", save: version(4) }); + }); +}); diff --git a/packages/engine/tsconfig.json b/packages/engine/tsconfig.json index 7680f99..e9ed20c 100644 --- a/packages/engine/tsconfig.json +++ b/packages/engine/tsconfig.json @@ -1,4 +1,7 @@ { "extends": "../../tsconfig.base.json", + // The engine runs in the app: browser APIs (IndexedDB, Web Locks) are in reach, + // but only through interfaces it is handed. + "compilerOptions": { "lib": ["ES2023", "DOM"] }, "include": ["src", "test"] } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 7a0ffaa..11af714 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -181,6 +181,10 @@ importers: zod: specifier: ^4.6.5 version: 4.6.5 + devDependencies: + fake-indexeddb: + specifier: ^6.2.5 + version: 6.2.5 packages/schema: dependencies: @@ -2781,6 +2785,10 @@ packages: resolution: {integrity: sha512-KfYbmpRm0VbLjEvVa9yGwCi9GI34xvi7A/HXYWQO65CSD2u3MczUJSuwXKFIxlGsgBQizV9q5J9NHj4VG0n+pA==} engines: {node: '>=12.0.0'} + fake-indexeddb@6.2.5: + resolution: {integrity: sha512-CGnyrvbhPlWYMngksqrSSUT1BAVP49dZocrHuK0SvtR0D5TMs5wP0o3j7jexDJW01KSadjBp1M/71o/KR3nD1w==} + engines: {node: '>=18'} + fast-decode-uri-component@1.0.1: resolution: {integrity: sha512-WKgKWg5eUxvRZGwW8FvfbaH7AXSh2cL+3j5fMGzUMCxWBJ3dV3a7Wz8y2f/uQ0e3B6WmodD3oS54jTQ9HVTIIg==} @@ -6587,6 +6595,8 @@ snapshots: expect-type@1.4.0: {} + fake-indexeddb@6.2.5: {} + fast-decode-uri-component@1.0.1: {} fast-deep-equal@3.1.3: {}