Step 12.2: tutorial economy check on measured numbers
CI / check (push) Successful in 6m14s Details

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
fireshaper 2026-10-05 19:23:44 -04:00
parent c02126c908
commit 1909eddfb2
6 changed files with 99 additions and 24 deletions

View File

@ -1266,3 +1266,8 @@ Implementation decisions not spelled out in `memory-bank/implementation-plan.md`
- **Targets are checked on the median of 7 world seeds**: one world's job luck moves the first purchase by tens of minutes (it lands right after the first big delivery). Invariants must hold in every world. The timing targets apply to the Lighter-first player (the plan's "buy the cheapest ship"); Freighter-first can't buy a 100,000 cr ship in 45–75 minutes with numbers that also keep Lighter-first in that window, so it is used for the four-ship comparison only. - **Targets are checked on the median of 7 world seeds**: one world's job luck moves the first purchase by tens of minutes (it lands right after the first big delivery). Invariants must hold in every world. The timing targets apply to the Lighter-first player (the plan's "buy the cheapest ship"); Freighter-first can't buy a 100,000 cr ship in 45–75 minutes with numbers that also keep Lighter-first in that window, so it is used for the four-ship comparison only.
- **The simulated player's choices** (not specified by the plan): best credits-per-minute contract after fuel and pilot pay; deadhead only to a place with work for that ship; rest a pilot with duty hours when there's nothing worth flying; skill points to Long Haul for ships with range past the first cut-off, alternating with the cargo skill that suits the hold (Heavy for 60 t and up, Expensive below); Market ships in policy order with a pilot hired at Earth, keeping 5,000 cr back for fuel; upgrades (cargo bay, then engines, on the most-used ship at an HQ) only once the planned ships are bought and only from 10 % of cash; goals claimed when ready; no extra bays. - **The simulated player's choices** (not specified by the plan): best credits-per-minute contract after fuel and pilot pay; deadhead only to a place with work for that ship; rest a pilot with duty hours when there's nothing worth flying; skill points to Long Haul for ships with range past the first cut-off, alternating with the cargo skill that suits the hold (Heavy for 60 t and up, Expensive below); Market ships in policy order with a pilot hired at Earth, keeping 5,000 cr back for fuel; upgrades (cargo bay, then engines, on the most-used ship at an HQ) only once the planned ships are bought and only from 10 % of cash; goals claimed when ready; no extra bays.
- **Invariant definitions:** "soft-lock" = after the deal, nothing in flight and no affordable fuel for any contract on a board where a crewed ship is; "idle with nowhere to go" = a crewed, rested ship with no contract and no deadhead it could take **with unlimited credits** (being short of cash is the soft-lock check's job); "first session" = the first two real hours. - **Invariant definitions:** "soft-lock" = after the deal, nothing in flight and no affordable fuel for any contract on a board where a crewed ship is; "idle with nowhere to go" = a crewed, rested ship with no contract and no deadhead it could take **with unlimited credits** (being short of cash is the soft-lock check's job); "first session" = the first two real hours.
## Step 12.2
- **The Step 2.5 economy check moved to the sim's tests**: it needs measured numbers from the balance script, and `packages/content` can't depend on `packages/sim`. It now checks what the simulated player actually earned in every seeded run instead of a "lowest-paying contract" assumption.
- **Decision 0.5.85 got its own board test** here (it was only covered indirectly by the balance targets in 12.1).

View File

@ -307,7 +307,7 @@ Spec: `memory-bank/change-requests/CR-013-veterans-clipper.md` (received as "CR-
## Phase 12 — Balance Harness and End-to-End Tests ## Phase 12 — Balance Harness and End-to-End Tests
- [x] **Step 12.1 — Balance simulation script.** `packages/sim/balance/simulate.ts`: a pure, deterministic simulated player that plays from the start using only sim and content functions the way the server applies them (tutorial runs, deal, boards per period, dispatch, deadhead, rest, resolution, goals, skill points, Market purchases with a hired pilot, upgrades from spare cash), with a trip log for tracing. Reports milestones (deal, first Market purchase, first discovery, loan paid off, four ships), credits/loan/ships/pilot levels every 10 minutes, and breaches of the invariants (soft-lock, idle ship with no work and no way out, dispatch while resting). `pnpm --filter @deadhead/sim balance [seed] [hours]` prints both policies. **Design changes (user-approved):** pay and fuel use economic distance (decision 0.5.84), and boards offer a first contact the fleet can fly (0.5.85). **Tuning:** economic-distance exponent 0.65, Clipper fuel 600 → 450 cr/M km, loan repayment share 0.3 → 0.25. Medians over 7 seeds (Lighter-first): deal 2.9 min, first purchase 66, first discovery 33, loan paid off 188, four ships 170 (Freighter-first 201); no breaches in any run. Test: `test/balance.test.ts` (determinism, each target on the median, four-ship comparison, invariants in every run, veteran contract in the first session, samples reported). - [x] **Step 12.1 — Balance simulation script.** `packages/sim/balance/simulate.ts`: a pure, deterministic simulated player that plays from the start using only sim and content functions the way the server applies them (tutorial runs, deal, boards per period, dispatch, deadhead, rest, resolution, goals, skill points, Market purchases with a hired pilot, upgrades from spare cash), with a trip log for tracing. Reports milestones (deal, first Market purchase, first discovery, loan paid off, four ships), credits/loan/ships/pilot levels every 10 minutes, and breaches of the invariants (soft-lock, idle ship with no work and no way out, dispatch while resting). `pnpm --filter @deadhead/sim balance [seed] [hours]` prints both policies. **Design changes (user-approved):** pay and fuel use economic distance (decision 0.5.84), and boards offer a first contact the fleet can fly (0.5.85). **Tuning:** economic-distance exponent 0.65, Clipper fuel 600 → 450 cr/M km, loan repayment share 0.3 → 0.25. Medians over 7 seeds (Lighter-first): deal 2.9 min, first purchase 66, first discovery 33, loan paid off 188, four ships 170 (Freighter-first 201); no breaches in any run. Test: `test/balance.test.ts` (determinism, each target on the median, four-ship comparison, invariants in every run, veteran contract in the first session, samples reported).
- [ ] Step 12.2 — Revisit the tutorial economy test - [x] **Step 12.2 — Revisit the tutorial economy test.** The amended Step 2.5 check ("the tutorial share plus starting credits covers fuel for three Courier Earth→Moon runs") no longer assumes the lowest-paying tutorial contract: the simulator records the tutorial share actually earned and the credits at the deal, and `packages/sim/test/balance.test.ts` checks, in every seeded run, four tutorial runs, credits at the deal = starting credits + share earned, and at least three Courier Earth→Moon fuel loads. The placeholder test left `packages/content/test/config.test.ts` (content can't import the sim). Also added `test/first-contact-fit.test.ts` for decision 0.5.85 (Step 12.1): every Earth board over four hours offers a first contact to Mars that the deal's Clipper crew can take, and none once Mars is discovered.
- [ ] Step 12.3 — End-to-end: tutorial to first ship - [ ] Step 12.3 — End-to-end: tutorial to first ship
- [ ] Step 12.4 — End-to-end: offline progress - [ ] Step 12.4 — End-to-end: offline progress
- [ ] Step 12.5 — End-to-end: discovery - [ ] Step 12.5 — End-to-end: discovery

View File

@ -1,6 +1,5 @@
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { import {
cargoTypes,
content, content,
dealLoanAmount, dealLoanAmount,
gameConfig, gameConfig,
@ -10,9 +9,6 @@ import {
skills, skills,
} from "../src/index"; } from "../src/index";
// The Moon sits a fixed 0.384M km from Earth (Step 3.3).
const EARTH_MOON_MILLION_KM = 0.384;
describe("game config", () => { describe("game config", () => {
it("passes the schema", () => { it("passes the schema", () => {
expect(() => gameConfigSchema.parse(gameConfig)).not.toThrow(); expect(() => gameConfigSchema.parse(gameConfig)).not.toThrow();
@ -27,24 +23,8 @@ describe("game config", () => {
expect(gameConfig.longHaulCutoffsMillionKm).toHaveLength(longHaul.maxLevel); expect(gameConfig.longHaulCutoffsMillionKm).toHaveLength(longHaul.maxLevel);
}); });
it("covers fuel for at least three Courier runs from Earth after the deal (no soft-lock)", () => { // The "fuel for three Courier runs after the deal" check (Step 2.5 as amended
// Amended Step 2.5 (CR-001). Assume the player always flies the lowest-paying // by CR-001) runs on measured numbers in packages/sim/test/balance.test.ts (Step 12.2).
// (light) tutorial contract; Phase 12 replaces these with measured numbers.
const courier = shipModels.find((s) => s.id === "courier")!;
const earth = locations.find((l) => l.id === "earth")!;
const standard = cargoTypes.find((c) => c.id === "standard")!;
const lightTutorialPayout =
standard.basePayPerTonnePerMillionKm *
gameConfig.tutorialMassTonnes.light *
EARTH_MOON_MILLION_KM *
gameConfig.tutorialPayoutShare;
const earned =
lightTutorialPayout * gameConfig.tutorialRunsBeforeDeal + gameConfig.startingCredits;
const fuelFromEarth =
EARTH_MOON_MILLION_KM * courier.fuelCostPerMillionKm * earth.launchCostMultiplier;
expect(earned).toBeGreaterThanOrEqual(3 * fuelFromEarth);
});
it("sizes the tutorial board and loaner for the three mass tiers", () => { it("sizes the tutorial board and loaner for the three mass tiers", () => {
expect(gameConfig.tutorialBoardSize).toBe(3); expect(gameConfig.tutorialBoardSize).toBe(3);

View File

@ -91,6 +91,10 @@ export interface SimulationResult {
/** Every 10 real minutes. */ /** Every 10 real minutes. */
samples: Sample[]; samples: Sample[];
tutorialRuns: number; tutorialRuns: number;
/** The player's tutorial share over all tutorial runs (Step 12.2). */
tutorialShareEarned: number;
/** Credits on hand the moment the deal is accepted. */
creditsAtDeal: number | null;
contractsDelivered: number; contractsDelivered: number;
/** Contracts the veteran delivered in the first two hours (CR-010 check). */ /** Contracts the veteran delivered in the first two hours (CR-010 check). */
veteranContractsFirstSession: number; veteranContractsFirstSession: number;
@ -186,6 +190,8 @@ export function simulate(options: SimulationOptions): SimulationResult {
const violations: SimulationResult["violations"] = []; const violations: SimulationResult["violations"] = [];
const purchases: string[] = []; const purchases: string[] = [];
let contractsDelivered = 0; let contractsDelivered = 0;
let tutorialShareEarned = 0;
let creditsAtDeal: number | null = null;
let priorityDeliveries = 0; let priorityDeliveries = 0;
let veteranContractsFirstSession = 0; let veteranContractsFirstSession = 0;
const seenViolations = new Set<string>(); const seenViolations = new Set<string>();
@ -332,6 +338,7 @@ export function simulate(options: SimulationOptions): SimulationResult {
} }
if (flight.record.tutorial) { if (flight.record.tutorial) {
runsCompleted++; runsCompleted++;
tutorialShareEarned += r.credits;
loanerLocationId = r.shipLocationId as "earth" | "moon"; loanerLocationId = r.shipLocationId as "earth" | "moon";
} else if (flight.record.contract) { } else if (flight.record.contract) {
contractsDelivered++; contractsDelivered++;
@ -425,6 +432,7 @@ export function simulate(options: SimulationOptions): SimulationResult {
}); });
loanBalance = content.dealLoanAmount; loanBalance = content.dealLoanAmount;
tutorialActive = false; tutorialActive = false;
creditsAtDeal = credits;
milestones.dealAccepted = minute(); milestones.dealAccepted = minute();
} }
@ -706,6 +714,8 @@ export function simulate(options: SimulationOptions): SimulationResult {
milestones, milestones,
samples, samples,
tutorialRuns: runsCompleted, tutorialRuns: runsCompleted,
tutorialShareEarned,
creditsAtDeal,
contractsDelivered, contractsDelivered,
veteranContractsFirstSession, veteranContractsFirstSession,
violations, violations,

View File

@ -1,6 +1,7 @@
import { beforeAll, describe, expect, it } from "vitest"; import { beforeAll, describe, expect, it } from "vitest";
import { simulate, type Milestones, type SimulationResult } from "../balance/simulate"; import { simulate, type Milestones, type SimulationResult } from "../balance/simulate";
import { MS_PER_HOUR } from "../src/index"; import { content } from "@deadhead/content";
import { MS_PER_HOUR, realToGame, tripFuelCost } from "../src/index";
/** /**
* Balance targets (plan Step 12.1 as amended by CR-001 to CR-013), checked on * Balance targets (plan Step 12.1 as amended by CR-001 to CR-013), checked on
@ -70,6 +71,22 @@ describe("balance simulation (Step 12.1)", () => {
expect(result.veteranContractsFirstSession).toBeGreaterThan(0); expect(result.veteranContractsFirstSession).toBeGreaterThan(0);
}); });
it("leaves fuel for at least three Courier runs from Earth after the deal (Step 2.5, measured)", () => {
// Step 12.2: the tutorial share and starting credits as actually earned.
const courier = content.shipModel("courier")!;
const [earth, moon] = [content.location("earth")!, content.location("moon")!];
const start = realToGame(Date.parse(content.config.epoch.realStart), content.config);
const fuel = tripFuelCost(courier, earth, moon, start, content.config);
for (const result of lighterFirst) {
expect(result.tutorialRuns).toBe(content.config.tutorialRunsBeforeDeal);
expect(result.tutorialShareEarned).toBeGreaterThan(0);
expect(result.creditsAtDeal).toBe(
content.config.startingCredits + result.tutorialShareEarned,
);
expect(result.creditsAtDeal!).toBeGreaterThanOrEqual(3 * fuel);
}
});
it("reports credits and pilot levels over time", () => { it("reports credits and pilot levels over time", () => {
const [run] = lighterFirst; const [run] = lighterFirst;
expect(run!.samples.length).toBeGreaterThan(5); expect(run!.samples.length).toBeGreaterThan(5);

View File

@ -0,0 +1,63 @@
import { content } from "@deadhead/content";
import { describe, expect, it } from "vitest";
import {
boardPeriodIndex,
generateContractBoard,
validateDispatch,
type FleetMember,
} from "../src/index";
const earth = content.location("earth")!;
const start = Date.parse(content.config.epoch.realStart);
const firstPeriod = boardPeriodIndex(start, content.config);
/** The deal's Clipper with the veteran after Long Haul 2: Mars is in reach at the start. */
const clipper: FleetMember = {
modelId: "clipper",
skills: { "expensive-cargo": 1, "long-haul": 2 },
};
const board = (periodIndex: number, discovered: string[], fleet: FleetMember[]) =>
generateContractBoard(
{ location: earth, discoveredIds: new Set(discovered), periodIndex, worldSeed: 4242, fleet },
content,
);
/** The fleet's Clipper can take it: range, landing, mass, skills, no stranding. */
const clipperCanFly = (contract: ReturnType<typeof board>[number]) =>
validateDispatch(
{
ship: { id: "s", modelId: "clipper", locationId: "earth", inFlight: false },
pilot: {
id: "p",
isAvatar: false,
assignedShipId: "s",
inFlight: false,
skills: clipper.skills,
restEndsAt: null,
},
contract,
tutorial: false,
credits: 1e9,
now: contract.offeredAt,
discoveredIds: new Set(["earth", "moon"]),
},
content,
).ok;
describe("a first contact the fleet can fly (decision 0.5.85)", () => {
it("offers one on every board while a reachable place is undiscovered", () => {
for (let k = 0; k < 24; k++) {
const contracts = board(firstPeriod + k, ["earth", "moon"], [clipper]);
const flyable = contracts.filter((c) => c.firstContact && clipperCanFly(c));
expect(flyable.length, `period ${k}`).toBeGreaterThanOrEqual(1);
expect(flyable.every((c) => c.destinationId === "mars")).toBe(true);
}
});
it("stops once the reachable places are discovered", () => {
for (let k = 0; k < 24; k++) {
const contracts = board(firstPeriod + k, ["earth", "moon", "mars"], [clipper]);
expect(contracts.filter((c) => c.firstContact && clipperCanFly(c))).toEqual([]);
}
});
});