Inference Is a Budgeted Search

A language model normally turns one request into one response. Today you will treat that response as one point in a search, give the request an explicit inference budget, and produce task-outcomes.json: a small task set with observable success rules and cost limits. Test-time compute means computation spent after training, while answering a particular request; inference is that answer-producing phase.

The model itself is unchanged. More inference compute can fund more candidate answers, longer revision chains, verification, or selection. More calls can raise candidate coverage—the chance a usable answer exists in the pool—but deployed pass@1 also depends on the task/model sampling distribution and selector accuracy. Either can plateau or regress. Stanford's lecture distinguishes this scaling axis from pre-training and introduces an inference-time scaling relationship at 00:04:43–00:05:24. This course is a conceptual reconstruction, not a source-figure transcription or a claim that extra compute always helps.

ShipRight, our recurring pricing service, has a bug report: “a half-cent discount sometimes rounds up; policy requires rounding down.” One answer may guess. A budgeted search records the request, caps work, generates a candidate, and records evidence.

Conceptual reconstruction; not a source-figure transcription:

Stable visual IDs are predeclared for the course: task, budget-router, generator, candidate-store, difficulty-profile, revision-loop, critic, deterministic-test-runner, learned-verifier, ranker, fuser, selected-answer, release-review, and evidence-store. Diagrams use underscore-safe Mermaid aliases where needed while printing those exact labels. Later days preserve identity and meaning; deterministic tests, learned verification, and ranking never collapse into one box.

A vague instruction such as “think harder” cannot be audited. A useful budget is a limit with a unit: candidate count, generated tokens, verifier calls, wall-clock seconds, or money. The allocation is a product decision because every extra unit consumes latency and capacity.

For ShipRight, define this Day 01 envelope:

A candidate proposes replacing floating-point rounding with integer arithmetic:

def discounted_cents(price_cents: int, discount_bps: int) -> int:
    return price_cents * (10_000 - discount_bps) // 10_000

bps means basis points: one basis point is one hundredth of one percent, so 5_000 means 50%. For price_cents=3, policy says 3 × 0.5 = 1.5 cents must round down to 1 cent. Integer floor division expresses that rule without binary floating-point.

Record task contracts before attempts. Save this as task-outcomes.json:

{
  "schema": "task-outcomes/v1",
  "tasks": [
    {
      "task_id": "shipright-price-017",
      "success_rule": "public_and_protected_tests_pass",
      "max_candidates": 1,
      "max_candidate_bytes": 8192,
      "max_wall_seconds": 10,
      "promotion": "review_only"
    }
  ]
}

This is normal-path evidence: one bounded task contract. It is not a success claim.

Budget enforcement needs a failure path. Suppose the model emits 12 KiB against the 8 KiB limit. Truncate nothing into plausible-looking code; reject the whole candidate with decision="rejected_byte_limit". If generation exceeds 10 seconds, terminate its process and record decision="rejected_timeout".

An unaffected positive control is a 2 KiB candidate returned in 2 seconds: it still receives decision="unverified", because staying inside cost limits does not prove correctness. Recovery means releasing the worker slot and temporary output, then accepting a later request. Cleanup is part of the evidence, not an optional housekeeping step.

Scored check, with one unknown: budget allows one candidate, 8 KiB, and 10 seconds; output is 12 KiB. What decision must the receipt contain? Expected: rejected_byte_limit. Choosing unverified confuses “not tested” with “violated execution limit.”

Assessment receipt: given one unknown—12 KiB output against an 8 KiB cap—the accepted answer is rejected_byte_limit; evidence is the single receipt field, misconception is treating “not yet tested” as “within budget,” and remediation is replaying the byte comparison.

Day 01 hands task-outcomes.json to Day 02, which samples candidates and measures task difficulty; no repair is promoted.