AI Services & Workflow Engine: behavior as data

Earlier days turned the UI into data — the frontend renders whatever JSON the backend sends. This day does the same thing to behavior. A workflow is a multi-step job, like "run the AI model on this slide." A blueprint is the mould of such a job: an ordered list of steps (small named units of work such as preprocess or inference). The trick is that the blueprint lives as a row in the database, not as code. The engine reads the blueprint, turns it into concrete steps, and runs each in order.

Because the pipeline is data, changing what a job does is an edit to a row — not a new deploy. Research ships new models constantly and different labs want different processing; if each of those were a code change, the field would be blocked on the engineering team forever. Making the pipeline a JSON document is what breaks that dependency.

Here is the mechanism in one loop. The blueprint sits in RDS as an ordered list, for example [preprocess, inference, postprocess]. The engine parses that list, generates one concrete step object per entry, and runs them in order. Each step is not run inline — it is published to a queue and picked up by a subscriber that executes it and records the result. That publish-then-subscribe arrangement is a plain pub/sub model (covered in detail later this day).

The two properties that make this safe are worth naming now: the blueprint is data (so the sequence is editable without code), and every step's state is written to the database before it runs (so nothing is lost on a crash). The rest of this day is those two ideas, pushed all the way.

The clearest way to see "behavior as data" is to watch one workflow grow by editing only its JSON. Each edit below is the whole change — no engine code is touched, because the engine already knows how to instantiate and run whatever step names the blueprint lists.

Build 1/3 — the smallest real workflow. Just run the model. One step, inference, on a GPU box, reading and writing S3.

{
  "blueprint": "ki67",
  "org_id":    "lab_b",
  "steps":     ["inference"]
}

Build 2/3 — wrap it. Add a preprocess step to prepare the tiles going in and a postprocess step to shape the annotations and stats coming out. Two new names in the list; nothing else changes.

{
  "blueprint": "ki67",
  "org_id":    "lab_b",
  "steps":     ["preprocess", "inference", "postprocess"]
}

Build 3/3 — a new step, live. A lab reports that its stain colour throws off the model. Insert color_normalization before inference — for that lab only. A new workflow is live by adding one JSON step, with zero code change.

{
  "blueprint": "ki67",
  "org_id":    "lab_b",
  "steps":     ["preprocess", "color_normalization", "inference", "postprocess"]
}

Because a workflow is a row keyed by org_id, two labs can run different pipelines off the same engine with no forked codebase and no per-customer branch. Lab A gets the plain pipeline; Lab B gets the one with colour normalization from the previous section. The difference between them is a single row in the database, keyed by org_id — not a special deploy.

This is what "behavior is data" buys across customers: one engine serves every lab, and a new lab that needs something different is a new blueprint, not new architecture.

The scope of the idea is best seen from the top: Orca did not write six services. It wrote one storage-first engine that knows a single loop — persist a step, run it, record the result, resume from the last good step on failure — and then expressed every capability on the platform as a blueprint. Uploading an image, correcting colour, tiling into DZI, running the full AI workflow, doing ROI inference, and generating statistics are all the same engine with a different list of steps.

A new capability is therefore a new blueprint, not a new engine. That is a large part of how a junior team stayed small: the hard part was solved once, generically, and then reused.

Each blueprint is a real ordered list of steps handed to the same engine. Note the shared fields: an org_id scopes the job to one lab, and "resume": "last_good_step" tells the engine to pick up from the last completed step rather than restart, if the job fails partway. Read each list as "what this job actually does, in order."

Upload — getting an image in safely. Bytes land in S3 first, then orientation is forced and EXIF stripped so every slide is normalized, and metadata is persisted last so the row exists before any downstream job can pick it up.

{
  "workflow": "upload",
  "org_id":   "orca-7f3a",
  "resume":   "last_good_step",
  "steps": [
    { "step": "upload_to_s3" },
    { "step": "force_autorotation" },
    { "step": "strip_exif_orientation" },
    { "step": "persist_metadata" }
  ]
}

Colour correction — normalizing stain and colour to a common profile. validate_output is its own step, so a bad correction fails the step, not the slide.

{
  "workflow": "color_correction",
  "org_id":   "orca-7f3a",
  "resume":   "last_good_step",
  "steps": [
    { "step": "load_image" },
    { "step": "normalize_color" },
    { "step": "validate_output" },
    { "step": "save_corrected" }
  ]
}

DZI tiling — turning a gigapixel slide into Deep Zoom tiles. Open the slide with random access, generate the pyramid, enrich metadata, upload the tiles. The pyramid is what lets a tablet stream only the tiles in view.

{
  "workflow": "dzi_tiling",
  "org_id":   "orca-7f3a",
  "resume":   "last_good_step",
  "steps": [
    { "step": "open_slide_random_access" },
    { "step": "generate_dzi_tiles" },
    { "step": "enrich_dzi_metadata" },
    { "step": "upload_dzi_tiles" }
  ]
}

AI workflow (end to end) — the full auto-annotate run. Ingest, tile, run the model on a GPU, postprocess predictions into annotations, then report. The heavy GPU step sits in the middle, so resume-from-last-step means a crash costs one step, not the whole run.

{
  "workflow": "ai_workflow",
  "org_id":   "orca-7f3a",
  "resume":   "last_good_step",
  "steps": [
    { "step": "upload_image" },
    { "step": "generate_deep_zoom_tiles" },
    { "step": "run_model_prediction" },
    { "step": "postprocess_annotations" },
    { "step": "generate_report" }
  ]
}

ROI inference — running a model inside a region a pathologist drew. Find the annotations overlapping the region of interest, segment, convert masks to geometric shapes, then write thousands of rows in one atomic statement.

{
  "workflow": "roi_inference",
  "org_id":   "orca-7f3a",
  "resume":   "last_good_step",
  "steps": [
    { "step": "identify_overlapping_annotations" },
    { "step": "roi_segmentation" },
    { "step": "convert_to_geometric_shapes" },
    { "step": "bulk_update_id_mapping" }
  ]
}

Statistics — counting and charting thousands of annotations at scale. Walk annotations in batches, aggregate counts and densities per class, downsample, then render. LTTB downsampling keeps even a million points inside a small function before rendering.

{
  "workflow": "statistics",
  "org_id":   "orca-7f3a",
  "resume":   "last_good_step",
  "steps": [
    { "step": "paginate_annotations" },
    { "step": "aggregate_metrics" },
    { "step": "downsample_lttb" },
    { "step": "render_chart" }
  ]
}

Storage-first means the engine writes a step and its state to the database before it runs any compute for that step. The database is the source of truth; the queue is only transport. Because state is written first, RDS naturally becomes an append-only action log — a record that only ever grows — of every job, every step, and every status transition it passed through.

That log is what makes replay, audit, and debugging fall out almost for free. For a small, fairly junior team, the answer to "what happened to this job?" is a query, not a reconstruction from scattered logs. A single job's log reads like this:

Read the log above and the payoff appears: run_model_prediction FAILED at 10:07:55, then went RUNNING again at 10:08:20 and COMPLETED. Because every step's state was persisted before it ran, a failure does not lose the run — the engine restarts from the last good step, not from the beginning. Resume-from-last-step means a crash costs one step, not the whole job, which matters most when the failed step is an expensive GPU run.

Here is a live run mid-flight. The first two steps succeeded and are durable; the GPU step failed and is being resumed; everything after it is still pending. The queue may redeliver a message, but that is safe: the durable row, not the message, decides what runs next.

The engine moves each step around with pub/sub — publish/subscribe, a pattern where one side posts messages and another side picks them up, with no direct call between them. The producer is the engine publishing a step to an SQS queue. The consumer is a scheduled worker that subscribes, polls the queue, runs the step, and records its status. No broker, no framework, no exotic messaging — just publish, subscribe, and a durable row.

This is storage-first pub/sub: the queue is only transport with at-least-once delivery, and the database holds the truth. That is precisely why redelivery is safe — if the same step message arrives twice, the durable state decides whether there is anything left to do. The exact same producer-consumer shape runs WSI tiling, AI inference, statistics, and data export; only the work inside the consumer differs.

Adding a brand-new kind of step is a copy-paste job, and two small patterns make it so. A Registry is a lookup table that maps a step name to the class that implements it. A Factory is the piece that, given a name and its config from the blueprint, builds the matching step object. A junior copies an existing step class, changes only the body, and registers the name — the engine never changes, because it just looks up whatever name the blueprint used.

Here is the whole of a new step. ColorNormalizeStep declares its name, and its run method loads the image, normalizes it, and saves the result. Register that name and any blueprint can use it.

# steps/color_normalize.py
class ColorNormalizeStep(Step):
    name = "color_normalize"

    def run(self, ctx):
        img = ctx.load()
        out = normalize(img)
        ctx.save(out)

The registry is just a set of known names — upload_to_s3, generate_dzi_tiles, run_model_prediction, roi_segmentation, and now color_normalize — and the Factory builds any of them from config. A fresh engineer adds a capability by copying a pattern, and the senior reviews a small, shaped diff instead of a bespoke pipeline.

A durable-execution engine is an off-the-shelf system that runs long, multi-step workflows and survives crashes — Temporal and Restate are two mature examples. Orca evaluated both and chose to build its own small engine instead. The reasons were specific to an early-stage product on a pre-seed budget, not a dismissal of those tools.

The deciding factor was uniformity: everything reuses the same few patterns — blueprint-to-steps, storage-first, producer/consumer — so a new engineer learns one shape and understands the whole system. Simplicity-to-scale was an explicit pillar, not a nice-to-have. And the choice is not a trap: if a given sub-workflow later needs stronger durability or scale, that one piece can move to Temporal or Restate without rewriting everything.

• Behavior is data: a blueprint is an ordered list of steps stored as a row in RDS, and the engine parses it, generates concrete steps, and runs them — so changing a pipeline is an edit, not a deploy.
• The build 1/2/3 walkthrough shows it concretely: ["inference"] grows to preprocess/inference/postprocess, then gains color_normalization in front — all by editing JSON, with zero code change.
• One engine serves every lab; two labs differ only by a blueprint row keyed by org_id, not by a forked codebase.
• Six blueprints — upload, colour correction, DZI tiling, the e2e AI run, ROI inference, and statistics — are all the same engine with a different ordered list of steps.
• Storage-first means state is written to the database before compute runs, which turns RDS into an append-only action log and makes replay, audit, and debugging a query.
• Resume-from-last-step means a failure costs one step, not the whole run — critical when the failed step is an expensive GPU job; the durable row, not the queue message, decides what runs next.
• Steps move over plain pub/sub: a producer publishes to SQS, a scheduled consumer runs the step and records status, and at-least-once redelivery is safe because the database holds the truth.
• A Registry maps step names to classes and a Factory builds them from config, so adding a step is copy-a-class-and-register-the-name — the engine never changes.
• Build-vs-buy was deliberate: Temporal and Restate were considered, but a small owned engine won on footprint, IP, tractability, and uniform patterns — with a clear migration path if scale later demands it.

• [ ] I can explain what a blueprint and a step are, and why storing the blueprint in RDS (not code) makes behavior data.
• [ ] I can walk through build 1/3 → 2/3 → 3/3 and say what changed at each edit and what did not.
• [ ] I can explain how two labs run different pipelines from one engine using an org_id-keyed row.
• [ ] I can name the six blueprints and describe roughly what each one's steps do.
• [ ] I can define storage-first and explain how it produces an append-only action log.
• [ ] I can read the action-log table and explain how resume-from-last-step recovers a failed GPU run.
• [ ] I can describe the producer/consumer pub/sub loop and why at-least-once redelivery is safe here.
• [ ] I can explain the Registry + Factory pattern and how a junior adds a new step type without touching the engine.
• [ ] I can give the build-vs-buy reasoning for building a small engine over adopting Temporal or Restate.