Infrastructure as Code

Infrastructure as code (IaC) means the servers, queues, databases and buckets a system runs on are defined in text files, kept in git, and reviewed like any other code — instead of being clicked together by hand in a cloud console. On Orca every AWS resource is declared this way, so the whole account can be rebuilt from the repository.

This mattered because the team was tiny — the CTO plus one engineer, then two. A tiny team cannot also be a 24/7 operations team. When infrastructure is code, a junior engineer never SSHes into a box or clicks around a console; they change a YAML file and open a pull request, and the infrastructure gets reviewed the same way the application code does. Repeatability is the whole point: no hand-built snowflake servers that only one person understands.

Before the pipeline, here is the full stack, layer by layer. The rule behind every row is the same: pick the cheapest, lowest-maintenance option that does the job, not the most powerful one. (Glossary: Vite is a frontend build tool; OpenLayers renders map-style image tiles; RDS is AWS's managed MySQL; ECR is AWS's container-image registry; OpenSlide reads whole-slide pathology images.)

Each choice is deliberately boring. The point is not the logo list; it is that no layer reaches for something exotic when a plain, well-understood tool will do.

Two tools describe the infrastructure, and both are just YAML. Pulumi provisions the AWS resources — S3 buckets, queues, the RDS database, EC2 — and also triggers the Docker image build into ECR (the registry that stores container images). The Serverless Framework declares the Lambda services (small functions AWS runs on demand). Continuous integration and delivery — CI/CD, the automation that tests and ships code — is split by concern: CircleCI ships the Python backend, Vercel ships the two React apps, and GitHub Actions ties the pipelines together.

Nothing is provisioned by hand, so the account can be rebuilt from the repo. The margin shows the pipeline: GitHub Actions orchestrates, CircleCI drives Pulumi and the Serverless Framework into AWS, and Vercel deploys the frontend. A single Lambda service is a few lines of declared configuration, not a server someone set up:

# serverless.yml — one Lambda service, declared not clicked
service: orca-core
provider: { name: aws, runtime: python3.11 }
functions:
  api:
    handler: app.handler
    events: [{ httpApi: "*" }]

Not every job wants the same kind of machine. Orca runs three deliberately different homes for compute, each matched to a workload's actual shape. (Glossary: "scales to zero" means it costs nothing when idle; a GPU is a graphics processor that AI models run fast on; Fargate runs containers without you managing any servers; WSI is a whole-slide image; DZI is the Deep Zoom tile format a viewer streams.)

The discipline is to match compute to the workload — and, because the team was small and on a tight budget, to default to the cheapest option that still does the job. Serverless is the default; the team only stepped off it where the workload forced it: containerized tiling on Fargate, GPU inference on EC2. Adding a new workload is just a Pulumi file granting the resources plus a queue to consume — it works with any workflow step.

Deploying a new AI model happens in two phases, and part of it is deliberately manual at this stage. (Glossary: MLOps is the practice of shipping machine-learning models to production; model artifacts are the trained weights plus config; a container image bundles code and its libraries so it runs the same everywhere.)

Phase one builds the image: a CircleCI trigger provisions AWS from Serverless-Framework YAML, Pulumi builds the Docker image bundling OpenSlide and PyTorch, and pushes it to ECR. MLOps uploads the model artifacts to S3. Phase two activates the model: any job trigger launches an EC2 instance, which pulls the image and the artifacts, activates the model, and the developer updates the backend with the inference URL. Reserved instances or fleets keep that step cheap.

The manual activation step is called out honestly in Day 11's tradeoffs — at this stage, running it occasionally by hand was faster and simpler than automating a pipeline used rarely.

On a very low budget, cost is a reliability concern — so the budget itself is enforced by code, not by a spreadsheet someone remembers to check. The Budget Manager is a real service, and it holds the only hard numbers in this whole case study.

It warns at 70% and 80% of budget. At 90% it stops asking and starts acting: it purges the SQS queues, stops all ECS tasks, and blocks new S3 puts — halting spend at the source. Separately, any ECS run over 100 tasks or 30 minutes is auto-killed, so a stuck job cannot run up the bill overnight. Billing alarms trip at $1,000 and $5,000, and a cost report lands in Slack every morning at 9am. Encoded as policy, the guardrails read like this:

budget_manager:
  warn_at: [0.70, 0.80]          # Slack warning only
  act_at:  0.90                  # stop spend at the source
  on_act:
    - purge_sqs_queues
    - stop_all_ecs_tasks
    - block_s3_puts
  ecs_autokill: { max_tasks: 100, max_minutes: 30 }
  billing_alarms_usd: [1000, 5000]
  daily_report: { to: slack, at: "09:00" }

• Infrastructure as code means every server, queue, bucket and database is declared in versioned files and reviewed like application code — the account rebuilds from the repo, with nothing clicked by hand.
• IaC was a tiny-team enabler: a junior changes a YAML file and opens a PR instead of operating servers, so the small team was never a full-time ops team.
• The stack picks the cheapest, lowest-maintenance tool at each layer — React/Vite/OpenLayers frontend, Python/Flask/RDS backend, and Sentry/CloudWatch/Amplitude monitoring.
• All-YAML IaC: Pulumi provisions AWS and builds Docker images into ECR; the Serverless Framework declares Lambda services; CircleCI ships the backend, Vercel the frontend, GitHub Actions ties the pipelines.
• Compute is placed by workload shape: Lambda for backend core (scales to zero), ECS + Fargate for bursty WSI→DZI tiling, EC2 GPU behind a load balancer for AI inference.
• Model deployment is a two-phase sequence — build the OpenSlide + PyTorch image to ECR via Pulumi and upload artifacts to S3, then launch EC2 to pull and activate — with the final activation deliberately manual for now.
• Budget guardrails are code: warn at 70/80%, act at 90% (purge SQS, stop ECS, block S3 puts), auto-kill ECS runs over 100 tasks or 30 minutes, billing alarms at $1k/$5k, and a 9am daily Slack cost report.

• [ ] I can define infrastructure as code and explain why it let a tiny team avoid being a full-time ops team.
• [ ] I can name the eight stack layers and the principle behind each choice (cheapest tool that does the job).
• [ ] I can explain the all-YAML split: Pulumi vs the Serverless Framework, and CircleCI vs Vercel vs GitHub Actions.
• [ ] I can justify the three compute placements — Lambda, ECS + Fargate, EC2 GPU — from each workload's shape.
• [ ] I can walk the two-phase model-deployment sequence and say which step is still manual and why.
• [ ] I can recite the budget guardrail numbers: 70/80/90%, >100 tasks / >30 min, $1k/$5k alarms, 9am Slack report.
• [ ] I understand why encoding cost limits as a service (not a spreadsheet) makes cost a reliability property.