The Patent: Adaptive Rendering

Day 5 left one decision unfinished: when the Adaptive Viewing Algorithm keeps a mark, how much of it does it draw? The answer is the piece of Orca that SpatialX filed a patent on. Patent P91594EP covers a method for showing very large annotated medical images — a whole-slide image (WSI), the gigapixel scan of tissue a pathologist reads — on ordinary devices like tablets and phones. A single scan can be around 100,000 × 100,000 pixels with tens of thousands of marks on it, and an ordinary tablet cannot draw all of that at once.

The idea fits on one line: show only what fits the device. Instead of loading the whole giant image and every mark, the method loads just what is needed, and it checks the device's power — memory, processor, screen — to decide how much detail to draw. A weaker device gets a lighter version; a stronger one gets more; either way it stays fast.

The method has four moving parts, and this day walks each one:

• Detail follows zoom (the core claim).
• Viewport culling — only draw what is on screen.
• Shape simplification — fewer points per shape.
• Clustering — group a dense crowd into a few marks with a QuadTree.

Plus the gesture layer that lets the pathologist edit on the same screen, with every change written back to the real record.

The central claim is a level-of-detail rule: the amount of detail rendered scales with zoom. Level of detail just means drawing a thing more or less finely depending on how much of it you can actually see. When the pathologist is zoomed way out, no one can see individual cell outlines anyway, so the method draws a heatmap — colour showing where marks are dense, with no outlines at all. At a middle zoom it draws simplified shapes. Only when zoomed in close does it draw full polygons, every vertex of every mark.

This is the claim everything else serves: the three techniques below are the concrete ways the method produces less detail when the zoom (or the device) asks for less.

Viewport culling means refusing to draw anything outside the visible rectangle. The method picks the subregion the pathologist is currently viewing — the viewport — not the entire scan, and draws only the marks inside it. Everything outside is unloaded. To keep panning quick, it gets the neighbouring areas ready in the background; and on a weaker device it keeps fewer marks while still showing the same overall pattern.

Viewport culling is a count control: it decides how many marks are in play at all. The next technique, shape simplification, is a cost-per-mark control: it decides how expensive each surviving mark is to draw.

A detailed tissue outline is normally drawn with many points, and every point costs work to store and render. Shape simplification keeps only a few of those points — enough that the simpler shape still follows the same size and location as the original — and, for very small marks, drops the outline entirely and draws a single dot at the shape's centre. The number of points, the vertex count, drops as you zoom out.

Drawing a complex outline takes a lot of points and processing; keeping fewer points, or replacing an outline with a dot or a basic shape, cuts the work while still showing where things are and roughly how big they are. Culling reduced the count; simplification reduced the per-mark cost. The last technique attacks the case those two cannot: a region so crowded that even simplified marks, all on screen, are too many to draw.

Some regions hold hundreds or thousands of similar marks. Drawing each one — even as a dot — is wasteful and looks like clutter. Clustering spots the crowd and replaces it with one summary marker plus an outline of the area. To find the crowds cheaply, the method uses a QuadTree: a tree that recursively subdivides space into four quadrants, subdividing again only where marks are dense, so "what is near here?" stays cheap no matter how many marks there are.

Walk the tree and each dense node becomes a single cluster marker rather than a pile of individual marks:

The same idea can emphasise the edges between regions instead of the interiors — highlighting the boundaries where one type of tissue meets another — so the pathologist gets the picture without the clutter or the heavy load.

The device is not just a viewer; it doubles as a touchscreen tool with multi-touch and stylus input. The pathologist adds, edits, and deletes marks directly with a finger or stylus, and a small gesture vocabulary maps naturally onto the actions:

Here is the subtle part. The pathologist is editing the simplified view — the culled, simplified, possibly clustered version the device chose to draw. So an edit cannot just change what is on screen; it has to change the real thing. Every change is traced back to the full-resolution dataset and written to the original source record, so nothing is lost.

This is what makes the whole adaptive-rendering method safe for diagnosis: the pathologist works on a light-weight picture, but the system always reconciles their edits against the authoritative data.

Tying the four techniques together is one governor: the device itself. Before deciding anything, the method measures the device's power — memory, CPU, and screen — and turns that into a detail budget. The budget is what each technique reads: how aggressively to cull, how few vertices to keep, when to collapse a crowd into a cluster, and which level of detail the current zoom is allowed.

The result is one rule applied four ways: show only what fits the device. A weaker tablet keeps fewer marks, drawn more simply, clustered sooner, at a lower level of detail — and still shows the same overall diagnostic picture as a workstation. That single idea, made precise across these four techniques and the write-back path, is patent P91594EP.

• Patent P91594EP covers adaptive rendering of gigapixel annotated medical images on ordinary devices — "show only what fits the device."
• The core claim is level-of-detail: detail scales with zoom — low zoom draws a heatmap, mid zoom simplified shapes, high zoom full polygons.
• Viewport culling is a count control: draw only marks inside the viewport, unload the rest, preload neighbours, keep fewer on weaker devices.
• Shape simplification is a cost-per-mark control: fewer vertices per shape as you zoom out, and tiny marks collapse to a single centre dot.
• Clustering uses a QuadTree to find dense crowds cheaply and replace each with one summary marker plus an area outline, or to highlight tissue-type boundaries.
• On-screen stylus/finger editing (tap = add, press & hold = reclassify, eraser, multi-select, drag vertex) writes every change back to the full-resolution source record.
• One governor drives all four: the device is measured (memory, CPU, screen) into a detail budget that each technique reads.

• [ ] I can state what patent P91594EP claims in one line.
• [ ] I can explain the level-of-detail ladder (heatmap → simplified → full polygon) and how it tracks zoom.
• [ ] I can define viewport culling and explain the neighbour-preload and weaker-device behaviour.
• [ ] I can explain shape simplification and when a mark becomes a single dot.
• [ ] I can explain how a QuadTree finds dense crowds and how clustering replaces them with a summary marker.
• [ ] I can describe the stylus gesture vocabulary and why edits must resolve back to the source dataset.
• [ ] I can explain how a measured device budget governs culling, simplification, clustering, and level of detail together.