Different Types of Data Storage

Every backend eventually becomes a set of decisions about where data lives and what it costs to read or write it. You rarely need to name every database on the market; what matters is picking storage that fits the shape of the data and the way the system touches it. The useful lens is a single question: what kind of data are we storing, how will we read and write it, and what guarantees do we need?

Six forces drive almost every choice, and they trade against each other:

Keep these in mind for the rest of the day. Each storage type below is really just a different answer to the same six questions.

Before reaching for a specific product, walk a short mental tree that branches on the shape of the data and the dominant query. This keeps the choice grounded in how the data behaves rather than in whichever database is fashionable. Read the branches top to bottom and stop at the first one that fits.

A good habit: label each store by the job it does, not only by the product name. "Postgres: source of truth" tells you more than "Postgres."

Most real systems do not pick one store — they place several around a single application service, each doing the one job it is best at. The map below shows the common arrangement: a relational database as the authority, a cache or key-value store for hot lookups, a search index for text, and object storage for large files. Seeing them side by side makes the division of labour obvious.

Useful labels to hang on each box: Postgres — source of truth, Redis — hot cache, locks, sessions, OpenSearch — search projection, S3 — durable object storage, Warehouse — analytics. The rest of this day walks each box in turn.

A relational database stores data as tables of rows with a fixed schema, and it can combine tables with joins. It is the workhorse of correctness-sensitive systems because it offers ACID transactions — Atomicity (all-or-nothing writes), Consistency (the data obeys its rules), Isolation (concurrent transactions don't corrupt each other), and Durability (a committed write survives a crash). Examples: Postgres, MySQL, Aurora, Cloud SQL.

Reach for a relational database when you need strong consistency, multi-row transactions, genuine relationships between entities, joins across those entities, and when correctness matters more than unlimited write throughput. Typical homes for it: users, orders, payments, permissions, account metadata, inventory, financial records.

Rule of thumb: start with a relational database (e.g. Postgres) as the source of truth unless a clear scale or access-pattern reason says otherwise. It keeps the design simple and gives strong correctness guarantees.

When one primary is no longer enough, scale in stages rather than jumping straight to sharding:

A key-value store maps a key directly to a value and is optimized for looking that value up fast — think of a giant, distributed hash map. It gives up rich querying in exchange for very low latency and high throughput. Examples: Redis, DynamoDB, Cassandra, and FoundationDB-style layers.

Use one when you mostly access data by key, need very low latency, have a simple and predictable query pattern, and want high throughput. Common uses: sessions, rate limits, idempotency keys, feature flags, caches, user preferences, shopping-cart state, counters.

Two flavours show up constantly:

• Redis — cache, short-lived state, rate limiting, distributed locks, pub/sub, leaderboards.
• DynamoDB-style — massive-scale key lookups, serverless workloads, event metadata, user settings, high-write workloads.

Rule of thumb: a key-value store shines when the access pattern is predictable. The main risk is designing the key badly and creating a hot partition — one key or key-range that soaks up a disproportionate share of traffic.

A document database stores each record as a self-contained document, usually JSON, with a flexible schema and nested structure. Reads and writes typically fetch or replace a whole document at once, which fits aggregate-shaped data well. Examples: MongoDB, Firestore, Couchbase.

Use one when the data is naturally JSON-like, the schema changes often, reads usually pull an entire document, and you want flexible nested structures. Common uses: user profiles, CMS content, product-catalog metadata, forms, app configuration, semi-structured records.

Rule of thumb: use a document database when a record is usually read and written as one aggregate. If you need complex relationships or multi-entity transactions, prefer relational storage.

A search index is a store built for text: it tokenizes content and builds an inverted index so it can answer relevance-ranked, fuzzy, and faceted queries quickly. It is almost never the source of truth — it is a derived view fed from a primary database. Examples: Elasticsearch, OpenSearch, Solr, Meilisearch.

Reach for one when users need full-text search, relevance ranking, faceted filtering, autocomplete, or fuzzy matching. Common uses: search boxes, product search, document search, log search, support-ticket search.

The standard pattern keeps the primary database authoritative and updates the index asynchronously — the write path stays fast and does not depend on the index being healthy.

Feeding the index asynchronously keeps writes fast, avoids coupling database writes to the search system's availability, and leaves room to re-index from scratch. The trade-off is that search results are eventually consistent — a brand-new record may take a moment to appear.

Rule of thumb: keep the primary database as the source of truth and feed the search index asynchronously. That buys fast search without putting search infrastructure on the critical write path.

Object storage holds opaque binary blobs — files — addressed by a key, with effectively unlimited capacity and very high durability. It is cheap and scalable but cannot query inside the objects, so it is paired with a database that holds the metadata. Examples: S3, GCS, Azure Blob.

Use it for images, videos, PDFs, uploads, logs, backups, exports, model artifacts, and any large file. The canonical upload pattern keeps big file bytes off the application server by issuing a signed URL — a short-lived, permissioned link the client uploads through directly:

This works because the application server never handles the large bytes, uploads scale independently, and the metadata stays queryable in the database.

Rule of thumb: store the file itself in object storage and keep only metadata, ownership, permissions, and references in the database.

Time-series storage is tuned for data that is append-heavy, indexed by time, and usually queried in time windows — "requests per minute over the last day." It compresses aggressively and supports retention policies so old data ages out automatically. Examples: TimescaleDB, InfluxDB, Prometheus, and ClickHouse for event analytics.

Use it when data is append-heavy, time-indexed, and often aggregated by time window. Common uses: metrics, monitoring, IoT readings, audit events, usage events, billing usage. A representative query:

SELECT service, count(*)
FROM requests
WHERE ts > now() - interval '24 hours'
GROUP BY service, time_bucket('1 minute', ts);

Rule of thumb: use time-series storage for operational metrics. For business-critical audit events, also keep the raw events in durable storage — a metrics system is not always the legal source of truth.

A data warehouse (or data lake) is built for scanning huge volumes of historical data for dashboards, BI, machine learning, and reporting. It accepts latency measured in seconds or minutes in exchange for cheap large scans, and it is kept separate from the production database so heavy analytical queries never slow user-facing traffic. Examples: BigQuery, Snowflake, Redshift, Databricks, S3 + Parquet.

Data usually flows in from the production database via CDC (Change Data Capture — streaming row-level changes out of the database) or an event pipeline:

Do not serve core user-facing requests directly from the warehouse unless the product is analytical by nature.

Rule of thumb: keep production serving storage separate from analytics storage so analytical queries never touch user-facing latency.

A graph database stores data as nodes and edges and is optimized for traversing those edges — following relationships many hops deep, which is slow and awkward in a relational join. Reach for one only when relationships are the primary thing you query. Examples: Neo4j, Neptune, TigerGraph.

Use it when relationships are the core data, queries traverse many edges, and you ask questions like "friends of friends," dependency paths, fraud rings, or recommendations. Common uses: social graphs, knowledge graphs, fraud detection, dependency graphs, recommendation systems, access-control relationship graphs.

Rule of thumb: do not start with a graph database unless relationship traversal is core to the product. If relationships are simple, a relational schema is usually enough.

This distinction underlies almost every multi-store design, so it is worth stating plainly. A source of truth is the authoritative, durable copy of the data; a cache is a fast, disposable copy you can always rebuild from that source. Confusing the two is how systems lose data.

"Store everything in Redis" is the classic mistake — it treats a cache as an authority. The sound version: Redis can cache hot reads, but the source of truth stays in durable storage such as Postgres or DynamoDB.

Most production systems are assembled from a handful of repeating patterns rather than invented from scratch. Knowing these four covers a large share of real designs. Each trades some complexity for speed, scale, or decoupling.

Pattern A — DB + cache. Put a cache in front of the database to serve hot reads. On a miss, read the database, populate the cache, and return.

Use for hot reads, expensive queries, low-latency APIs. Trade-offs: cache-invalidation complexity, possible stale reads, an extra operational component.

Pattern B — DB + async index. Write to the database, emit an event, and a worker updates a derived store (search index, denormalized read model).

Use for search, read models, projections, denormalized views. Trade-offs: eventual consistency, the need for retries and a dead-letter queue (a holding area for events that keep failing), and a re-indexing path.

Pattern C — Object storage + metadata DB. File bytes go to object storage; ownership, permissions, and references go to a relational database. (Diagrammed in the object-storage section above.) Use for uploads, images, PDFs, videos, exports. Trade-offs: two systems must stay consistent, orphaned files need cleanup, and access needs signed URLs or a proxy.

Pattern D — event log + materialized views. Every change is appended to an ordered event log; consumers read the log and build materialized views (precomputed read models). The log is the source of truth and views can be rebuilt by replaying it.

Use for audit-heavy systems, workflows, financial ledgers, event-driven architectures. Trade-offs: a more complex mental model, event versioning, replay semantics, and eventual consistency.

CAP is a small idea that gets over-applied, so use it lightly. It says that when a network partition happens — replicas can no longer talk to each other — a distributed system must choose between Consistency (every read sees the latest correct value) and Availability (every request still gets a response); Partition tolerance is simply the reality that the network can fail. You do not choose partition tolerance; you choose how to behave when a partition occurs.

The practical framing: network failures happen, so for each feature decide whether stale reads are acceptable or whether correctness must win.

• Bank transfer → prefer consistency.
• Social-media like count → availability / eventual consistency is fine.
• Inventory checkout → consistency, or explicit reservation logic.
• User-profile read → eventual consistency is often acceptable.

"Consistency" is not all-or-nothing; there are named points on a spectrum, and different data in the same product can sit at different points. Choosing the right one per feature is one of the highest-leverage storage decisions you make. The three that come up most:

Rule of thumb: require strong consistency for authorization and payment state; accept eventual consistency for search and analytics.

When one database can no longer hold the data or serve the load, you split it: partitioning divides data into pieces, and sharding spreads those pieces across separate machines. The shard key — the field used to decide which shard a row lives on — is the most important decision, because a poor key creates hot partitions or blocks future queries.

Good real-world shard keys are tenant_id, user_id, account_id, or region. A good key is evenly distributed, appears in most queries, is stable over time, and avoids hot partitions. Keys that go wrong:

• Timestamp only — every new write lands on the latest partition.
• Country — one country can dominate all traffic.
• Tenant ID — fine until one tenant is vastly larger than the rest and isn't isolated separately.

Rule of thumb: avoid sharding until you actually need it, but choose IDs and access patterns that won't block sharding later.

The most reliable way to model data is to start from how it will be used, not from an abstract entity diagram. Before choosing tables or keys, answer: what are the top reads and top writes? Which queries must be fast? What can be async? What must be correct immediately, and what can be eventually consistent? The answers point straight at the storage.

Take a messaging system. Its core operations sort cleanly into writes and reads:

• Writes: send a message, store the message, update conversation metadata.
• Reads: load recent conversations, load messages in a conversation, search messages.

Each read or write maps to the store that serves it best:

The design falls out of the access patterns; it isn't chosen up front.

For a large share of product systems, one arrangement is a strong starting point: a relational database for correctness, a cache for hot paths, a queue to decouple slow work, a search index for derived search, and object storage for blobs. Start here and add pieces only when a real need appears, rather than reaching for every technology at once.

Read it as a division of responsibility: the relational database handles correctness, Redis handles hot paths, the queue decouples slow work, search is a derived view, and object storage holds large blobs. The discipline that keeps this clean: start with Postgres as the source of truth, add Redis only when hot reads hurt, add search only for search, add object storage for files, and keep analytics separate. Judgment beats a pile of buzzwords.

A compact reference to scan when deciding where a new kind of data should live. Match the data's dominant behaviour to a row, then confirm nothing in the "avoid when" column applies.

• Choose storage from the data's shape and dominant query, not from product fashion — access pattern, consistency, scale, latency, query shape, and durability are the six forces.
• Default to a relational database as the source of truth; add other stores only when a concrete need appears.
• Keep source of truth and derived stores distinct: caches and search indexes are rebuildable views, never the authority.
• Feed search indexes, analytics, and read models asynchronously so the write path stays fast — at the cost of eventual consistency.
• Store large files in object storage and keep only their metadata in the database.
• Pick a consistency model per feature: strong for payments and permissions, eventual for search and analytics, read-your-writes for a user's own edits.
• Delay sharding until you need it, but choose shard keys that stay evenly distributed and don't block it later.