EKM System Design for Writer Inc.

1. What Is EKM and Why It Matters

Enterprise Key Management (EKM) gives each customer control over the encryption keys that protect their data inside a SaaS product. Slack popularised the model: the customer creates a CMK (Customer Master Key) in their own AWS account; Slack uses that key -- via cross-account IAM grants -- to wrap and unwrap per-tenant Data Encryption Keys (DEKs) at rest. If the customer revokes the IAM role, Slack loses the ability to decrypt and data becomes inaccessible within the DEK cache TTL.

For Writer Inc., a 100-person company building enterprise LLMs, the same semantics apply to prompts, completions, fine-tuning datasets, model artifacts, and conversation history.

2. EKM Design Options (A - D)

3. Recommended Architecture (Slack-Style EKM)

The recommended Slack-style EKM architecture consists of these components:

┌──────────────────────────────────────────────────────────────┐
│  Customer AWS / GCP Account                                  │
│  ┌────────────┐   IAM role / service-account grant           │
│  │  CMK (KMS) │ ◄──── Writer Crypto Service calls            │
│  └────────────┘       GenerateDataKey / Decrypt              │
│  CloudTrail / Cloud Audit Logs                               │
└──────────────────────────────────────────────────────────────┘
                          │
           wrap/unwrap DEK│
                          ▼
┌──────────────────────────────────────────────────────────────┐
│  Writer Infrastructure                                       │
│                                                              │
│  ┌─────────────────┐    ┌───────────────────────────┐        │
│  │  Writer App      │──► │  Crypto Service            │       │
│  │  (LLM Platform)  │    │  • envelope encrypt/decrypt│       │
│  └─────────────────┘    │  • DEK cache (TTL 5-15 min)│       │
│                          │  • circuit breaker          │       │
│                          │  • audit log                │       │
│                          └───────────┬───────────────┘        │
│                                      │                        │
│                          ┌───────────▼───────────────┐        │
│                          │  Global Key Registry       │       │
│                          │  tenant → cloud, key_arn,  │       │
│                          │  status, rotation_schedule │       │
│                          └───────────┬───────────────┘        │
│                                      │                        │
│                          ┌───────────▼───────────────┐        │
│                          │  Encrypted Tenant Data     │       │
│                          │  S3/GCS, RDS/CloudSQL,     │       │
│                          │  prompts, completions,     │       │
│                          │  fine-tuning data          │       │
│                          └───────────────────────────┘        │
└──────────────────────────────────────────────────────────────┘
            

Envelope Encryption Flow

1. Write path: Crypto Service calls customer KMS GenerateDataKey → receives plaintext DEK + wrapped DEK. Encrypt data with plaintext DEK (AES-256-GCM). Store wrapped DEK alongside ciphertext. Wipe plaintext DEK from memory.
2. Read path: Check DEK cache. On miss, call customer KMS Decrypt with wrapped DEK → get plaintext DEK. Decrypt data. Cache plaintext DEK (TTL 5 - 15 min).
3. Revoke path: Customer removes IAM role → next cache miss fails → data inaccessible within cache TTL.

Implementation Milestones

4. Multi-Cloud Strategy (AWS + GCP)

When Writer has tenant deployments in both GCP and AWS, EKM becomes a multi-cloud CMK problem. Three viable designs exist:

Product Changes for Multi-Cloud

• Global Key Registry -- central DB table: tenant_id, cloud_provider, key_arn_or_resource_id, status, rotation_schedule, created_at
• Unified Envelope Format -- ciphertext header includes cloud_provider field so any service can route to the correct KMS
• Two Onboarding Packs -- Terraform module for AWS customers, gcloud / Terraform module for GCP customers

Pricing Reference (Multi-Cloud)

5. Top Third-Party Systems

Two third-party platforms stand out for providing multi-cloud EKM / HYOK posture without building everything in-house:

6. Full Comparison: All Approaches

Beyond the two top third-party systems, several other approaches and tools were evaluated:

WorkOS Vault -- System Design

Acra -- System Design

7. Decision Guidance for Writer

Packaging Suggestions

8. Operational Considerations

• Key Rotation: Automate annual CMK rotation. DEKs re-wrapped with new CMK version; data re-encryption is not needed (envelope model).
• Availability: DEK cache (TTL 5 - 15 min) ensures continued operation during brief KMS outages. Circuit breaker prevents cascading failures.
• Audit: Every wrap/unwrap call logged in customer's CloudTrail / Cloud Audit Logs. Writer maintains internal audit log for compliance reporting.
• Onboarding: Provide IaC templates (Terraform / CloudFormation) that create IAM role + CMK policy. Target: customer completes setup in < 30 minutes.
• Incident Response: If customer revokes access, Writer surfaces a clear “data unavailable -- contact your admin” message. No data loss; re-grant IAM role restores access.
• Performance: Expect ~2-5ms overhead for KMS calls on cache miss. With 5 - 15 min TTL, 99%+ of operations served from cache.