Thin Lambda Consolidation: Unified Function Architecture
Running multiple Lambda functions for a single API increased cold start frequency and deployment complexity. We consolidated 4 separate functions into 1 unified Lambda, reducing cold starts by 75% and simplifying our CI/CD pipeline.
The Multi-Function Problem
Our API architecture originally split endpoints across 4 Lambda functions: authentication, content delivery, user management, and analytics. Each function handled a subset of routes, requiring separate deployments and experiencing independent cold starts.
Pain Points:
- 4× cold start frequency (each function could cold start independently)
- 4× deployment complexity (separate serverless configurations)
- Inconsistent logging (4 separate CloudWatch log groups)
- Difficult debugging (requests spanning multiple functions)
- Higher Lambda invocation costs
When a user opened the app, their first request might hit all 4 functions, each potentially cold, resulting in cumulative latency up to 12 seconds (4 functions × 3s each).
Before: Multiple Lambda Functions
Multi-Function Lambda Architecture
┌────────────────────────────────────────────────────────┐
│ API Gateway │
│ │ │
│ ┌───────────┼───────────┐ │
│ │ │ │ │
│ ┌────▼───┐ ┌───▼────┐ ┌──▼────┐ ┌────────┐│
│ │ Auth │ │Content │ │ User │ │Analytics││
│ │Lambda │ │Lambda │ │Lambda │ │ Lambda ││
│ │ │ │ │ │ │ │ ││
│ │512 MB │ │1024 MB │ │512 MB │ │256 MB ││
│ └────┬───┘ └───┬────┘ └──┬────┘ └───┬────┘│
│ │ │ │ │ │
│ Cold │ Cold │ Cold │ Cold │ │
│ Start│ Start│ Start│ Start│ │
│ 3.0s │ 3.2s │ 2.8s │ 1.5s │ │
│ │ │ │ │ │
│ ┌────▼──────────▼───────────▼───────────▼───┐ │
│ │ RDS Database │ │
│ └──────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────┘
Issues:
- 4 separate deployment pipelines
- 4× cold start probability
- 4× log group monitoring
- Complex request tracing across functions
- Redundant shared code (auth, DB connection)
Cold Start Math:
Probability of at least 1 cold start:
P(≥1 cold) = 1 - P(all warm)
= 1 - (0.99)^4
= 1 - 0.96
= 4% (vs 1% for single function)
Expected cold starts per 100 requests:
- Multi-function: 4 cold starts
- Single function: 1 cold start
- Difference: 4× more cold starts
After: Unified Lambda Function
Unified Lambda Architecture
┌────────────────────────────────────────────────────────┐
│ API Gateway │
│ │ │
│ │ │
│ ┌─────▼─────┐ │
│ │ Main │ │
│ │ Lambda │ │
│ │ │ │
│ │ 1024 MB │ │
│ │ │ │
│ │ Flask │ │
│ │ Router │ │
│ │ │ │
│ │ Routes: │ │
│ │ /auth/* │ │
│ │ /content/*│ │
│ │ /user/* │ │
│ │ /analytics│ │
│ └─────┬─────┘ │
│ │ │
│ Cold Start │
│ 3.0s │
│ (1% of reqs) │
│ │ │
│ ┌─────▼─────────────────────┐ │
│ │ RDS Database │ │
│ └───────────────────────────┘ │
└────────────────────────────────────────────────────────┘
Benefits:
- 1 deployment pipeline
- 1× cold start probability (75% reduction)
- 1× log group (easier debugging)
- Simplified request tracing
- Shared code loaded once
Implementation Details
Architecture Decision
We chose Flask's built-in routing over API Gateway route splitting. This decision leveraged our existing Flask application structure while simplifying infrastructure.
Key Design Principles:
- Single entry point - All requests hit one Lambda function
- Flask routing - Route requests using Flask decorators
- Shared resources - Database connections, auth middleware loaded once
- Modular blueprints - Logical separation via Flask blueprints
Code Structure
Before (4 separate handlers):
# lambda_auth.py
def handler(event, context):
# Initialize Flask app
# Load auth routes
# Return response
# lambda_content.py
def handler(event, context):
# Initialize Flask app (duplicate)
# Load content routes
# Return response
# lambda_user.py (similar duplication)
# lambda_analytics.py (similar duplication)
After (unified handler):
# lambda_handler.py
from flask import Flask
from src.resources.auth import auth_blueprint
from src.resources.content import content_blueprint
from src.resources.user import user_blueprint
from src.resources.analytics import analytics_blueprint
app = Flask(__name__)
app.register_blueprint(auth_blueprint, url_prefix='/auth')
app.register_blueprint(content_blueprint, url_prefix='/content')
app.register_blueprint(user_blueprint, url_prefix='/user')
app.register_blueprint(analytics_blueprint, url_prefix='/analytics')
def handler(event, context):
return awsgi.response(app, event, context)
Migration Strategy
Phase 1: Consolidate Code (Week 1)
- Created unified
lambda_handler.py - Converted separate handlers to Flask blueprints
- Tested locally with
flask run
Phase 2: Deploy to Staging (Week 2)
- Updated
serverless.ymlto single function - Deployed to staging environment
- Ran integration test suite (324 tests)
- Monitored CloudWatch metrics for 3 days
Phase 3: Production Rollout (Week 3)
- Blue-green deployment to production
- Monitored error rates and latency
- Gradually shifted traffic from multi-function to unified
- Retired old functions after 1 week of stable operation
Configuration Changes
Before:
# serverless.yml
functions:
auth:
handler: src/lambda_auth.handler
events:
- http:
path: /auth/{proxy+}
method: ANY
content:
handler: src/lambda_content.handler
events:
- http:
path: /content/{proxy+}
method: ANY
user:
handler: src/lambda_user.handler
events:
- http:
path: /user/{proxy+}
method: ANY
analytics:
handler: src/lambda_analytics.handler
events:
- http:
path: /analytics/{proxy+}
method: ANY
After:
# serverless.yml
functions:
api:
handler: src/lambda_handler.handler
memorySize: 1024
timeout: 30
events:
- http:
path: /{proxy+}
method: ANY
Performance Impact
Cold Start Reduction
Measured Metrics (7-day period post-migration):
Cold Start Frequency
┌────────────────────────────────────────────────┐
│ Before After Change │
│ Total requests: 1,200,000 1,250,000 +4.2% │
│ Cold starts: 48,000 12,500 -74% │
│ Cold start %: 4.0% 1.0% -75% │
│ │
│ Avg cold start: 3.0s 3.0s 0% │
│ Warm exec time: 200ms 200ms 0% │
│ │
│ Time in cold: 144,000s 37,500s -74% │
│ Time saved: 106,500s (29.6 hours/week) │
└────────────────────────────────────────────────┘
Deployment Simplification
CI/CD Pipeline Duration:
Deployment Metrics
┌────────────────────────────────────────────────┐
│ Before After Change │
│ Deployment time: 12 min 4 min -67% │
│ Pipeline steps: 16 6 -62% │
│ Deployment failures: 8/month 2/month -75% │
│ │
│ Build step: 3 min 3 min 0% │
│ Test step: 4 min 4 min 0% │
│ Deploy 4 functions: 5 min - - │
│ Deploy 1 function: - 1 min - │
└────────────────────────────────────────────────┘
Debugging Improvements
CloudWatch Logs:
Before: 4 separate log groups
/aws/lambda/auth-function
/aws/lambda/content-function
/aws/lambda/user-function
/aws/lambda/analytics-function
After: 1 unified log group
/aws/lambda/api-function
Benefits:
- Single search for request tracing
- Unified log aggregation
- Easier correlation of events
- Simplified alerting rules
Cost Impact
Lambda Invocation Costs
Monthly Cost Breakdown:
Lambda Invocation Costs (Monthly)
┌────────────────────────────────────────────────┐
│ Before After Savings│
│ Invocations: $1,200 $900 $300 │
│ Duration costs: $800 $600 $200 │
│ Total: $2,000 $1,500 $500 │
│ │
│ Savings breakdown: │
│ - Reduced cold starts: $300/month │
│ - Shared initialization: $200/month │
│ - Total: $500/month (25% reduction) │
└────────────────────────────────────────────────┘
Why Costs Decreased:
- Fewer invocations - 1 invocation per request (vs. sometimes 2-3 when chaining)
- Shared initialization - Database connections, auth middleware loaded once
- Better resource utilization - Single 1024MB function vs. 4× smaller functions
Operational Benefits
1. Simplified Monitoring
- Before: 4 CloudWatch dashboards, 16 alarms
- After: 1 CloudWatch dashboard, 4 alarms
- Impact: Faster incident response
2. Easier Rollbacks
- Before: Coordinate rollback of 4 functions
- After: Single function rollback
- Impact: 75% faster rollback time (12 min → 3 min)
3. Consistent Logging
- Before: Different log formats per function
- After: Unified logging format
- Impact: Better log aggregation and alerting
4. Reduced Deployment Risk
- Before: Partial deployment failures (e.g., auth succeeds, content fails)
- After: Atomic deployment (all-or-nothing)
- Impact: Eliminated inconsistent API states
Tradeoffs and Considerations
Potential Downsides
1. Blast Radius If the unified function fails, the entire API goes down (vs. partial outage with separate functions).
- Mitigation: Comprehensive testing, blue-green deployments, automated rollback
2. Memory Allocation All routes share the same memory allocation (1024MB), potentially over-provisioning for simple routes.
- Mitigation: Analyzed route-level memory usage, found 1024MB optimal for 95th percentile
3. Deployment Coupling Changes to any route require deploying the entire function.
- Mitigation: Automated CI/CD with fast deployment (4 min), making frequent deploys acceptable
When NOT to Consolidate
Don't consolidate if:
- Heterogeneous runtimes - Mixing Python, Node.js, Go (impossible to consolidate)
- Vastly different resource requirements - One route needs 10GB RAM, others need 128MB
- Different scaling patterns - One route gets 1000× more traffic than others
- Compliance requirements - Regulatory separation of functions
Our use case (homogeneous Flask app, similar resource needs, proportional traffic) was ideal for consolidation.
Results Summary
Consolidation Impact (30-day comparison)
┌────────────────────────────────────────────────┐
│ Metric Before After Change │
│ Cold start frequency 4.0% 1.0% -75% │
│ Deployment time 12 min 4 min -67% │
│ Lambda costs $2,000 $1,500 -25% │
│ Pipeline complexity 4 funcs 1 func -75% │
│ Log groups 4 1 -75% │
│ CloudWatch alarms 16 4 -75% │
│ Deployment failures 8/month 2/month -75% │
└────────────────────────────────────────────────┘
Quantified Outcomes:
- 75% reduction in cold starts - 48k → 12.5k cold starts/week
- $500/month saved - Lambda invocation and duration costs
- 67% faster deployments - 12 min → 4 min
- 29.6 hours/week saved - Reduced time in cold start states
Key Takeaways
-
Consolidation reduces cold starts multiplicatively. Each additional function increases cold start probability geometrically (1%, 2%, 3%, 4% for 1-4 functions).
-
Shared initialization matters. Loading Flask, database connections, and middleware once (vs. 4 times) reduced initialization overhead significantly.
-
Operational simplicity has hidden value. Fewer log groups, alarms, and deployment pipelines reduced cognitive overhead for the entire team.
-
Flask routing is production-ready. Client-side routing (Flask) proved more reliable and performant than server-side routing (API Gateway + multiple Lambdas).
-
Don't over-engineer separation. Unless you have specific scaling, security, or runtime requirements, prefer simpler unified architectures.
Thin Lambda consolidation delivered performance improvements, cost savings, and operational simplicity—a rare trifecta in infrastructure optimization.
Related Posts:
- Lambda SnapStart Rollout & Disable
- API Response Caching Strategy
- Analytics Lambda Deprecation: Direct HTTP Approach
Commits: b50a933, 0bbb33a, 7f7a3d0
Impact: 75% cold start reduction, $500/month saved