From Procedural Chaos to MVC: Refactoring for Testability

Key Takeaway

Our budget manager was a single 100-line handler.py with all logic mixed together. Refactoring to MVC architecture with services, handlers, and configuration layers reduced code duplication by 90%, enabled comprehensive testing, and improved maintainability.

The Problem

Original monolithic structure:

# handler.py (everything in one file)
def lambda_handler(event, context):
    # AWS client creation
    s3 = boto3.client('s3')
    ecs = boto3.client('ecs')
    slack_url = os.environ['SLACK_WEBHOOK']

    # Business logic
    bucket = s3.list_buckets()
    # ... 80 more lines of mixed concerns ...

    # Notification
    requests.post(slack_url, json={'text': message})

Issues:

• Hard to test (AWS clients coupled to logic)
• Code duplication across functions
• No separation of concerns
• Difficult to extend

The Solution

MVC architecture with clear separation:

src/
├── lambda/handlers/          # Entry points
│   ├── s3_monitor_handler.py
│   ├── sqs_monitor_handler.py
│   ├── fargate_monitor_handler.py
│   └── base_handler.py       # Common patterns
├── services/                 # Business logic
│   ├── aws/
│   │   ├── s3_service.py
│   │   ├── sqs_service.py
│   │   └── ecs_service.py
│   └── notification/
│       └── slack_service.py
├── config/                   # Configuration
│   ├── config_validator.py
│   └── monitoring/
└── core/                     # Shared utilities
    ├── exceptions.py
    └── interfaces.py

Example refactored handler:

# lambda/handlers/base_handler.py
class BaseHandler:
    """Base handler with common error handling"""

    def __init__(self, logger):
        self.logger = logger

    def handle(self, event, context):
        try:
            result = self._process_event(event)
            return {'statusCode': 200, 'body': json.dumps(result)}

        except MonitoringError as e:
            self.logger.error(f"Monitoring error: {e}")
            return {'statusCode': 500, 'body': json.dumps({'error': str(e)})}

        except Exception as e:
            self.logger.error(f"Unexpected error: {e}")
            return {'statusCode': 500, 'body': json.dumps({'error': 'Internal error'})}

    def _process_event(self, event):
        raise NotImplementedError

# lambda/handlers/s3_monitor_handler.py
class S3MonitorHandler(BaseHandler):
    def __init__(self):
        super().__init__(logging.getLogger(__name__))
        self.s3_service = S3Service()
        self.slack_service = SlackService()

    def _process_event(self, event):
        # Business logic using services
        storage_size = self.s3_service.get_bucket_size(bucket_name)

        if storage_size > threshold:
            self.slack_service.send_alert(f'S3 storage exceeded: {storage_size}GB')

        return {'storage_size': storage_size}

# Entry point
def lambda_handler(event, context):
    handler = S3MonitorHandler()
    return handler.handle(event, context)

Service layer:

# services/aws/s3_service.py
class S3Service:
    def __init__(self):
        self.client = boto3.client('s3')

    def get_bucket_size(self, bucket_name):
        """Get total size of bucket in GB"""
        cloudwatch = boto3.client('cloudwatch')

        metric = cloudwatch.get_metric_statistics(
            Namespace='AWS/S3',
            MetricName='BucketSizeBytes',
            Dimensions=[
                {'Name': 'BucketName', 'Value': bucket_name},
                {'Name': 'StorageType', 'Value': 'StandardStorage'}
            ],
            StartTime=datetime.utcnow() - timedelta(days=1),
            EndTime=datetime.utcnow(),
            Period=86400,
            Statistics=['Average']
        )

        bytes_size = metric['Datapoints'][0]['Average']
        return bytes_size / (1024 ** 3)  # Convert to GB

Implementation Details

Testing Benefits

Services can be unit tested independently:

def test_s3_service_get_bucket_size():
    # Mock boto3 client
    with patch('boto3.client') as mock_client:
        mock_cloudwatch = Mock()
        mock_cloudwatch.get_metric_statistics.return_value = {
            'Datapoints': [{'Average': 5368709120}]  # 5GB
        }
        mock_client.return_value = mock_cloudwatch

        service = S3Service()
        size = service.get_bucket_size('test-bucket')

        assert size == 5.0

Handlers can be tested with mock services:

def test_s3_monitor_handler():
    handler = S3MonitorHandler()

    # Inject mock services
    handler.s3_service = Mock()
    handler.s3_service.get_bucket_size.return_value = 100

    handler.slack_service = Mock()

    result = handler._process_event({})

    # Verify alert sent
    handler.slack_service.send_alert.assert_called_once()

Impact and Results

• Code Duplication: Reduced from ~100 lines repeated to ~10 lines in base handler
• Test Coverage: 0% → 85%
• Maintainability: Changes isolated to specific services
• Team Velocity: New features added 3x faster

Lessons Learned

1. Separate Concerns: Handlers, services, and configuration should be distinct
2. Test-Driven Refactoring: Refactoring enables testing enables confidence
3. Base Classes: Common patterns extracted to base handlers reduce duplication
4. Service Layer: Business logic separate from Lambda entry points

Refactoring monolithic Lambda functions to MVC architecture dramatically improves testability, maintainability, and code quality. The upfront investment pays dividends in development velocity and system reliability.