8.9 KiB

Raw Blame History

Migration Guide: Monolith → Modular Architecture

🎯 Why Migrate?

Problems with Monolith

❌ Single file > 800 lines - hard to navigate
❌ Mixed responsibilities - S3, SQS, SMTP, DynamoDB all in one place
❌ Hard to test - can't test components in isolation
❌ Difficult to debug - errors could be anywhere
❌ Critical bugs - signalIGINT typo, missing audit trail
❌ Performance issues - N DynamoDB calls for N recipients

Benefits of Modular

✅ Separation of Concerns - each module has one job
✅ Easy to Test - mock S3Handler, test in isolation
✅ Better Performance - batch DynamoDB calls
✅ Maintainable - changes isolated to specific files
✅ Extensible - easy to add new features
✅ Bug Fixes - all critical bugs fixed

🔄 Migration Steps

Step 1: Backup Current Setup

# Backup monolith
cp unified_worker.py unified_worker.py.backup

# Backup any configuration
cp .env .env.backup

Step 2: Clone New Structure

# Download modular version
git clone <repo> email-worker-modular
cd email-worker-modular

# Copy environment variables
cp .env.example .env
# Edit .env with your settings

Step 3: Update Configuration

The modular version uses the SAME environment variables, so your existing .env should work:

# No changes needed to these:
AWS_REGION=us-east-2
DOMAINS=example.com,another.com
SMTP_HOST=localhost
SMTP_PORT=25
# ... etc

New variables (optional):

# For internal delivery (bypasses transport_maps)
INTERNAL_SMTP_PORT=2525

# For blocklist feature
DYNAMODB_BLOCKED_TABLE=email-blocked-senders

Step 4: Install Dependencies

pip install -r requirements.txt

Step 5: Test Locally

# Run worker
python3 main.py

# Check health endpoint
curl http://localhost:8080/health

# Check metrics
curl http://localhost:8000/metrics

Step 6: Deploy

Docker Deployment

# Build image
docker build -t unified-email-worker:latest .

# Run with docker-compose
docker-compose up -d

# Check logs
docker-compose logs -f email-worker

Systemd Deployment

# Create systemd service
sudo nano /etc/systemd/system/email-worker.service

[Unit]
Description=Unified Email Worker
After=network.target

[Service]
Type=simple
User=worker
WorkingDirectory=/opt/email-worker
EnvironmentFile=/opt/email-worker/.env
ExecStart=/usr/bin/python3 /opt/email-worker/main.py
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

# Enable and start
sudo systemctl enable email-worker
sudo systemctl start email-worker
sudo systemctl status email-worker

Step 7: Monitor Migration

# Watch logs
tail -f /var/log/syslog | grep email-worker

# Check metrics
watch -n 5 'curl -s http://localhost:8000/metrics | grep emails_processed'

# Monitor S3 metadata
aws s3api head-object \
  --bucket example-com-emails \
  --key <message-id> \
  --query Metadata

🔍 Verification Checklist

After migration, verify all features work:

Email Delivery

# Send test email via SES
# Check it arrives in mailbox

Bounce Rewriting

# Trigger a bounce (send to invalid@example.com)
# Verify bounce comes FROM the failed recipient

Auto-Reply (OOO)

# Set OOO in DynamoDB:
aws dynamodb put-item \
  --table-name email-rules \
  --item '{"email_address": {"S": "test@example.com"}, "ooo_active": {"BOOL": true}, "ooo_message": {"S": "I am away"}}'

# Send email to test@example.com
# Verify auto-reply received

Forwarding

# Set forward rule:
aws dynamodb put-item \
  --table-name email-rules \
  --item '{"email_address": {"S": "test@example.com"}, "forwards": {"L": [{"S": "other@example.com"}]}}'

# Send email to test@example.com
# Verify other@example.com receives forwarded email

Blocklist

# Block sender:
aws dynamodb put-item \
  --table-name email-blocked-senders \
  --item '{"email_address": {"S": "test@example.com"}, "blocked_patterns": {"L": [{"S": "spam@*.com"}]}}'

# Send email from spam@bad.com to test@example.com
# Verify email is blocked (not delivered, S3 deleted)

Metrics

curl http://localhost:8000/metrics | grep emails_processed

Health Check
```
curl http://localhost:8080/health | jq
```

🐛 Troubleshooting Migration Issues

Issue: Worker not starting

# Check Python version
python3 --version  # Should be 3.11+

# Check dependencies
pip list | grep boto3

# Check logs
python3 main.py  # Run in foreground to see errors

Issue: No emails processing

# Check queue URLs
curl http://localhost:8080/domains

# Verify SQS permissions
aws sqs list-queues

# Check worker logs for errors
tail -f /var/log/email-worker.log

Issue: Bounces not rewriting

# Verify DynamoDB table exists
aws dynamodb describe-table --table-name ses-outbound-messages

# Check if Lambda is writing bounce records
aws dynamodb scan --table-name ses-outbound-messages --limit 5

# Verify worker can read DynamoDB
# (Check logs for "DynamoDB tables connected successfully")

Issue: Performance degradation

# Check if batch calls are used
grep "batch_get_blocked_patterns" main.py  # Should exist in modular version

# Monitor DynamoDB read capacity
aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ConsumedReadCapacityUnits \
  --dimensions Name=TableName,Value=email-blocked-senders \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 300 \
  --statistics Sum

📊 Comparison: Before vs After

Feature	Monolith	Modular	Improvement
Lines of Code	800+ in 1 file	~150 per file	✅ Easier to read
DynamoDB Calls	N per message	1 per message	✅ 10x faster
Error Handling	Missing in places	Comprehensive	✅ More reliable
Testability	Hard	Easy	✅ Can unit test
Audit Trail	Incomplete	Complete	✅ Better compliance
Bugs Fixed	-	4 critical	✅ More stable
Extensibility	Hard	Easy	✅ Future-proof

🎓 Code Comparison Examples

Example 1: Blocklist Check

Monolith (Inefficient):

for recipient in recipients:
    if is_sender_blocked(recipient, sender, worker_name):
        # DynamoDB call for EACH recipient!
        blocked_recipients.append(recipient)

Modular (Efficient):

# ONE DynamoDB call for ALL recipients
blocked_by_recipient = blocklist.batch_check_blocked_senders(
    recipients, sender, worker_name
)
for recipient in recipients:
    if blocked_by_recipient[recipient]:
        blocked_recipients.append(recipient)

Example 2: S3 Blocked Email Handling

Monolith (Missing Audit Trail):

if all_blocked:
    s3.delete_object(Bucket=bucket, Key=key)  # ❌ No metadata!

Modular (Proper Audit):

if all_blocked:
    s3.mark_as_blocked(domain, key, blocked, sender, worker)  # ✅ Set metadata
    s3.delete_blocked_email(domain, key, worker)              # ✅ Then delete

Example 3: Signal Handling

Monolith (Bug):

signal.signal(signal.SIGTERM, handler)
signal.signal(signalIGINT, handler)  # ❌ Typo! Should be signal.SIGINT

Modular (Fixed):

signal.signal(signal.SIGTERM, handler)
signal.signal(signal.SIGINT, handler)  # ✅ Correct

🔄 Rollback Plan

If you need to rollback:

# Stop new worker
docker-compose down
# or
sudo systemctl stop email-worker

# Restore monolith
cp unified_worker.py.backup unified_worker.py

# Restart old worker
python3 unified_worker.py
# or restore old systemd service

💡 Best Practices After Migration

Monitor Metrics: Set up Prometheus/Grafana dashboards
Set up Alerts: Alert on queue buildup, high error rates
Regular Updates: Keep dependencies updated
Backup Rules: Export DynamoDB rules regularly
Test in Staging: Always test rule changes in non-prod first

📚 Additional Resources

ARCHITECTURE.md - Detailed architecture diagrams
README.md - Complete feature documentation
Makefile - Common commands

❓ FAQ

Q: Will my existing DynamoDB tables work?
A: Yes! Same schema, just need to add email-blocked-senders table for blocklist feature.

Q: Do I need to change my Lambda functions?
A: No, bounce tracking Lambda stays the same.

Q: Can I migrate one domain at a time?
A: Yes! Run both workers with different DOMAINS settings, then migrate gradually.

Q: What about my existing S3 metadata?
A: New worker reads and writes same metadata format, fully compatible.

Q: How do I add new features?
A: Just add a new module in appropriate directory (e.g., new file in email/), import in worker.py.

8.9 KiB Raw Blame History