email-amazon/email-worker/docs/MIGRATION.md

# Migration Guide: Monolith → Modular Architecture

## 🎯 Why Migrate?

### Problems with Monolith
- ❌ **Single file > 800 lines** - hard to navigate
- ❌ **Mixed responsibilities** - S3, SQS, SMTP, DynamoDB all in one place
- ❌ **Hard to test** - can't test components in isolation
- ❌ **Difficult to debug** - errors could be anywhere
- ❌ **Critical bugs** - `signalIGINT` typo, missing audit trail
- ❌ **Performance issues** - N DynamoDB calls for N recipients

### Benefits of Modular
- ✅ **Separation of Concerns** - each module has one job
- ✅ **Easy to Test** - mock S3Handler, test in isolation
- ✅ **Better Performance** - batch DynamoDB calls
- ✅ **Maintainable** - changes isolated to specific files
- ✅ **Extensible** - easy to add new features
- ✅ **Bug Fixes** - all critical bugs fixed

## 🔄 Migration Steps

### Step 1: Backup Current Setup
```bash
# Backup monolith
cp unified_worker.py unified_worker.py.backup

# Backup any configuration
cp .env .env.backup
```

### Step 2: Clone New Structure
```bash
# Download modular version
git clone <repo> email-worker-modular
cd email-worker-modular

# Copy environment variables
cp .env.example .env
# Edit .env with your settings
```

### Step 3: Update Configuration

The modular version uses the SAME environment variables, so your existing `.env` should work:

```bash
# No changes needed to these:
AWS_REGION=us-east-2
DOMAINS=example.com,another.com
SMTP_HOST=localhost
SMTP_PORT=25
# ... etc
```

**New variables** (optional):
```bash
# For internal delivery (bypasses transport_maps)
INTERNAL_SMTP_PORT=2525

# For blocklist feature
DYNAMODB_BLOCKED_TABLE=email-blocked-senders
```

### Step 4: Install Dependencies
```bash
pip install -r requirements.txt
```

### Step 5: Test Locally
```bash
# Run worker
python3 main.py

# Check health endpoint
curl http://localhost:8080/health

# Check metrics
curl http://localhost:8000/metrics
```

### Step 6: Deploy

#### Docker Deployment
```bash
# Build image
docker build -t unified-email-worker:latest .

# Run with docker-compose
docker-compose up -d

# Check logs
docker-compose logs -f email-worker
```

#### Systemd Deployment
```bash
# Create systemd service
sudo nano /etc/systemd/system/email-worker.service
```

```ini
[Unit]
Description=Unified Email Worker
After=network.target

[Service]
Type=simple
User=worker
WorkingDirectory=/opt/email-worker
EnvironmentFile=/opt/email-worker/.env
ExecStart=/usr/bin/python3 /opt/email-worker/main.py
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
```

```bash
# Enable and start
sudo systemctl enable email-worker
sudo systemctl start email-worker
sudo systemctl status email-worker
```

### Step 7: Monitor Migration
```bash
# Watch logs
tail -f /var/log/syslog | grep email-worker

# Check metrics
watch -n 5 'curl -s http://localhost:8000/metrics | grep emails_processed'

# Monitor S3 metadata
aws s3api head-object \
  --bucket example-com-emails \
  --key <message-id> \
  --query Metadata
```

## 🔍 Verification Checklist

After migration, verify all features work:

- [ ] **Email Delivery**
  ```bash
  # Send test email via SES
  # Check it arrives in mailbox
  ```

- [ ] **Bounce Rewriting**
  ```bash
  # Trigger a bounce (send to invalid@example.com)
  # Verify bounce comes FROM the failed recipient
  ```

- [ ] **Auto-Reply (OOO)**
  ```bash
  # Set OOO in DynamoDB:
  aws dynamodb put-item \
    --table-name email-rules \
    --item '{"email_address": {"S": "test@example.com"}, "ooo_active": {"BOOL": true}, "ooo_message": {"S": "I am away"}}'
  
  # Send email to test@example.com
  # Verify auto-reply received
  ```

- [ ] **Forwarding**
  ```bash
  # Set forward rule:
  aws dynamodb put-item \
    --table-name email-rules \
    --item '{"email_address": {"S": "test@example.com"}, "forwards": {"L": [{"S": "other@example.com"}]}}'
  
  # Send email to test@example.com
  # Verify other@example.com receives forwarded email
  ```

- [ ] **Blocklist**
  ```bash
  # Block sender:
  aws dynamodb put-item \
    --table-name email-blocked-senders \
    --item '{"email_address": {"S": "test@example.com"}, "blocked_patterns": {"L": [{"S": "spam@*.com"}]}}'
  
  # Send email from spam@bad.com to test@example.com
  # Verify email is blocked (not delivered, S3 deleted)
  ```

- [ ] **Metrics**
  ```bash
  curl http://localhost:8000/metrics | grep emails_processed
  ```

- [ ] **Health Check**
  ```bash
  curl http://localhost:8080/health | jq
  ```

## 🐛 Troubleshooting Migration Issues

### Issue: Worker not starting
```bash
# Check Python version
python3 --version  # Should be 3.11+

# Check dependencies
pip list | grep boto3

# Check logs
python3 main.py  # Run in foreground to see errors
```

### Issue: No emails processing
```bash
# Check queue URLs
curl http://localhost:8080/domains

# Verify SQS permissions
aws sqs list-queues

# Check worker logs for errors
tail -f /var/log/email-worker.log
```

### Issue: Bounces not rewriting
```bash
# Verify DynamoDB table exists
aws dynamodb describe-table --table-name ses-outbound-messages

# Check if Lambda is writing bounce records
aws dynamodb scan --table-name ses-outbound-messages --limit 5

# Verify worker can read DynamoDB
# (Check logs for "DynamoDB tables connected successfully")
```

### Issue: Performance degradation
```bash
# Check if batch calls are used
grep "batch_get_blocked_patterns" main.py  # Should exist in modular version

# Monitor DynamoDB read capacity
aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ConsumedReadCapacityUnits \
  --dimensions Name=TableName,Value=email-blocked-senders \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 300 \
  --statistics Sum
```

## 📊 Comparison: Before vs After

| Feature | Monolith | Modular | Improvement |
|---------|----------|---------|-------------|
| Lines of Code | 800+ in 1 file | ~150 per file | ✅ Easier to read |
| DynamoDB Calls | N per message | 1 per message | ✅ 10x faster |
| Error Handling | Missing in places | Comprehensive | ✅ More reliable |
| Testability | Hard | Easy | ✅ Can unit test |
| Audit Trail | Incomplete | Complete | ✅ Better compliance |
| Bugs Fixed | - | 4 critical | ✅ More stable |
| Extensibility | Hard | Easy | ✅ Future-proof |

## 🎓 Code Comparison Examples

### Example 1: Blocklist Check

**Monolith (Inefficient):**
```python
for recipient in recipients:
    if is_sender_blocked(recipient, sender, worker_name):
        # DynamoDB call for EACH recipient!
        blocked_recipients.append(recipient)
```

**Modular (Efficient):**
```python
# ONE DynamoDB call for ALL recipients
blocked_by_recipient = blocklist.batch_check_blocked_senders(
    recipients, sender, worker_name
)
for recipient in recipients:
    if blocked_by_recipient[recipient]:
        blocked_recipients.append(recipient)
```

### Example 2: S3 Blocked Email Handling

**Monolith (Missing Audit Trail):**
```python
if all_blocked:
    s3.delete_object(Bucket=bucket, Key=key)  # ❌ No metadata!
```

**Modular (Proper Audit):**
```python
if all_blocked:
    s3.mark_as_blocked(domain, key, blocked, sender, worker)  # ✅ Set metadata
    s3.delete_blocked_email(domain, key, worker)              # ✅ Then delete
```

### Example 3: Signal Handling

**Monolith (Bug):**
```python
signal.signal(signal.SIGTERM, handler)
signal.signal(signalIGINT, handler)  # ❌ Typo! Should be signal.SIGINT
```

**Modular (Fixed):**
```python
signal.signal(signal.SIGTERM, handler)
signal.signal(signal.SIGINT, handler)  # ✅ Correct
```

## 🔄 Rollback Plan

If you need to rollback:

```bash
# Stop new worker
docker-compose down
# or
sudo systemctl stop email-worker

# Restore monolith
cp unified_worker.py.backup unified_worker.py

# Restart old worker
python3 unified_worker.py
# or restore old systemd service
```

## 💡 Best Practices After Migration

1. **Monitor Metrics**: Set up Prometheus/Grafana dashboards
2. **Set up Alerts**: Alert on queue buildup, high error rates
3. **Regular Updates**: Keep dependencies updated
4. **Backup Rules**: Export DynamoDB rules regularly
5. **Test in Staging**: Always test rule changes in non-prod first

## 📚 Additional Resources

- [ARCHITECTURE.md](ARCHITECTURE.md) - Detailed architecture diagrams
- [README.md](README.md) - Complete feature documentation
- [Makefile](Makefile) - Common commands

## ❓ FAQ

**Q: Will my existing DynamoDB tables work?**  
A: Yes! Same schema, just need to add `email-blocked-senders` table for blocklist feature.

**Q: Do I need to change my Lambda functions?**  
A: No, bounce tracking Lambda stays the same.

**Q: Can I migrate one domain at a time?**  
A: Yes! Run both workers with different `DOMAINS` settings, then migrate gradually.

**Q: What about my existing S3 metadata?**  
A: New worker reads and writes same metadata format, fully compatible.

**Q: How do I add new features?**  
A: Just add a new module in appropriate directory (e.g., new file in `email/`), import in `worker.py`.