diff --git a/README.md b/README.md index 47e2af7..480a197 100644 --- a/README.md +++ b/README.md @@ -1,290 +1,290 @@ -# Website Change Detection Monitor - -A SaaS platform that monitors web pages for changes and sends smart alerts with noise filtering. - -## ๐Ÿš€ Features - -### MVP (Current Implementation) -- โœ… User authentication (register/login) -- โœ… Monitor creation and management -- โœ… Automatic page checking at configurable intervals -- โœ… Change detection with hash comparison -- โœ… Email alerts on changes -- โœ… History timeline with snapshots -- โœ… Smart noise filtering (timestamps, cookie banners) -- โœ… Manual check triggering -- โœ… Plan-based limits (free/pro/business) - -### Coming Soon -- Keyword-based alerts -- Element-specific monitoring -- Visual diff viewer -- Slack/Discord integrations -- AI-powered change summaries -- Team collaboration - -## ๐Ÿ“‹ Prerequisites - -- Node.js 18+ and npm -- PostgreSQL 15+ -- Redis 7+ -- Docker (optional, for local development) - -## ๐Ÿ› ๏ธ Installation - -### 1. Clone the repository -```bash -git clone -cd website-monitor -``` - -### 2. Start database services (Docker) -```bash -docker-compose up -d -``` - -This starts: -- PostgreSQL on port 5432 -- Redis on port 6379 - -### 3. Setup Backend - -```bash -cd backend -npm install - -# Copy environment file -cp .env.example .env -# Edit .env with your settings - -# Run migrations -npm run migrate - -# Start development server -npm run dev -``` - -Backend will run on http://localhost:3001 - -### 4. Setup Frontend - -```bash -cd frontend -npm install - -# Start development server -npm run dev -``` - -Frontend will run on http://localhost:3000 - -## ๐Ÿ—„๏ธ Database Schema - -The database includes: -- `users` - User accounts and authentication -- `monitors` - Website monitors configuration -- `snapshots` - Historical page snapshots -- `alerts` - Alert records - -Run migrations: -```bash -cd backend -npm run migrate -``` - -## ๐Ÿ”ง Configuration - -### Backend (.env) -```env -PORT=3001 -DATABASE_URL=postgresql://admin:admin123@localhost:5432/website_monitor -REDIS_URL=redis://localhost:6379 -JWT_SECRET=your-secret-key -SMTP_HOST=smtp.sendgrid.net -SMTP_PORT=587 -SMTP_USER=apikey -SMTP_PASS=your-api-key -``` - -### Frontend (.env.local) -```env -NEXT_PUBLIC_API_URL=http://localhost:3001 -``` - -## ๐Ÿ“– Usage - -### Register an Account -1. Go to http://localhost:3000 -2. Click "Sign up" -3. Enter email and password -4. You'll be logged in automatically - -### Create a Monitor -1. Go to Dashboard -2. Click "+ Add Monitor" -3. Enter URL and select frequency -4. Click "Create Monitor" -5. First check happens immediately - -### View History -1. Click "History" on any monitor -2. See timeline of all checks -3. View changes and errors - -## ๐Ÿ”‘ API Endpoints - -### Authentication -- `POST /api/auth/register` - Register new user -- `POST /api/auth/login` - Login user - -### Monitors (Requires Auth) -- `GET /api/monitors` - List all monitors -- `POST /api/monitors` - Create monitor -- `GET /api/monitors/:id` - Get monitor details -- `PUT /api/monitors/:id` - Update monitor -- `DELETE /api/monitors/:id` - Delete monitor -- `POST /api/monitors/:id/check` - Trigger manual check -- `GET /api/monitors/:id/history` - Get check history - -## ๐Ÿ“Š Plan Limits - -### Free Plan -- 5 monitors -- Minimum 60-minute frequency -- 7-day history retention -- Email alerts only - -### Pro Plan -- 50 monitors -- Minimum 5-minute frequency -- 90-day history retention -- All alert channels - -### Business Plan -- 200 monitors -- Minimum 1-minute frequency -- 1-year history retention -- API access -- Team features - -## ๐Ÿ—๏ธ Architecture - -### Backend -- **Express + TypeScript** - API server -- **PostgreSQL** - Relational database -- **Redis + BullMQ** - Job queue (coming soon) -- **Nodemailer** - Email alerts -- **Axios** - HTTP requests -- **Cheerio** - HTML parsing -- **Diff** - Change detection - -### Frontend -- **Next.js 14** - React framework -- **TypeScript** - Type safety -- **Tailwind CSS** - Styling -- **React Query** - Data fetching -- **Axios** - API client - -## ๐Ÿงช Testing - -```bash -# Backend tests -cd backend -npm test - -# Frontend tests -cd frontend -npm test -``` - -## ๐Ÿ“ Development - -### Running in Development -```bash -# Terminal 1: Start databases -docker-compose up - -# Terminal 2: Backend -cd backend -npm run dev - -# Terminal 3: Frontend -cd frontend -npm run dev -``` - -### Building for Production -```bash -# Backend -cd backend -npm run build -npm start - -# Frontend -cd frontend -npm run build -npm start -``` - -## ๐Ÿš€ Deployment - -### Backend Deployment -- Deploy to Railway, Render, or AWS -- Set environment variables -- Run migrations -- Start with `npm start` - -### Frontend Deployment -- Deploy to Vercel or Netlify -- Set `NEXT_PUBLIC_API_URL` -- Build with `npm run build` - -### Database -- Use managed PostgreSQL (AWS RDS, Railway, Supabase) -- Run migrations before deploying - -## ๐Ÿ› Troubleshooting - -### Database connection error -- Check PostgreSQL is running -- Verify DATABASE_URL is correct -- Run migrations: `npm run migrate` - -### Frontend can't connect to backend -- Check API_URL in .env.local -- Ensure backend is running -- Check CORS settings - -### Email alerts not working -- Configure SMTP settings in backend .env -- For development, use a service like Mailtrap -- For production, use SendGrid or similar - -## ๐Ÿ“š Documentation - -See `/docs` folder for: -- `spec.md` - Complete feature specifications -- `task.md` - Development roadmap -- `actions.md` - User workflows -- `claude.md` - Project context - -## ๐Ÿค Contributing - -1. Fork the repository -2. Create a feature branch -3. Commit your changes -4. Push to the branch -5. Open a Pull Request - -## ๐Ÿ“„ License - -MIT License - see LICENSE file for details - -## ๐Ÿ™ Acknowledgments - -Built with: -- Next.js -- Express -- PostgreSQL -- Tailwind CSS -- TypeScript +# Website Change Detection Monitor + +A SaaS platform that monitors web pages for changes and sends smart alerts with noise filtering. + +## ๐Ÿš€ Features + +### MVP (Current Implementation) +- โœ… User authentication (register/login) +- โœ… Monitor creation and management +- โœ… Automatic page checking at configurable intervals +- โœ… Change detection with hash comparison +- โœ… Email alerts on changes +- โœ… History timeline with snapshots +- โœ… Smart noise filtering (timestamps, cookie banners) +- โœ… Manual check triggering +- โœ… Plan-based limits (free/pro/business) + +### Coming Soon +- Keyword-based alerts +- Element-specific monitoring +- Visual diff viewer +- Slack/Discord integrations +- AI-powered change summaries +- Team collaboration + +## ๐Ÿ“‹ Prerequisites + +- Node.js 18+ and npm +- PostgreSQL 15+ +- Redis 7+ +- Docker (optional, for local development) + +## ๐Ÿ› ๏ธ Installation + +### 1. Clone the repository +```bash +git clone +cd website-monitor +``` + +### 2. Start database services (Docker) +```bash +docker-compose up -d +``` + +This starts: +- PostgreSQL on port 5432 +- Redis on port 6379 + +### 3. Setup Backend + +```bash +cd backend +npm install + +# Copy environment file +cp .env.example .env +# Edit .env with your settings + +# Run migrations +npm run migrate + +# Start development server +npm run dev +``` + +Backend will run on http://localhost:3001 + +### 4. Setup Frontend + +```bash +cd frontend +npm install + +# Start development server +npm run dev +``` + +Frontend will run on http://localhost:3000 + +## ๐Ÿ—„๏ธ Database Schema + +The database includes: +- `users` - User accounts and authentication +- `monitors` - Website monitors configuration +- `snapshots` - Historical page snapshots +- `alerts` - Alert records + +Run migrations: +```bash +cd backend +npm run migrate +``` + +## ๐Ÿ”ง Configuration + +### Backend (.env) +```env +PORT=3001 +DATABASE_URL=postgresql://admin:admin123@localhost:5432/website_monitor +REDIS_URL=redis://localhost:6379 +JWT_SECRET=your-secret-key +SMTP_HOST=smtp.sendgrid.net +SMTP_PORT=587 +SMTP_USER=apikey +SMTP_PASS=your-api-key +``` + +### Frontend (.env.local) +```env +NEXT_PUBLIC_API_URL=http://localhost:3001 +``` + +## ๐Ÿ“– Usage + +### Register an Account +1. Go to http://localhost:3000 +2. Click "Sign up" +3. Enter email and password +4. You'll be logged in automatically + +### Create a Monitor +1. Go to Dashboard +2. Click "+ Add Monitor" +3. Enter URL and select frequency +4. Click "Create Monitor" +5. First check happens immediately + +### View History +1. Click "History" on any monitor +2. See timeline of all checks +3. View changes and errors + +## ๐Ÿ”‘ API Endpoints + +### Authentication +- `POST /api/auth/register` - Register new user +- `POST /api/auth/login` - Login user + +### Monitors (Requires Auth) +- `GET /api/monitors` - List all monitors +- `POST /api/monitors` - Create monitor +- `GET /api/monitors/:id` - Get monitor details +- `PUT /api/monitors/:id` - Update monitor +- `DELETE /api/monitors/:id` - Delete monitor +- `POST /api/monitors/:id/check` - Trigger manual check +- `GET /api/monitors/:id/history` - Get check history + +## ๐Ÿ“Š Plan Limits + +### Free Plan +- 5 monitors +- Minimum 60-minute frequency +- 7-day history retention +- Email alerts only + +### Pro Plan +- 50 monitors +- Minimum 5-minute frequency +- 90-day history retention +- All alert channels + +### Business Plan +- 200 monitors +- Minimum 1-minute frequency +- 1-year history retention +- API access +- Team features + +## ๐Ÿ—๏ธ Architecture + +### Backend +- **Express + TypeScript** - API server +- **PostgreSQL** - Relational database +- **Redis + BullMQ** - Job queue (coming soon) +- **Nodemailer** - Email alerts +- **Axios** - HTTP requests +- **Cheerio** - HTML parsing +- **Diff** - Change detection + +### Frontend +- **Next.js 14** - React framework +- **TypeScript** - Type safety +- **Tailwind CSS** - Styling +- **React Query** - Data fetching +- **Axios** - API client + +## ๐Ÿงช Testing + +```bash +# Backend tests +cd backend +npm test + +# Frontend tests +cd frontend +npm test +``` + +## ๐Ÿ“ Development + +### Running in Development +```bash +# Terminal 1: Start databases +docker-compose up + +# Terminal 2: Backend +cd backend +npm run dev + +# Terminal 3: Frontend +cd frontend +npm run dev +``` + +### Building for Production +```bash +# Backend +cd backend +npm run build +npm start + +# Frontend +cd frontend +npm run build +npm start +``` + +## ๐Ÿš€ Deployment + +### Backend Deployment +- Deploy to Railway, Render, or AWS +- Set environment variables +- Run migrations +- Start with `npm start` + +### Frontend Deployment +- Deploy to Vercel or Netlify +- Set `NEXT_PUBLIC_API_URL` +- Build with `npm run build` + +### Database +- Use managed PostgreSQL (AWS RDS, Railway, Supabase) +- Run migrations before deploying + +## ๐Ÿ› Troubleshooting + +### Database connection error +- Check PostgreSQL is running +- Verify DATABASE_URL is correct +- Run migrations: `npm run migrate` + +### Frontend can't connect to backend +- Check API_URL in .env.local +- Ensure backend is running +- Check CORS settings + +### Email alerts not working +- Configure SMTP settings in backend .env +- For development, use a service like Mailtrap +- For production, use SendGrid or similar + +## ๐Ÿ“š Documentation + +See `/docs` folder for: +- `spec.md` - Complete feature specifications +- `task.md` - Development roadmap +- `actions.md` - User workflows +- `claude.md` - Project context + +## ๐Ÿค Contributing + +1. Fork the repository +2. Create a feature branch +3. Commit your changes +4. Push to the branch +5. Open a Pull Request + +## ๐Ÿ“„ License + +MIT License - see LICENSE file for details + +## ๐Ÿ™ Acknowledgments + +Built with: +- Next.js +- Express +- PostgreSQL +- Tailwind CSS +- TypeScript