# WA-Filter Complete Refactoring Plan
## SECTION 1: Proposed Folder Structure
```
wa-filter/
├── .env
├── .gitignore
├── package.json
├── server.js # NEW: Slim entry point (~30 lines) — creates http server, starts app
│
├── src/
│ ├── app.js # Express app setup, middleware, static files, mount all routes
│ ├── socket.js # Socket.io init + session sharing + connection handler dispatcher
│ ├── startup.js # Global startup: initDB → loadAllSessions → startWarmer → cleanup
│ │
│ ├── config/
│ │ ├── db.js # MySQL pool creation (from current db.js)
│ │ ├── schema.js # All CREATE TABLE + migrations (from db.js initDB)
│ │ ├── session.js # express-session + MySQLStore config
│ │ └── env.js # Centralized env var access with defaults
│ │
│ ├── middlewares/
│ │ ├── auth.js # requireAuth, requireAdmin middleware
│ │ └── upload.js # Multer config
│ │
│ ├── routes/
│ │ ├── auth.routes.js # /auth/* — login, register, logout, me
│ │ ├── agent.routes.js # /api/agents/* — CRUD + number assignment
│ │ ├── contact.routes.js # /upload, /download, /clear-history
│ │ ├── session.routes.js # /sessions, /sessions-info, /logout, /delete-session
│ │ ├── automation.routes.js # /api/automation-flows/*, /api/wa-chatbots/*, /api/executions/*
│ │ ├── api.routes.js # /api/send, /api/generate-token, /api-settings
│ │ └── misc.routes.js # /download-extracted
│ │
│ ├── controllers/
│ │ ├── agent.controller.js # Handler functions for agent routes
│ │ ├── contact.controller.js # Upload CSV, download, clear
│ │ ├── session.controller.js # List sessions, logout WA, delete session
│ │ ├── automation.controller.js # Flows + chatbots + executions CRUD
│ │ └── api.controller.js # External send API, token generation
│ │
│ ├── services/
│ │ ├── whatsapp.service.js # initWhatsApp(), Baileys socket creation, event wiring
│ │ ├── session-manager.js # activeSocks{}, sessionStatus{}, sessionQRs{}, activeChats{} registry
│ │ ├── filter.service.js # Filtering logic (onWhatsApp checks)
│ │ ├── bulk-send.service.js # Bulk message sending logic
│ │ ├── warmer.service.js # Warmer engine: runCycle, executeForConfig, startEngine
│ │ ├── chatbot.service.js # Rule matching, flow traversal, auto-reply
│ │ ├── message.service.js # logMessage(), LID mapping, history sync
│ │ └── settings.service.js # getUserSettings, updateUserSettings, warmer/chatbot config
│ │
│ ├── queries/
│ │ ├── user.queries.js # Users table queries
│ │ ├── contact.queries.js # Contacts table queries
│ │ ├── session.queries.js # wa_accounts table queries
│ │ ├── message.queries.js # messages_history queries
│ │ ├── automation.queries.js# automation_flows, wa_chatbots, chat_executions queries
│ │ └── settings.queries.js # user_settings, warmer_config, chatbot_rules queries
│ │
│ ├── sockets/
│ │ ├── index.js # Main io.on('connection') — auth check, dispatch to handlers
│ │ ├── session.handler.js # switch-session, create-session, restart-session
│ │ ├── filter.handler.js # start-filter, stop-filter + progress events
│ │ ├── bulk.handler.js # start-bulk-send + progress events
│ │ ├── chat.handler.js # get-chats, get-history, send-message, start-new-chat
│ │ ├── chatbot.handler.js # get/add/delete-chatbot-rule
│ │ ├── warmer.handler.js # get/update-warmer-config
│ │ ├── group.handler.js # get-groups, extract-group-participants
│ │ └── settings.handler.js # get-settings, update-settings
│ │
│ └── utils/
│ ├── jid.js # formatJid(), maskPhoneNumber(), cleanNumber()
│ └── qr.js # QR code generation helper (emitSessionUpdate)
│
├── public/
│ ├── index.html # Main SPA shell (keep as-is, just update script tags)
│ ├── login.html
│ ├── register.html
│ ├── style.css
│ └── js/
│ ├── main.js # Auth check, socket init, navigation, global state
│ ├── modules/
│ │ ├── dashboard.js # Dashboard stats, charts, updateDashboardUI
│ │ ├── accounts.js # Session cards, add/switch/restart/delete session
│ │ ├── inbox.js # Chat list, chat window, message rendering, send
│ │ ├── filter.js # CSV upload, start filter, progress UI
│ │ ├── bulk-send.js # Bulk send form, progress UI
│ │ ├── warmer.js # Warmer toggle, script management, device list
│ │ ├── chatbot.js # Smart Bot rules CRUD
│ │ ├── automation.js # Drawflow editor, flows CRUD
│ │ ├── wa-chatbots.js # WA Chatbot linker, executions viewer
│ │ ├── agents.js # Agent CRUD, number assignment modal
│ │ ├── groups.js # Group scraper, export CSV
│ │ ├── settings.js # Auto-reply settings
│ │ ├── rest-api.js # API token, docs UI
│ │ └── utils.js # showNotification, formatDisplayJid, isSameJid, clipboard
│ └── lib/ # (optional) local copies of chart.js, drawflow if needed
│
├── sessions/ # Baileys auth state (gitignored)
├── uploads/ # Temp CSV uploads (gitignored)
└── auth_info/ # Legacy Baileys auth (gitignored)
```
---
## SECTION 2: Migration Map (server.js → New Files)
| Lines | Current Content | → New File |
|-------|----------------|------------|
| 1–13 | Imports, require statements | `src/app.js` (Express), `src/config/*.js` |
| 14–26 | Express + HTTP + Socket.io + Multer setup | `server.js` (entry), `src/app.js`, `src/socket.js`, `src/middlewares/upload.js` |
| 29–44 | Session middleware + Socket session sharing | `src/config/session.js` |
| 46–68 | Static files, auth middleware, auth routes mount | `src/app.js`, `src/middlewares/auth.js` |
| 70–75 | requireAdmin middleware | `src/middlewares/auth.js` |
| 78–161 | Employee/Agent API routes | `src/routes/agent.routes.js` → `src/controllers/agent.controller.js` |
| 163–278 | Automation Flows + WA Chatbots + Executions API | `src/routes/automation.routes.js` → `src/controllers/automation.controller.js` |
| 280–319 | CSV upload + contact processing | `src/routes/contact.routes.js` → `src/controllers/contact.controller.js` |
| 321–329 | Clear history | `src/controllers/contact.controller.js` |
| 330–377 | WA logout endpoint | `src/routes/session.routes.js` → `src/controllers/session.controller.js` |
| 379–426 | Delete session endpoint | `src/controllers/session.controller.js` |
| 428–466 | Download CSV endpoints | `src/controllers/contact.controller.js` |
| 468–544 | External Send API + Token + Settings | `src/routes/api.routes.js` → `src/controllers/api.controller.js` |
| 546–590 | /sessions, /sessions-info endpoints | `src/controllers/session.controller.js` |
| 592–596 | In-memory state declarations | `src/services/session-manager.js` |
| 598–635 | Warmer config get/update functions | `src/services/settings.service.js` |
| 637–1049 | `initWhatsApp()` — Baileys init, events, message handling | `src/services/whatsapp.service.js` + `src/services/chatbot.service.js` + `src/services/message.service.js` |
| 1051–1083 | `emitSessionUpdate()` | `src/utils/qr.js` |
| 1085–1170 | Warmer engine functions | `src/services/warmer.service.js` |
| 1172–1183 | `loadAllSessions()` | `src/startup.js` |
| 1185–1223 | `logMessage()` | `src/services/message.service.js` |
| 1225–1291 | Settings + Chatbot rule functions | `src/services/settings.service.js` |
| 1293–1322 | Global startup IIFE | `src/startup.js` |
| 1324–1822 | `io.on('connection')` — all socket handlers | `src/sockets/index.js` + individual `src/sockets/*.handler.js` |
| 1824–1830 | Server listen + export | `server.js` |
### Frontend Migration (public/app.js → modules)
| Lines | Content | → New File |
|-------|---------|------------|
| 1–22 | Auth check, role detection | `js/main.js` |
| 24–50 | Socket init, global state, helpers | `js/main.js` + `js/modules/utils.js` |
| 52–108 | Navigation, screen switching | `js/main.js` |
| 110–173 | Socket events: connection, QR, status | `js/modules/accounts.js` |
| 175–243 | History + chats-list socket handlers | `js/modules/inbox.js` |
| 247–538 | Session management UI functions | `js/modules/accounts.js` |
| 540–565 | API token + notifications | `js/modules/rest-api.js` + `js/modules/utils.js` |
| 570–586 | Add session button | `js/modules/accounts.js` |
| 588–812 | Chat list rendering + chat window | `js/modules/inbox.js` |
| 814–848 | Message append + quick reply | `js/modules/inbox.js` |
| 850–929 | Bulk send + Filter UI | `js/modules/bulk-send.js` + `js/modules/filter.js` |
| 931–946 | Settings save/load | `js/modules/settings.js` |
| 948–970 | Groups socket handlers | `js/modules/groups.js` |
| 972–1103 | Charts + Dashboard | `js/modules/dashboard.js` |
| 1104–1203 | Warmer UI | `js/modules/warmer.js` |
| 1205–1416 | Init sequence + refreshSessions + new-message | `js/main.js` + `js/modules/accounts.js` |
| 1417–1507 | Group scraper UI | `js/modules/groups.js` |
| 1509–1666 | Agent management UI | `js/modules/agents.js` |
| 1668–1937 | Drawflow automation builder | `js/modules/automation.js` |
| 1938–2148 | WA Chatbots linker + screen hooks | `js/modules/wa-chatbots.js` |
---
## SECTION 3: Dependency & Initialization Flow
### New Startup Sequence
```
server.js (entry point)
│
├── require('./src/config/env') // Load .env first
├── require('./src/config/db') // Create MySQL pool
├── require('./src/app') // Build Express app
│ ├── Apply session middleware
│ ├── Apply static files
│ └── Mount all route files
├── http.createServer(app)
├── require('./src/socket')(server) // Init Socket.io, share session
└── require('./src/startup').boot() // async:
├── initDB (schema migrations)
├── loadAllSessions (reconnect Baileys)
├── startWarmerEngine
└── cleanupOrphanedSessions
```
### Session Manager (Singleton Pattern)
```js
// src/services/session-manager.js
const activeSocks = {}; // sessionId → Baileys socket
const sessionStatus = {}; // sessionId → 'connecting'|'open'|'closed'
const sessionQRs = {}; // sessionId → raw QR string
const activeChats = {}; // sessionId → { jid: chatObj }
let isFiltering = false;
module.exports = {
activeSocks, sessionStatus, sessionQRs, activeChats,
get isFiltering() { return isFiltering; },
set isFiltering(v) { isFiltering = v; },
getSock(sid) { return activeSocks[sid]; },
setSock(sid, sock) { activeSocks[sid] = sock; },
removeSock(sid) { delete activeSocks[sid]; /* + cleanup others */ }
};
```
### Socket.io Sharing (No Circular Deps)
```js
// src/socket.js
let io;
module.exports = {
init(server, sessionMiddleware) {
const { Server } = require('socket.io');
const BASE_PATH = process.env.PASSENGER_BASE_URI || '';
io = new Server(server, { path: BASE_PATH + '/socket.io' });
// Share session
const wrap = mw => (socket, next) => mw(socket.request, {}, next);
io.use(wrap(sessionMiddleware));
// Register all handlers
require('./sockets')(io);
return io;
},
getIO() { return io; } // Any service can call this
};
```
Services that need `io` (like `message.service.js`) call `require('../socket').getIO()` — no circular dependency because `socket.js` doesn't import services.
---
## SECTION 4: Frontend Modularization Plan
### Loading Strategy: ES Modules (`type="module"`)
Since there's no bundler and cPanel serves static files fine, use native ES modules:
```html
```
### Module Pattern
```js
// js/main.js
import { initSocket, socket } from './modules/utils.js';
import { initDashboard } from './modules/dashboard.js';
import { initAccounts } from './modules/accounts.js';
import { initInbox } from './modules/inbox.js';
// ... etc
// Global shared state
export const state = {
userRole: 'admin',
currentScreen: 'screen-dashboard',
currentActiveSession: localStorage.getItem('lastSession') || 'default',
activeChatJid: null,
allMessages: [],
};
// Auth check then init all modules
fetch('/auth/me').then(r => r.json()).then(data => {
if (!data.success) return window.location.href = '/login.html';
state.userRole = data.user.role;
initDashboard(socket, state);
initAccounts(socket, state);
initInbox(socket, state);
// ...
});
```
Each module exports an `init(socket, state)` function and registers its own event listeners. Modules share state through the imported `state` object.
> [!NOTE]
> `login.html` and `register.html` stay as standalone pages — they don't need modules.
---
## SECTION 5: Database Layer
### Query Organization
Each `queries/*.js` file exports pure async functions that accept `pool` and parameters:
```js
// src/queries/contact.queries.js
const { pool } = require('../config/db');
module.exports = {
async insertContact(userId, phoneNumber, originalData) {
const [result] = await pool.query(
'INSERT IGNORE INTO contacts (user_id, phone_number, original_data) VALUES (?, ?, ?)',
[userId, phoneNumber, JSON.stringify(originalData)]
);
return result;
},
async getUnprocessed(userId, limit) {
const [rows] = await pool.query(
'SELECT * FROM contacts WHERE user_id = ? AND status = "unprocessed" LIMIT ?',
[userId, limit]
);
return rows;
},
async updateStatus(contactId, status) {
await pool.query('UPDATE contacts SET status = ? WHERE id = ?', [status, contactId]);
},
async getByStatus(userId, status) {
const [rows] = await pool.query(
'SELECT phone_number, original_data FROM contacts WHERE user_id = ? AND status = ?',
[userId, status]
);
return rows;
},
async clearAll(userId) {
await pool.query('DELETE FROM contacts WHERE user_id = ?', [userId]);
}
};
```
### End-to-End Example: Filter Feature
```
POST /upload (file)
→ contact.routes.js
→ contact.controller.js::uploadCSV()
→ contact.queries.js::insertContact() (loop)
→ res.json({ total, new })
Socket: 'start-filter'
→ sockets/filter.handler.js
→ filter.service.js::runFilter(userId, options)
→ contact.queries.js::getUnprocessed()
→ session-manager.js::getSock(sid)
→ sock.onWhatsApp(jid)
→ contact.queries.js::updateStatus()
→ socket.getIO().to(`user_${userId}`).emit('progress', stats)
```
### Schema Improvements Noticed
```sql
-- messages_history: Add composite index for chat queries
CREATE INDEX idx_msg_session_jid ON messages_history(session_id, jid, timestamp);
CREATE INDEX idx_msg_user_session ON messages_history(user_id, session_id);
-- contacts: Add index for filtering queries
CREATE INDEX idx_contacts_status ON contacts(user_id, status);
```
---
## SECTION 6: Configuration Management
```js
// src/config/env.js
require('dotenv').config();
module.exports = {
db: {
host: process.env.DB_HOST || 'localhost',
user: process.env.DB_USER || 'root',
password: process.env.DB_PASS || '',
database: process.env.DB_NAME || 'wafilter',
connectionLimit: parseInt(process.env.DB_POOL_SIZE) || 10,
},
session: {
secret: process.env.SESSION_SECRET || 'fallback_secret_for_wafilter_123',
maxAge: 1000 * 60 * 60 * 24 * 30,
},
server: {
port: parseInt(process.env.PORT) || 3000,
basePath: process.env.PASSENGER_BASE_URI || '',
}
};
```
Legacy JSON files (`settings.json`, `warmer_config.json`, `chatbot_rules.json`) are **already migrated to MySQL** in the current code. They exist as dead files. Add them to `.gitignore` (already done) and don't touch them.
---
## SECTION 7: cPanel Deployment Considerations
### Entry Point for Passenger
cPanel Passenger expects `app.js` in root. Rename or create:
```js
// app.js (root — Passenger entry point)
module.exports = require('./server.js');
```
Or keep `server.js` as main and configure `.htaccess`:
```
PassengerAppRoot /home/user/wa-filter
PassengerStartupFile server.js
```
### Package.json Updates
```json
{
"main": "server.js",
"scripts": {
"start": "node server.js",
"dev": "node --watch server.js"
}
}
```
### Path Handling
All paths in the codebase use **relative paths** (`sessions/`, `uploads/`). After refactoring, services move into `src/` but paths are relative to CWD (project root), not the file location. This is fine as long as `node server.js` is run from the project root (which Passenger does).
> [!IMPORTANT]
> `sessions/` and `uploads/` directories must stay at **project root**, not inside `src/`. The `useMultiFileAuthState()` call uses these paths relative to CWD.
### `auth_info/` Folder
This is legacy from `index.js` (the old CLI filter script). It's not used by `server.js`. Safe to ignore.
---
## SECTION 8: Step-by-Step Migration Phases
### Phase 1: Config Extraction (Zero Risk)
- Create `src/config/env.js`, `src/config/db.js`, `src/config/session.js`
- Update `server.js` imports to use new config files
- Keep everything else in `server.js`
- **Test:** App starts and works identically
### Phase 2: Middleware + Auth Routes
- Create `src/middlewares/auth.js` (requireAdmin, requireAuth)
- Create `src/middlewares/upload.js` (multer config)
- Move `routes/auth.js` → `src/routes/auth.routes.js`
- **Test:** Login/register/logout still work
### Phase 3: Session Manager Singleton
- Create `src/services/session-manager.js` with all state objects
- Update `server.js` to import from session-manager instead of local vars
- **Test:** Multi-account connection still works
### Phase 4: Extract HTTP Routes (One Group at a Time)
- **4a:** Agent routes → `agent.routes.js` + `agent.controller.js`
- **4b:** Contact routes → `contact.routes.js` + `contact.controller.js`
- **4c:** Session routes → `session.routes.js` + `session.controller.js`
- **4d:** Automation routes → `automation.routes.js` + `automation.controller.js`
- **4e:** API routes → `api.routes.js` + `api.controller.js`
- **Test after each:** Hit the endpoints, verify responses
### Phase 5: Extract Services
- **5a:** `settings.service.js` (getUserSettings, updateUserSettings, warmer config, chatbot rules)
- **5b:** `message.service.js` (logMessage)
- **5c:** `warmer.service.js` (warmer engine)
- **5d:** `chatbot.service.js` (rule matching, flow traversal)
- **5e:** `whatsapp.service.js` (initWhatsApp — the big one, ~400 lines)
- **Test after each:** Full feature test of the moved logic
### Phase 6: Extract Socket Handlers
- Create `src/sockets/index.js` with auth check and handler registration
- Move each `socket.on(event)` block to its handler file
- **Test:** All real-time features (filtering progress, messages, QR codes)
### Phase 7: Extract Query Layer
- Create `src/queries/*.js` files
- Update services/controllers to use query functions
- **Test:** Full regression
### Phase 8: Frontend Modularization
- Create `public/js/modules/utils.js` with shared helpers
- Extract one module at a time (start with `dashboard.js`, lowest coupling)
- Update `index.html` script tags to `type="module"`
- **Test in browser after each module extraction**
### Phase 9: Startup Sequence
- Create `src/startup.js`
- Create slim `server.js` entry point
- Remove old startup IIFE from server.js
- **Test:** Cold start, session reconnection, warmer auto-start
---
## SECTION 9: What NOT to Change
### Baileys Constraints
- **`useMultiFileAuthState()`** must receive a path relative to CWD — keep `sessions/{userId}_{sessionId}` format
- **`sock.ev.on()` event handlers** must stay tightly coupled with the socket instance — keep them in `whatsapp.service.js`, don't try to further split
- **LID mapping logic** (reading/writing `lid_mappings.json`) is a Baileys workaround — keep as-is in `message.service.js`
- **Version fallback** `[2, 3000, 1015901307]` — keep this hardcoded fallback
### cPanel/Passenger Constraints
- **Single process** — no worker threads, no cluster mode
- **`PASSENGER_BASE_URI`** support must be preserved in all route mounts and Socket.io path
- **No build step** — frontend must work without webpack/vite
- **`app.js` or `server.js`** must exist at root for Passenger
### Patterns That Are Fine As-Is
- **MySQL connection pooling** via `mysql2/promise` — already correct
- **Session stored in MySQL** via `express-mysql-session` — correct for shared hosting
- **bcrypt for passwords** — correct, don't change
- **Socket.io room-based broadcasting** (`io.to(user_${userId})`) — correct pattern
- **Round-robin sock assignment** in filter/bulk — simple and effective
- **`index.html` as SPA shell** with screen toggling — works well, don't convert to multi-page
- **Inline CSS in `index.html` for Drawflow** — keep it, Drawflow needs specific scoped styles
> [!WARNING]
> **`index.js`** at root is the OLD CLI-only filter script. It is **not used** by the web app. Do NOT confuse it with the app entry point. It can be archived or deleted.