v10 RICH model card: 11-stage pipeline walkthrough, full version history v2-v10, detailed step-by-step usage

README.md

@@ -1,19 +1,41 @@
 ---
-language: en
+language:
+  - en
 license: mit
 library_name: phishbyte
 pipeline_tag: text-classification
 tags:
   - phishing-detection
   - email-security
+  - cybersecurity
+  - security
   - pytorch
   - from-scratch
   - no-pretrained-weights
   - cascading-inference
   - lightweight
   - explainable-ai
+  - nlp
+  - phishing
+  - spam-detection
+  - malware-detection
+  - threat-detection
+  - email-classification
+  - text-classification
+  - feature-engineering
+  - interpretable-ml
+  - tfidf
+  - residual-network
+  - context-gating
+  - calibrated-probabilities
+  - dmarc-alignment
 datasets:
-  - CEAS-2008
+  - ceas-2008
+  - enron-email
+  - spamassassin
+  - ling-spam
+  - nazario-phishing
+  - nigerian-fraud
 metrics:
   - f1
   - precision
@@ -26,102 +48,344 @@ model-index:
           type: text-classification
           name: Phishing Email Detection
         dataset:
-          name: CEAS-2008
+          name: 7-source benchmark (CEAS, Enron, SpamAssassin, Ling-Spam, Nazario, Nigerian, farshad72, puyang2025)
           type: ceas-2008
         metrics:
           - type: f1
-            value: 0.948
+            value: 0.9445
+            name: F1 Score
           - type: accuracy
-            value: 0.944
+            value: 0.9477
+            name: Accuracy
           - type: precision
-            value: 0.954
+            value: 0.9402
+            name: Precision
           - type: recall
-            value: 0.943
+            value: 0.9489
+            name: Recall
+widget:
+  - text: "From: PayPal Security <security@paypa1-alert.tk>\nReply-To: attacker@evil-domain.ru\nSubject: URGENT: Your account will be suspended\n\nDear Customer, your PayPal account has been suspended. Verify now at http://paypal-login.tk/verify"
+    example_title: "Phishing email example"
+  - text: "From: alice@company.com\nReply-To: alice@company.com\nSubject: Team lunch tomorrow\n\nHi everyone, lunch is at noon in the usual spot. See you there!"
+    example_title: "Legitimate email example"
 ---
 
-# Phish_Byte
+# Phish_Byte v10
 
-A from-scratch PyTorch model for **email phishing detection**.
-**F1 0.948** on CEAS-2008. **12,545 parameters** (≈9,000× smaller than DistilBERT).
-**1,500+ emails/sec** on a laptop GPU. Every verdict explains itself.
+A from-scratch PyTorch model for **email phishing detection** — no pretrained language model, no transformer, no fine-tuning.
 
-## Why this exists
+**743,571 parameters.** Every signal is a feature computed directly from the email itself, fed through a **context-gating architecture** that learns how one piece of evidence should change the interpretation of another — rather than a fixed hand-written rule deciding that for it.
 
-Every phishing detection model on HuggingFace is currently a fine-tuned
-transformer (DistilBERT, BERT, RoBERTa) — 65 to 110 million parameters,
-~250 MB on disk, ~50 ms per email on GPU. Phish_Byte takes a different
-bet: a small custom MLP trained from scratch, fed by 29 carefully chosen
-features, routed through a cascading inference pipeline. The model is
-**9,000× smaller** than DistilBERT, performs competitively, deploys
-without a GPU, and explains every decision.
+> The only non-transformer phishing detection model on HuggingFace.
+
+---
+
+## Table of contents
+
+- [Install](#install--no-pypi-package-yet)
+- [Usage](#usage)
+- [How it works, stage by stage](#how-it-works-stage-by-stage)
+- [Architecture](#architecture)
+- [Version history](#version-history)
+- [Feature groups](#feature-groups)
+- [Training data](#training-data)
+- [Benchmarks](#benchmarks)
+- [Limitations](#limitations)
+
+---
+
+## Install — no PyPI package yet
+
+`pip install phishbyte` does **not** work yet. The only supported path is cloning the source repository. Five steps, in order:
+
+**Step 1 — Clone the repository**
+```bash
+git clone https://github.com/AnonymousSingh-007/Phish_Byte.git
+cd Phish_Byte
+```
+
+**Step 2 — Create a virtual environment**
+```bash
+python -m venv venv
+```
+
+**Step 3 — Activate it**
+```bash
+# Windows (PowerShell)
+.\venv\Scripts\Activate.ps1
+
+# Mac / Linux
+source venv/bin/activate
+```
+
+**Step 4 — Install dependencies**
+```bash
+pip install -r requirements.txt
+```
+Minimal set: `torch`, `huggingface_hub`, `safetensors`, `dnspython`, `numpy`, `pandas`.
+
+For GPU acceleration on RTX 50-series (Blackwell) cards:
+```bash
+pip install torch --index-url https://download.pytorch.org/whl/cu128
+```
+
+**Step 5 — Verify the install**
+```bash
+python verify_install.py
+```
+This checks every Python package and every source file is present, then does a live test download of the model weights from this Hub repo. If anything is missing, it tells you exactly what — not a confusing traceback. Expected output when everything is correct:
+
+```
+✅ Python 3.11.x
+✅ torch
+✅ huggingface_hub
+✅ safetensors
+✅ dns
+✅ numpy
+✅ pandas
+✅ phishbyte/__init__.py
+... (all source files)
+✅ from phishbyte import PhishByteEngine — works
+✅ Model loaded from Hub successfully
+✅ INSTALLATION VERIFIED
+```
+
+---
 
 ## Usage
 
+### Basic usage — analyze any raw email
+
 ```python
 from phishbyte import PhishByteEngine
 
-engine  = PhishByteEngine.from_pretrained("AnonymousSingh-007/phishbyte")
+# First call downloads ~3 MB (weights + thresholds + vocabulary) from this
+# Hub repo and caches it locally. Every call after is instant.
+engine = PhishByteEngine.from_pretrained("SamSec007/phishbyte")
+
 verdict = engine.analyze(raw_email_string)
 
-print(verdict.label)             # 'phishing'
-print(verdict.probability)       # 0.9735
-print(verdict.confidence)        # 'high'
-print(verdict.layer_used)        # 2 — MLP made this call
-print(verdict.feature_weights)   # full per-feature attribution
+print(verdict.label)             # "phishing" or "legitimate"
+print(verdict.probability)       # calibrated confidence, 0.0 to 1.0
+print(verdict.confidence)        # "high" / "medium" / "low"
+print(verdict.layer_used)        # 1 = a fast rule made the call, 2 = the full network did
+print(verdict.feature_weights)   # every signal computed for this specific email
+```
+
+### Analyze a real email from your own Gmail
+
+**Step 1.** Open the suspicious email in Gmail.
+
+**Step 2.** Click the **⋮** menu in the top right of the email, then click **Show original**. This opens a new tab with the complete raw email, including every header.
+
+**Step 3.** Select all the text (Ctrl+A) and copy it (Ctrl+C).
+
+**Step 4.** Run the CLI:
+```bash
+python cli.py
+```
+
+**Step 5.** Paste the email when prompted, then press Enter followed by Ctrl+Z on Windows (or Ctrl+D on Mac/Linux) to submit it.
+
+### Analyze a saved `.eml` file
+
+```bash
+python cli.py --file suspicious.eml
+```
+
+### Quick demo — no files needed
+
+```bash
+python cli.py --demo phish     # a representative phishing example
+python cli.py --demo legit     # a representative legitimate example
+```
+
+### Reading the verdict object
+
+```python
+PhishVerdict(
+    label           = "phishing",
+    probability     = 0.9735,
+    confidence      = "high",
+    layer_used      = 2,
+    feature_weights = {
+        "display_name_mismatch":       1.00,   # "PayPal" in display name, unrelated domain
+        "mcld_mismatch":                1.00,   # most-linked domain isn't the sender's
+        "auth_alignment_score":         0.95,   # authentication does NOT validate this sender
+        "coercive_urgency_score":       0.82,   # pressure language: "verify now", "suspended"
+        "professional_formality_score": 0.02,   # essentially none — this isn't formal writing
+        ...
+    },
+    detail = "MLP probability (calibrated): 97.35%. Trust consistency: 0.10. Auth alignment: 0.95.",
+)
 ```
 
+---
+
+## How it works, stage by stage
+
+**Stage 1 — Parsing.** The raw email string is split into its headers (`From`, `Reply-To`, `Return-Path`, `Subject`, `Authentication-Results`) and its body, using Python's standard email parser. This handles both plain-text and HTML/multipart emails.
+
+**Stage 2 — Domain analysis.** Checks whether the From, Reply-To, and Return-Path addresses are consistent with each other; whether the display name claims a known brand (like "PayPal Security") while the actual domain is unrelated; and whether the domain itself looks auto-generated, based on digit density, hyphen count, and length.
+
+**Stage 3 — URL and body analysis.** Extracts every link in the email, checks whether visible link text matches where the link actually points, measures how many distinct destination domains the links spread across, and looks at structural characteristics of the body like unusual capitalization density.
+
+**Stage 4 — Authentication validation (SPF, DKIM, DMARC).** Reads the `Authentication-Results` header that the receiving mail server already computed, extracting whether SPF passed, whether DKIM signed the message with a domain that matches the sender, and what DMARC — the policy that ties SPF and DKIM together — concluded. This is a live, structural check, not a keyword guess.
+
+**Stage 5 — Subject line analysis.** The same kind of pattern-matching as the body, scoped to the subject: brand names, currency symbols, ALL-CAPS shouting, fake "RE:" prefixes designed to look like an ongoing conversation.
+
+**Stage 6 — Link and form forensics.** Finds the single most common destination domain across every link in the email and compares it to the sender. Checks whether any form on the page submits directly to a raw IP address instead of a domain — legitimate sites essentially never do this. Detects "open redirect" URL patterns commonly used to disguise a final destination.
+
+**Stage 7 — Lexical domain analysis.** A character-by-character look at domain names: does the domain have an unusual run of digits? Does it read like a real word or a randomly generated string? Is it a near-miss spelling of a known brand, once common digit-for-letter substitutions are normalized (`micros0ft` → `microsoft`)?
+
+**Stage 8 — Cross-signal agreement check.** Looks at whether the independent modules above agree with each other. Three modules independently raising concern is much stronger evidence than one module alone.
+
+**Stage 9 — Context feature computation.** This is where v10 diverges most from earlier versions. Urgency language in the body is split into two independent numbers — how *coercive* it is ("verify immediately or your account will be suspended") versus how *professionally formal* it is ("we kindly ask for your commitment to this important task") — because a single blended urgency score cannot tell these apart, and they mean very different things. A separate signal captures how strongly DMARC and DKIM validate the sender, independent of what the link-forensics stage found — so the network can weigh "the links go somewhere else" differently depending on whether the sender proved its identity or not. A similar context signal exists for Reply-To addresses that use free email providers like Gmail.
+
+**Stage 10 — Fusion.** All the evidence from stages 2 through 9 is split into two groups — *raw evidence* (facts about this email) and *context evidence* (facts that should change how the raw evidence is read) — and handed to a small neural layer whose only job is learning a gate: for this particular email, how much should the context evidence turn up or down the weight given to each piece of raw evidence. This gate is learned from real data, not hardcoded.
+
+**Stage 11 — Decision.** The fused representation, plus 50 word-frequency signals learned from the training corpus, feeds a residual neural network. Its output passes through a learned temperature parameter before being converted into a final probability, so that "80% confident" is empirically close to being right 80% of the time.
+
+---
+
 ## Architecture
 
 ```
-Layer 1 — rule scorers (~1 ms): domain + URL + SPF + subject
-            │
-            ├──► obvious phishing? short-circuit verdict
-            │
-            └──► otherwise route to MLP
-                       │
-Layer 2 — MLP (~3 ms): 29 → 96 → 48 → 1 (sigmoid)
-            │
-            ▼
-        PhishVerdict {label, probability, confidence, layer_used, feature_weights}
-```
-
-## Performance (CEAS-2008, n=2000 held-out)
-
-| Metric           | Value     |
-|------------------|----------:|
-| F1 score         | **0.948** |
-| Accuracy         | 94.40%    |
-| Precision        | 0.9537    |
-| Recall           | 0.9432    |
-| Parameters       | 12,545    |
-| Model size       | ~50 KB    |
-| Throughput (GPU) | 1,527 /s  |
-| Throughput (CPU) | ~800 /s   |
-
-## Features (29 inputs)
-
-- **Domain (5)**: From/Reply-To/Return-Path mismatch, freemail flag, brand impersonation
-- **URL (5)**: HTTPS ratio, anchor mismatch, suspicious TLD, urgency, link density
-- **SPF (3)**: SPF fail, no record, no sending IP
-- **Subject (7)**: urgency, security theme, brand name, currency, all caps, fake RE, fake transaction ID
-- **Character-level (5)**: caps ratio, digit ratio, special chars, avg word length, HTML/text ratio
-- **Composite (4)**: per-layer normalized scores
+raw email
+  │
+  ▼
+parsing → domain / URL / auth (SPF+DKIM+DMARC) / subject / link-forensics / lexical
+  │
+  ▼
+cross-signal agreement check
+  │
+  ▼
+context feature computation
+  (coercive vs. professional urgency, auth-verified sender context,
+   freemail Reply-To context, observational tracking-pattern signal)
+  │
+  ├──────────────┬───────────────┐
+  ▼              ▼               │
+RAW evidence   CONTEXT evidence  │
+(38 numbers)   (13 numbers)      │
+  │              │               │
+  └──────┬───────┘               │
+         ▼                       │
+  Context Fusion Layer            │
+  (learns a gate: how much        │
+   should context reweight        │
+   each raw signal, per email)    │
+         │                        │
+         ▼                        │
+  fused representation (64) ──────┘
+         │         TF-IDF (50)
+         └────┬────────┘
+              ▼
+  residual MLP: 620 → 310 (×2 residual blocks) → 155 → 76 → 1
+              │
+              ▼
+  temperature-calibrated confidence score
+              │
+              ▼
+  PhishVerdict — label, confidence, and every signal that fired
+```
+
+**743,571 parameters total.** For comparison, DistilBERT-based phishing detectors on HuggingFace use 66,000,000+ parameters — roughly 90× more.
+
+---
+
+## Version history
+
+**v2 — 12,545 parameters, 29 features, single dataset (CEAS-2008, ~39K emails).** The original prototype. A small MLP over hand-picked domain, URL, and subject features, with a cascading Layer 1 (cheap rules) → Layer 2 (neural network) design that every later version kept.
+
+**v7 — 254K parameters, 85 features, 83K emails across 6 datasets.** Added a TF-IDF vocabulary learned directly from the training corpus, and Body Domain Identification — checking the most common link destination against the sender. First version tested for generalization beyond a single dataset.
+
+**v8 — 716K parameters, 104 features, 166K emails across 7 datasets.** Added character-level lexical domain analysis (digit runs, entropy, typosquat distance) and a cross-signal layer that checks whether independent modules agree with each other rather than scoring each one in isolation.
+
+**v9 — 718K parameters, 107 features.** Replaced a naive SPF-only authentication check — which misfired constantly on legitimate marketing platforms like Marketo and Mailgun that relay mail on a brand's behalf — with a proper DMARC/DKIM alignment check that reads what the receiving mail server already validated.
+
+**v10 (current) — 743,571 parameters, context-gated fusion architecture.** Split urgency detection into two independent signals — coercive pressure versus professional formality — that were previously conflated into a single blended number. Restructured the network so raw evidence and context evidence enter as separate inputs to a learned fusion layer, instead of being concatenated together and left for the network to disentangle unaided.
+
+---
+
+## Feature groups
+
+| Group | Count | What it measures |
+|-------|:-----:|-------------------|
+| Domain (raw) | 7 | header consistency, brand impersonation, display-name spoofing |
+| URL + body (raw) | 4 | link security, anchor/href mismatch, link density |
+| SPF (raw) | 3 | basic sender authorization signal |
+| Subject (raw) | 7 | brand mentions, currency, formatting, fake reply prefixes |
+| Char-level (raw) | 5 | capitalization, digit density, HTML/text ratio |
+| BDI (raw) | 5 | most-common-link-domain mismatch, IP-target forms, open redirects |
+| Lexical — sender domain (raw) | 6 | digit runs, hyphen runs, entropy, typosquat distance |
+| Coercive urgency (raw) | 1 | pressure/threat language, independent of formal tone |
+| Auth-verified ESP context | 1 | how strongly DMARC/DKIM validate the sender |
+| Professional formality (context) | 1 | formal/professional tone, independent of coercive urgency |
+| Freemail Reply-To context | 2 | raw fact + how much authentication offsets it |
+| Tracking pattern (observational) | 1 | structural resemblance to ESP tracking infrastructure — never used to exempt anything on its own |
+| Cross-signal fusion (context) | 5 | agreement between independent modules |
+| Auth alignment (context) | 3 | DMARC pass, DKIM alignment, composite score |
+| TF-IDF | 50 | words learned directly from the training corpus |
+
+**101 features total** (38 raw + 13 context + 50 TF-IDF).
+
+---
+
+## Training data
+
+| Dataset | Source | Contribution |
+|---------|--------|---------------|
+| CEAS-2008 | Kaggle | ~39K emails, 2008-era phishing |
+| Enron | Kaggle | ~29K emails, legitimate corporate correspondence |
+| SpamAssassin | Kaggle | ~10K emails, mixed spam/legitimate |
+| Nigerian Fraud | Kaggle | ~3.3K emails, advance-fee fraud |
+| Nazario | Kaggle | ~1.5K emails, phishing corpus |
+| Ling-Spam | Kaggle | ~2.8K emails |
+| farshad72/spam_email | HuggingFace | 83K rows, includes modern notification-style legitimate email |
+| puyang2025/seven-phishing-email-datasets | HuggingFace | 203K rows, unified 7-source corpus |
+| **Combined, after deduplication** | — | **~166,000 emails, ~56% phishing / 44% legitimate** |
+
+---
+
+## Benchmarks
+
+Evaluated on 12,000 held-out samples, self-reported.
+
+| Metric | Value |
+|--------|:-----:|
+| F1 score | 0.9445 |
+| Accuracy | 94.77% |
+| Precision | 0.9402 |
+| Recall | 0.9489 |
+| Parameters | 743,571 |
+| Model size on disk | ~3 MB |
+| Throughput (GPU) | ~630 emails/sec |
+| GPU required | No |
+
+---
 
 ## Limitations
 
-- ~5% of decisions are wrong (F1 0.948, not 1.0). Use as one signal in defence-in-depth, not the only gate.
-- Trained on CEAS-2008 — English-language phishing from 2008. Modern attack patterns and non-English emails will degrade performance.
-- SPF validation is bypassed for training (historical domains don't resolve) but runs live at inference time.
-- Adversarial emails crafted specifically to game these features will get through.
+Read this before deploying anywhere real.
+
+- **Most training data predates 2010.** Modern phishing techniques — OAuth abuse, QR code lures, redirect chains through legitimate cloud services — are underrepresented even after adding modern HuggingFace datasets.
+- **False positives on legitimate marketing and notification email are reduced but not eliminated.** The context-gating architecture measurably helps here, but this remains an active area of work rather than a solved problem.
+- **No adversarial robustness testing has been performed.** An attacker aware of the exact feature set could plausibly craft targeted bypasses. Use as one signal in a defence-in-depth stack, not a standalone gate.
+- **Benchmark numbers are self-reported** on a held-out split of the training corpus, not independently verified or peer-reviewed.
+- **Not production-hardened** — no retry logic, rate limiting, or async network handling.
+- **English-language only.**
 
 ## Citation
 
+No peer-reviewed paper exists yet. Until then, cite the repository directly:
+
 ```bibtex
 @software{phishbyte2026,
-  author  = {Singh, Samratth},
-  title   = {Phish_Byte: A cascading from-scratch PyTorch model for email phishing detection},
-  year    = {2026},
-  url     = {https://github.com/AnonymousSingh-007/Phish_Byte}
+  author = {Singh, Samratth},
+  title  = {Phish_Byte: Context-gated fusion architecture for from-scratch email phishing detection},
+  year   = {2026},
+  url    = {https://github.com/AnonymousSingh-007/Phish_Byte}
 }
 ```