Geometric Formula Catalog
Token Topology & Loss System — AbstractPhil + Claude
ROSE loss discarded. These are the active formulas.
1. Multi-Scale Crystal Loss
Classification through learnable crystal prototypes at multiple projection dimensions. Each class has a crystal centroid at each scale. No softmax — geometric distance IS the classifier.
Scales: [64, 128, 256, 512, 1024] (each is a projection dimension, not spatial)
1.1 Per-Scale Crystal Similarity
sim(x, c_k) = (x̂ · ĉ_k) / τ
where:
x̂ = normalize(proj_k(features)) # [B, scale_dim]
ĉ_k = normalize(crystals_k) # [num_classes, scale_dim]
τ = temperature (default 0.07)
1.2 Per-Scale Coherence Loss
Pull features toward their correct class crystal:
L_coherence = -mean(log(exp(sim(x, c_y)) / Σ_j exp(sim(x, c_j))))
where y = true class label
1.3 Per-Scale Separation Loss
Push class crystals apart with margin:
L_separation = Σ_{i≠j} max(0, margin - ||ĉ_i - ĉ_j||₂)² / (C(C-1))
where C = num_classes, margin = 1.0
1.4 Per-Scale Discretization Loss (Cantor Targets)
Cluster crystal Cantor values toward {0.0, 0.5, 1.0}:
L_discretization = mean(min_t(||cantor(c_i) - t||²))
where t ∈ {0.0, 0.5, 1.0}
1.5 Per-Scale Crystal Geometry Loss
Maintain target distance from features to class prototypes:
L_geometry = mean((||x - c_y||₂ - d_target)²)
where d_target = 1.0
1.6 Total Multi-Scale Crystal Loss
L_crystal = (1/S) Σ_{k=1}^{S} w_k · (
w_coh · L_coherence_k +
w_sep · L_separation_k +
w_disc · L_discretization_k +
w_geom · L_geometry_k
)
Proven weights: w_coh=1.0, w_sep=0.5, w_disc=1.0, w_geom=0.5
1.7 Crystal Prediction (No Softmax Head)
logits = Σ_k w_k · (α · cos_sim_k + β · cantor_coherence_k + γ · crystal_geometry_k)
where prediction = argmax(logits)
Results: 86% ImageNet (CLIP bigG features), 74.87% CIFAR-100 (393K params), ~92% CIFAR-100 (78KB model)
2. Geometric Basin Compatibility Loss
Classification through geometric formula satisfaction. Four structural checks produce compatibility scores ∈ [0,1]. No cross-entropy needed.
2.1 Triadic Compatibility
T(x, c) = exp(-||proj(x) - c||₂² / (2σ²))
where c = class centroid, σ = learned bandwidth
2.2 Self-Similarity Check
S(x) = exp(-Var(cantor_levels(x)))
where cantor_levels extracts per-level Cantor measures
High self-similarity → low variance across levels → high score
2.3 Cantor Coherence Check
C(x, p_y) = exp(-||cantor(x) - p_y||₂²)
where p_y = class Cantor prototype
2.4 Hierarchical Check
H(x) = Σ_{k=1}^{L} 0.5^k · match(level_k(x), expected_k)
2.5 Combined Compatibility Score
compat(x, class_j) = T(x, c_j) · S(x) · C(x, p_j) · H(x)
Product of four factors ∈ [0,1] → output ∈ [0,1]
2.6 Basin Loss (Three-Term, No Cross-Entropy)
L_correct = -mean(log(compat(x, y) + ε))
L_incorrect = -mean(log(1 - compat(x, j≠y) + ε))
L_contrastive = NLL(log_softmax(compat / τ), y)
L_basin = L_correct + 0.5 · L_incorrect + 0.5 · L_contrastive
Results: 67.69% CIFAR-100 with NO attention, NO cross-entropy, NO transformers (geo-beatrix). Beat ViT-beatrix (66.0%).
3. K-Simplex Channel Formulas
Tokens represented as k-simplices with Cayley-Menger validated geometry. Shape [B, T, K+1, F] where K+1 = vertices.
3.1 Template + Deformation
v_i = v_i^{template} + α · Δv_i
where:
v_i^{template} = regular k-simplex vertices (frozen)
α = deformation scale (0.05 base, per-k scaled)
Δv_i = learned offset from neural network
3.2 K-Scaled Deformation
Volume scales as edge^k, so higher k needs smaller deformation:
α_k = α_base / √(k + 1)
k=1: α × 0.71 k=3: α × 0.50
k=2: α × 0.58 k=4: α × 0.45
3.3 Per-Token Simplex Coordinates
coords = proj(token_embedding) # [B, T, edim]
vertex_weights = softmax(route(token_embedding)) # [B, T, K+1]
simplex_state = vertex_weights @ vertices # [B, T, edim]
3.4 K-Simplex Attention (Proven Superior to K-Simplex Classification)
For each token pair (i, j):
d²_ij = ||simplex_i - simplex_j||² # pairwise simplex distance
attn_ij = softmax(-d²_ij / τ) # geometric attention weights
Output = attn @ V # standard value projection
Results: 89.13% FMNIST, 84.59% CIFAR-10, 69.08% CIFAR-100 as attention. Entropy decreases through layers (sharpening). Fewer tokens = sharper attention (25 patches > 64 patches).
4. Cayley-Menger Formulas
The structural invariant. If CM fails, geometry is invalid. Non-negotiable.
4.1 Cayley-Menger Matrix
CM = | 0 1 1 ... 1 |
| 1 0 d₀₁² ... d₀ₖ² |
| 1 d₀₁² 0 ... d₁ₖ² |
| ⋮ ⋮ ⋮ ⋱ ⋮ |
| 1 d₀ₖ² d₁ₖ² ... 0 |
Size: (K+2) × (K+2) for a K-simplex
4.2 Volume Formula (Corrected)
Vol² = (-1)^(K+1) / (2^K · (K!)²) · det(CM)
Validity: Vol² > 0 indicates non-degenerate simplex
4.3 Gram Determinant Alternative (More Stable)
X_translated = X[:, 1:, :] - X[:, 0:1, :] # [B, K, D]
G = X_translated @ X_translated.T # [B, K, K]
Vol = √(det(G)) / K!
4.4 Validity Loss
L_validity = mean(ReLU(-Vol²))
Penalizes collapsed simplices (Vol² < 0)
4.5 Volume Consistency Loss
L_vol_consistency = Var(Vol²) across batch
Encourages uniform geometric structure
4.6 Hierarchical Cell Loss (k=4 pentachoron)
5 cells (tetrahedra), each with 4 vertices, 6 edges:
L_cell = mean(ReLU(ε - Vol²_cell_i))
for i = 1..5 cells of the pentachoron
4.7 Vol² Scaling Reference
k=1: Vol² ~ 1e+0 (edge length squared)
k=2: Vol² ~ 1e-1 (triangle area squared)
k=3: Vol² ~ 1e-2 (tetrahedron volume squared)
k=4: Vol² ~ 1e-3 (5-cell hypervolume squared)
5. Cantor Lens Formulas
The Devil's Staircase as a hierarchical lens for viewing token relationships.
5.1 Devil's Staircase (Beatrix Staircase)
C(x) = Σ_{k=1}^{levels} bit_k × 0.5^k
where:
y_k = x × 3^k # scale to level k
p = softmax(-d²/τ) over centers [0.5, 1.5, 2.5]
bit_k = p_right + α × p_middle # soft ternary assignment
α = learnable middle-third fill (default 0.5)
τ = softmax temperature (default 0.25)
5.2 Branch Path Extraction
branch_path(x) = [argmax(p_1), argmax(p_2), ..., argmax(p_L)]
Each level: L (left third), M (middle third), R (right third)
5.3 Hierarchical Alignment (NOT Distance)
CRITICAL: Distance is meaningless on Cantor set.
alignment(i, j) = Σ_{k=1}^{L} 0.5^k · 𝟙(path_i[k] == path_j[k])
Level weights: [0.5, 0.25, 0.125, 0.0625, 0.03125]
Coarse matches = routing highways (wormholes). Fine matches = local structure only.
5.4 Euclidean Bridge (Lossy but Necessary)
distance(i, j) = |C(x_i) - C(x_j)|
Use ONLY when interfacing with Euclidean systems (optimizers, standard losses).
Alignment is the Cantor-native metric.
5.5 Cantor Routing Bias (for Attention)
bias[i,j] = alignment(i, j) # precomputed [S, S] matrix
attn_scores = (Q @ K.T / √d) + λ · bias
where λ = learnable routing weight
5.6 Alpha Modulation
α → 0.0: Pure ternary (Cantor dust, maximally disconnected)
α → 0.5: Triadic equilibrium (proven stable zone: 0.44-0.50)
α → 1.0: Filled (continuous, no fractal structure)
6. Cantor Topological Ropes
Position encodings that encode structural hierarchy, not just sequence order.
6.1 Standard RoPE (Baseline)
θ_i = 10000^(-2i/d)
R(m) = [cos(mθ_i), -sin(mθ_i); sin(mθ_i), cos(mθ_i)]
for dimension pair (2i, 2i+1) at position m
6.2 BeatrixRoPE (Devil's Staircase Warping)
pos_beatrix(m) = C(m / seq_len) # Cantor function of normalized position
R_beatrix(m) = R(pos_beatrix(m) × seq_len)
Tokens in same ternary branch get similar positions → attend easily. Creates hierarchical plateaus.
6.3 CantorRoPE (Wormhole Shortcuts)
pos_cantor(m) = trend × m + deviation × wormhole(m)
where:
trend = 1.0 (aligns macro slope with standard RoPE)
deviation = learnable perturbation scale
wormhole(m) = branch_path_alignment signal
Tokens with aligned branch paths can shortcut regardless of sequential distance.
6.4 Aligned Triad (Proven Configuration)
Standard: linear baseline "this comes after that"
Beatrix: hierarchical plateaus "these belong together"
Cantor: wormhole perturbations "these can shortcut"
All share same macro slope (trend=1.0), different micro structure.
6.5 Tower Assignment
Tower_positive = BeatrixRoPE(...) # hierarchical reasoning
Tower_negative = CantorRoPE(...) # wormhole reasoning
Signed pairs create differential forces in oscillator fusion.
7. Beatrix Oscillation Formulas (GeoFractal Router)
Physics-based fusion replacing static weighted sums. Tower outputs are force fields, not opinions to average.
7.1 Covariant Dynamics
dx/dt = v
dv/dt = -2β(t)·v - ω²·Log_x(x_ref) + κ(t)·u_towers + γ(t)·ξ_guide
where:
x = position on manifold
v = velocity in tangent space
β(t) = damping schedule
ω = spring frequency
x_ref = conditioning anchor
κ(t) = tower coupling strength
u_towers = force from tower opinions
γ(t) = guidance strength
ξ_guide = external guidance (DINO, text, etc.)
7.2 Manifold Operations
Log_x(y) = y - x # tangent vector from x toward y
Exp_x(v) = x + v # move along tangent vector
PT_{x→y}(v) = v # parallel transport (flat approx)
7.3 Tower Force Generation
For N towers with signed pairs:
force_i = proj_i(tower_output_i) # [B, manifold_dim]
u_towers = Σ_i w_i · force_i # weighted combination
Positive towers push toward structure.
Negative towers push away from collapse.
7.4 Tesla 3-6-9 Schedule
β(t) = β_base + resonance(t)
resonance(t) = 0.1·sin(3πt) + 0.05·sin(6πt) + 0.025·sin(9πt)
Resonant peaks at t = 1/3, 2/3, 1.0
Energy doesn't flow linearly — it oscillates.
7.5 Schedule Types
| Schedule | Formula |
|---|---|
| Constant | s(t) = start |
| Linear | s(t) = start + (end - start) · t |
| Cosine | s(t) = end + (start - end) · 0.5(1 + cos(πt)) |
| Sigmoid | s(t) = start + (end - start) · σ(12(t - 0.5)) |
| Tesla 3-6-9 | s(t) = linear(t) + resonance(t) |
7.6 Intrinsic Tension τ
τ = σ(gain · (Σ_i w_i · invariant_i - equilibrium))
where:
invariant_i = geometric invariants (Vol², edge stats, etc.)
w_i = learned per-invariant weights
gain = steepness of sigmoid response
equilibrium = learned bias
τ → 0: Pure spring (geometric constraint dominates)
τ → 1: Pure control (tower forces dominate)
7.7 Stability Criterion
Eigenvalues of linearized system:
λ = -β ± √(β² - (1-τ)ω²)
Overdamped: β² > (1-τ)ω² (stable, no oscillation)
Underdamped: β² < (1-τ)ω² (oscillatory)
Critical: β² = (1-τ)ω² (fastest convergence)
7.8 Energy Tracking
E_kinetic = 0.5 · ||v||²
E_potential = 0.5 · ω² · ||Log_x(x_ref)||²
E_total = E_kinetic + E_potential
Healthy training: E_total decreases over integration steps.
8. K-Simplex Linear (Near-Zero Params)
Replaces nn.Linear with geometric routing through simplex structure.
8.1 Architecture
Input (B, input_dim)
→ chunk into (B, num_simplices, K+1) groups
→ per-scalar entry into vertex (K+1 options)
→ private hidden projection per vertex (depth = K+1)
→ pairwise signal passages between all vertex pairs
→ attenuation gates on pairwise influence
→ exit: weighted sum of vertex states
Output (B, output_dim)
8.2 Parameter Count
Per simplex (K+1 inputs):
Entry: (K+1) × (K+1) × hidden
Vertex: (K+1) × hidden
Pairwise: C(K+1, 2) × 3 × hidden
Attenuate: C(K+1, 2) × 2
Exit: (K+1) × hidden + (K+1)
For K=4, input_dim=512:
103 simplices × 300 params = 30,900
vs nn.Linear: 262,656
Ratio: 0.118x (11.8% of linear params)
8.3 Structural Comparison
Structure size per simplex: (K+1) × (K+1) × C(K+1,2)
K=2: 3×3×3 = 27
K=4: 5×5×10 = 250
K=6: 7×7×21 = 1029
8.4 Results
Fashion-MNIST:
KSimplex-k4: 85.94% with 8,511 params
MLP baseline: 89.00% with 101,770 params
Ratio: 11.5× more parameter-efficient
Epoch 1: 84.28% test (instant useful signal)
Epoch 19: 85.94% test (stable convergence)
9. K-Simplex Deformation Limitations
Critical stability boundaries from extensive geometric explorer experiments.
9.1 Stability Zones by Configuration
| Configuration | Differentiation Zone | Collapse Threshold |
|---|---|---|
| k=1-4, edim=16 | 0.15 - 0.35 | ~0.50 |
| k=1-4, edim=32 | 0.15 - 0.50 | >2.0 |
| k=1-6, edim=16 | 0.35 - 0.45 | ~0.50 |
| k=1-6, edim=32 | 0.25 - 0.60 | >2.0 |
9.2 Embedding Dimension Safety Ratio
stability_ratio = edim / k_max
ratio ≥ 8× → Very stable, deform up to 2.0
ratio ≥ 4× → Comfortable margin
ratio ≥ 2× → Tight but functional
ratio < 2× → Dangerous, frequent invalidity
9.3 Deformation Behavior
Low deform (0 - 0.15):
Clear k-level hierarchy
Vol² decreases exponentially with k
Conservative but safe
Medium deform (0.15 - 0.35): ← OPTIMAL ZONE
Distinct geometric signatures per k
Maximum useful differentiation
Training should target this range
High deform (> 0.5):
Noise dominates
k-levels converge (lose meaning)
Geometric structure destroyed
9.4 Late-Stage K-Simplex Invalidity
As k increases:
- CM determinant computation becomes numerically unstable
- More edge configurations become geometrically impossible
- Deeper layers produce invalid simplex configurations
k=4 in 32D: stable with wide margin
k=5 in 32D: functional but tighter
k=6 in 32D: approaching invalidity ceiling
Recommendation: k=4 (pentachoron) as primary, k≤3 for tight budgets
9.5 Cross-Entropy Degeneracy Problem
Cross-entropy applied directly to simplex features:
→ Vertices converge (minimizing distance to class boundary)
→ Volume → 0 (simplex collapses)
→ α diverges from triadic equilibrium
→ Geometric structure destroyed after sufficient epochs
Solution: Use crystal loss or basin loss, NOT cross-entropy on geometric features.
10. Cross-Contrast Capacity Tests
Validating that geometric structure survives training and provides meaningful classification signal.
10.1 Geometric Cross-Contrastive Loss
sim_matrix = (x̂ @ x̂.T) / τ # [B, B] embedding similarity
cantor_positives = (|C(i) - C(j)| < θ_cantor) AND (|Vol(i) - Vol(j)| < θ_vol)
L_cross = -log(Σ_j∈positives exp(sim_ij) / Σ_j∈all exp(sim_ij))
where positives are defined by geometric proximity, not class labels
10.2 Capacity Invariants to Monitor
1. Vol² > 0 for all simplices (validity)
2. α ∈ [0.44, 0.50] (triadic equilibrium)
3. Edge length variance < threshold (structural uniformity)
4. Cantor prototype separation > margin (class distinctness)
5. Crystal distance to prototype ~ d_target (geometric alignment)
10.3 Differential Cross-Contrast (Tower Pairs)
For positive/negative tower pairs:
Δ_force = force_positive - force_negative
L_differential = -log(σ(Δ_force · direction_to_correct_class))
+ log(σ(Δ_force · direction_to_incorrect_class))
Signed pairs create differential forces, not just different opinions.
10.4 Cross-Scale Consistency
For scales s₁, s₂:
features_s1 = proj_s1(backbone_features)
features_s2 = proj_s2(backbone_features)
L_consistency = ||rank_order(sim_s1) - rank_order(sim_s2)||₂
Ensures geometric relationships are preserved across crystal scales.
10.5 OOD Detection via Geometric Violation
In-distribution: Vol² > 0, α stable, Cantor coherent
Out-of-distribution: Violations of above
OOD_score = (1 - σ(Vol² · 10⁶)) + (|α - 0.5|) + (1 - compat_max)
10.6 Scaling Limitation (Known)
Cross-contrastive loss across full vocabulary:
O(V²) pairwise comparisons
V=100 (CIFAR-100): 10K pairs → feasible
V=1000 (ImageNet): 1M pairs → expensive
V=50000 (tokenizer): 2.5B pairs → infeasible
Solution: Hierarchical contrastive within Cantor branches.
Only contrast within same coarse branch (routing highways).
Fine branches → local contrast only.
Appendix A: Proven Results Summary
| Model | Task | Accuracy | Params | Key Innovation |
|---|---|---|---|---|
| David | ImageNet (CLIP bigG) | 86% | ~120K | Multi-scale crystal |
| David | CIFAR-100 | 74.87% | 393K | Crystal prototypes |
| David | CIFAR-100 | ~92% | 78KB | Extreme compression |
| geo-beatrix | CIFAR-100 | 67.69% | — | NO attention, NO CE |
| KSimplex Attention | FMNIST | 89.13% | — | Geometric attention |
| KSimplex Attention | CIFAR-10 | 84.59% | — | Conv stem + geo attn |
| KSimplex Attention | CIFAR-100 | 69.08% | — | Multi-layer sharpening |
| KSimplex Linear | FMNIST | 85.94% | 8,511 | 11.5× efficiency |
| KSimplex LLM | Shakespeare | PPL 113 | 54M | 100% geo validity |
| Beeper v5 | Ethics | Coherent | Random | Architecture IS intelligence |
Appendix B: Formula Dependencies
┌─────────────┐
│ Cayley-Menger│ ← structural invariant
└──────┬──────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ K-Simplex│ │ Crystal │ │ Basin │
│ Channel │ │ Loss │ │ Compat │
└────┬─────┘ └────┬─────┘ └────┬─────┘
│ │ │
▼ ▼ ▼
┌──────────────────────────────────┐
│ Cantor Lens │
│ (Staircase + Alignment + Bias) │
└──────────────┬───────────────────┘
│
┌────────┼────────┐
▼ ▼ ▼
┌─────────┐ ┌──────┐ ┌──────────┐
│ Topo │ │ Osc │ │ KSimplex │
│ Ropes │ │ Fuse │ │ Linear │
└─────────┘ └──────┘ └──────────┘
Appendix C: What Kills Geometry (Known Failure Modes)
- Cross-entropy on geometric features → simplex collapse
- Distance on Cantor set → meaningless (use alignment)
- Deformation > 0.35 at edim/k < 4 → invalidity
- k > 4 without edim ≥ 8k → numerical instability
- Uniform Cantor level weights → hides 8× routing significance difference
- Resizing crystal anchors across scales → destroys pentachoron geometry (use separate init per scale)
- Dropout scaling with √dim → inconsistent information flow across scales