Hugging Face's logo

cloverx-id
/
LuminaV-Optimizer

PyTorch
Rust
English
optimizer
custom-optimizer
luminamoon
luminav
clm
td-clm
lso-eval
simd
avx512
vulkan
triton
memory-efficient
stochastic-rounding
low-vram
bf16
fp16
cuda
rocm
xpu
llm
transformer
deep-learning
research
Model card Files
xet
 Community

LuminaV Optimizer v1.6.0 (luminamoon v1.0.9)

Low-Precision BF16/FP16 Deep Learning Optimizer with Dual-Buffer Convex Lookahead Momentum (CLM), Single-Buffer Triangle-Defect Convex Lookahead Momentum (TD-CLM), Rational Padé [13/6] Bounding, and Stateless 32-Bit MurmurHash3 Stochastic Rounding

Official Codebase: github.com/cloverx-id/luminamoon | Docs Portal: cloverx-id.github.io/luminamoon

PyPI luminamoon Documentation Portal Changelog LSO-Eval v1.1.0 Config JSON License

PROJECT LOOMY

DOI PyPI version CI Status LSO-Eval v1.1.0 Documentation Python Versions License

Loomy (luminamoon) is the reference implementation and central repository of PROJECT LOOMY, providing low-precision numerical optimization, rational neural network layers, dataset text preprocessing utilities, and evaluation tools developed by Silver Moon (@cloverxion) and Lumina Moon (@cloverx-id) contributors.

Built with a modular Rust and Python architecture under PROJECT LOOMY, luminamoon provides components for CPU/GPU parameter updates, text dataset preprocessing, and numerical verification:

LSO-Eval v1.1.0 Evaluation & Verification Notebook: Open in Google Colab — Empirical Stage II–IV benchmarks executed on LuminaV v1.5.3 under native torch.bfloat16 (centered_defect=False).

Projects & Components

Project / Component Path Status Description Documentation
LuminaV Optimizer optimizer/ Active (Math v1.6.0) LuminaV v1.6.0 optimizer implementation with CPU SIMD (AVX-512/AVX2), Vulkan UMA, Triton AOT/JIT kernels, Triangle Family (buffer="T1", "T2"), optional contiguous bucket fusion, and Triton JIT pre-warmup. Optimizer Guide
LuminaMoon Data Engine data/ Active (v1.1.0) Modular text preprocessing library: AVX-512/AVX2/NEON SIMD text normalization, dual-layout StringTensor (CPU StringView & GPU contiguous buffers), Shannon entropy filtering, MinHash/XXH3-64 deduplication, PII masking, and 128B Ping-Pong checkpoint recovery. Data Engine Guide
LSO-Eval eval/ Active (v1.1.1) Algorithm-agnostic optimizer evaluation & verification (LSO-Eval v1.1.0): Z3 SMT solver checks, 2D trajectory benchmarks, numerical parity tests, and Two-Class 4B LLM & Vision evaluation. LSO-Eval Guide
Changelogs changelogs/ Active Version histories separating Software Package releases (v1.0.0–v1.1.0) and LuminaV Algorithm Math versions (v1.0.0–v1.6.0). Changelogs
Documentation docs/ Active Documentation index, developer guides (docs/optimize/, docs/dataloader/), and dependency version notes. Docs Index
Ra-Family Architecture python/luminamoon/nn/ Active (v1.1.0) Rational, tanh-free & square-root-free neural components: RaNorm, RaBatchNorm2d (with Conv2d weight folding), RaTanh, RaGELU, PosRaGLU (bounded gating in (0, 1)), and RaAttention with Cayley-Padé rational softmax. Triton GPU and Rust AVX2/AVX-512 SIMD acceleration. Ra-Family Guide
Python Package python/luminamoon/ Active (v1.1.0) Unified free-threaded (abi3t) Python namespace exposing LuminaV Optimizer, DataStreamLoader, Ra-Family neural architecture, and PyTorch/NumPy/HuggingFace integration. API Reference
Theoretical References references/ Active Mathematical formulas, component mapping, literature comparisons, and BibTeX citations for LuminaV, Ra-Family, and Data Engine. References Guide

Quickstart

Installation

Install the compiled package wheel from PyPI:

pip install luminamoon

Or build from source using Maturin and the Rust 2024 toolchain:

git clone https://github.com/cloverx-id/luminamoon.git
cd luminamoon
pip install maturin
maturin develop --release

Basic Usage (LuminaV Optimizer) 

import numpy as np
from luminamoon import LuminaV

# Initialize parameters
weights = np.random.randn(64, 10).astype(np.float32)
biases = np.zeros(10, dtype=np.float32)

# Instantiate LuminaV optimizer (uses default hyperparameters)
optimizer = LuminaV([weights, biases], lr=8e-4, weight_decay=0.08, buffer=2)

# Execute optimization step with explicit gradients
grad_w = np.random.randn(64, 10).astype(np.float32)
grad_b = np.random.randn(10, dtype=np.float32)
optimizer.step(grads=[grad_w, grad_b])

For PyTorch deep learning workflows, see the PyTorch Integration Guide.

LuminaV Hyperparameter Quick Reference

All default values below are exposed in IDE autocompletion and LuminaV.__init__ docstrings:

Parameter Type Default Description
lr float 8e-4 (0.0008) Learning rate (η).
betas Tuple[float, float] (0.9, 0.999) Exponential moving average factors (β₁, β₂) for first-moment momentum and second-moment variance.
eps float 1e-8 Numerical stability constant (ε) in the adaptive denominator.
weight_decay float 0.08 (8e-2) Decoupled weight decay coefficient.
tau float 0.8 Denominator temperature scaling factor (τ) for 2-buffer variance damping.
lam float 0.3 Triangle-defect penalty weight (λ) on `
cautious bool True Enables sign-aligned cautious update masking and lagged EMA scaling.
cautious_clamp_min float 0.5 Lower bound in (0.0, 1.0] for cautious alignment scaling.
buffer int | str 2 State mode: 2 ("dual", "clm") for 2-Buffer CLM, 1 ("single", "td-clm") for 1-Buffer TD-CLM, "T2" for 2-Buffer Triangle Family (tanh-free), or "T1" for 1-Buffer Triangle Family (tanh-free).
stochastic_rounding bool True Applies stateless 32-bit MurmurHash3 stochastic rounding when writing to bfloat16 / float16 weights.
bound bool True Enables adaptive update bounding.
bound_type str | int "coordinate" Clipping mode: "coordinate" (element-wise clip) or "radial" (global norm ratio).
bound_ratio float 0.05 Maximum update magnitude relative to parameter scale.
master_weights str | bool "none" Mixed-precision master weight mode for bfloat16/float16: "none" (False), "semi", or "full" (True).
execution str | int "auto" Hardware backend: "auto", "triton", "vulkan", "simd", or "scalar".
fused_single_pass bool True Fuses parameter update and scalar reduction statistics into a single kernel pass.
fuse_tensors bool False Coalesces parameters into contiguous 1D memory buffers to reduce kernel launch latency on fragmented architectures (e.g. Vision models / ViT). Not recommended for large LLMs or distributed FSDP training.
bucket_cap_mb float 500.0 Maximum memory capacity per contiguous bucket chunk in megabytes when fuse_tensors=True. Partitions large models into bounded chunks. Emits a UserWarning if explicitly specified when fuse_tensors=False.
prewarm_jit bool True Pre-compiles Triton GPU kernels on 64-element dummy buffers (lr=0.0) during initialization to avoid first-step JIT compilation delay.
ema_decay str | float "auto" EMA decay for lagged cautious/radial scalars ("auto" resolves to 0.80 for buffer=2/"T2" and 0.95 for buffer=1/"T1").
fsdp_sync bool False Synchronizes global reduction scratch buffers across torch.distributed FSDP shards.
seed int 1337 Deterministic PRNG seed for stateless stochastic rounding.
version Optional[str] "1.6.0" Selects the active LuminaV mathematical specification version.

Software vs. Algorithm Versioning (v1.1.0 / Math v1.6.0)

luminamoon maintains two distinct, decoupled version lineages (documented in changelogs/):

  1. Software Package Version (v1.1.0):

    • Tracks the monorepo distribution, CPython FFI bindings, Rust SIMD crates, neural architecture (luminamoon.nn), and Data Engine utilities on PyPI (pip install luminamoon).
    • To reproduce or execute older software package releases in Python, install the desired release directly via standard package management (pip install luminamoon==1.0.9) or check out official git release tags.
    • See changelogs/SOFTWARE_CHANGELOG.md.

  2. LuminaV Algorithm Math Version (v1.6.0):

    • Tracks the mathematical optimization formulas, convergence envelopes, and hardware kernels of the LuminaV Optimizer independently from pip package updates.
    • Four canonical mathematical formulations are supported: 1.6.0 (canonical latest with Triangle Family T1/T2 and Padé rational bounds), 1.5.5 (bucket fusion and Triton pre-warmup), 1.5.4 (Bitwise-BF16 state promotion), and 1.5.3 (single-buffer TD-CLM).
    • Features strict capability enforcement (Option B): locking to older mathematical versions (e.g., version="1.5.4") strictly rejects features introduced in later eras (such as buffer="T2" or fuse_tensors=True) with actionable error messages, ensuring mathematical reproducibility for scientific publications.
    • See changelogs/LUMINAV_ALGORITHM_CHANGELOG.md.
import luminamoon
from luminamoon import LuminaV
import luminamoon.data as lmd

# 1. Inspect Software Package Version vs. Algorithm Math Version
print(luminamoon.__version__)    # "1.1.0" (Software package release)
print(lmd.get_version())         # "1.1.0" (Data Engine package module)
print(LuminaV.ALGORITHM_MATH)    # "1.6.0" (Canonical mathematical formulation)
print(LuminaV.ALGORITHM_SPEC)    # "1.6.0" (Backward-compatible alias)
print(LuminaV.getVersion())      # "1.6.0"

# 2. Inspect capability matrix of any supported specification
caps = LuminaV.get_capabilities("1.5.5")
print(caps["allowed_buffers"])       # (1, 2)
print(caps["supports_triangle_family"]) # False (introduced in 1.6.0)

# 3. Lock algorithm version for scientific checkpoint reproducibility
opt = LuminaV([weights, biases], lr=8e-4, version="1.5.5")
opt.setVersion("1.5.5")          # Instance-level version lock
LuminaV.setVersion("1.5.5")      # Class-level default specification

Basic Usage (LuminaMoon Data Engine: Fused Text Preprocessing Pipeline) 

from luminamoon.data import FusedPipeline, StringTensor, create_resumable_collate, ResumablePipeline

# 1. Fused single-pass text preprocessing in Rust
pipeline = FusedPipeline(
    trim_whitespace=True,
    normalize_newlines=True,  # AVX-512 / AVX2 / NEON SIMD
    min_chars=10,
    min_entropy=2.0,
)
string_tensor = pipeline(["  sample 1\r\n", "sample 2 with proper prose text"])

# 2. Export to Apache Arrow, Polars, and PyTorch buffers
polars_series = string_tensor.to_polars()
arrow_array = string_tensor.to_arrow()
offsets_tensor, payload_tensor = string_tensor.to_torch_gpu(device="cuda")

# 3. Stream dataset with background prefetching & 128B Ping-Pong checkpointing
pipe = ResumablePipeline(
    source=raw_data,
    fused_pipeline=pipeline,
    batch_size=64,
    prefetch_batches=4,
    checkpoint_path="./cache/checkpoint.meta",
)
for batch in pipe:
    # Training step...
    pipe.commit()  # Saves Dual-Slot A/B checkpoint and deduplication WAL

For detailed architecture, Shannon entropy filtering, MinHash deduplication, and PyTorch collate_fn adapters, see the Data Engine Guide.

Basic Usage (Ra-Family Rational Neural Architecture) 

import torch
from luminamoon.nn import RaNorm, PosRaGLU, RaAttention

# 1. Square-root-free token normalization (L1-mean)
norm = RaNorm(512)
x = torch.randn(2, 64, 512)
x_norm = norm(x)

# 2. Rational gated linear unit with strictly positive bounds in (0, 1)
mlp = PosRaGLU(512, hidden_features=2048)
x_ffn = mlp(x_norm)

# 3. Cayley-Padé rational multi-head attention (SFU-free softmax)
attn = RaAttention(dim=512, num_heads=8, qk_norm=True, causal=True)
out = attn(x_norm)

For complete architecture, parameter folding for Conv2d, and standalone NumPy usage, see the Ra-Family Guide.

Verification & CI/CD

All pull requests and releases are tested across Linux, Windows, and macOS:

# Run static analysis across the Rust workspace
cargo clippy --workspace --all-targets -- -D warnings

# Run Rust workspace unit and parity tests (Optimizer + Data Engine)
cargo test --workspace

# Run Python static type check
npx pyright python/luminamoon

# Run standalone NumPy, PyTorch, and Data Engine tests
python optimizer/tests/test_standalone_numpy.py
python optimizer/tests/test_end_to_end_training.py
python -B tests/test_data_modular.py
python -B tests/test_gpu_vs_cpu_strings.py

# Run LSO-Eval verification
python -m eval

Developer Resources

For contributors working on PROJECT LOOMY (luminamoon):

Theoretical References & Citation

LuminaMoon's algorithms are built upon published mathematical principles, including Maligranda's triangle inequality envelopes, rational neural networks, and decoupled optimization mechanics.

For the full component mapping, literature comparisons, and BibTeX entries, see references/README.md and references/bibliography.bib.

To cite this repository in academic work:

@misc{loomy2026,
  author = {Silver Moon and {Lumina Moon contributors}},
  title = {Loomy: Low-Precision Numerical Optimization, Rational Neural Architecture, and Data Processing Repository},
  year = {2026},
  publisher = {GitHub},
  howpublished = {\url{https://github.com/cloverx-id/luminamoon}},
  doi = {10.5281/zenodo.23973326},
  note = {PROJECT LOOMY repository}
}

Authors & Maintenance

PROJECT LOOMY (luminamoon) is developed and maintained by:

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

LuminaV Optimizer v1.6.0 (luminamoon v1.1.0)

LuminaV is a numerical optimization library implementing the LuminaV algorithm for low-precision training (BF16/FP16) without requiring FP32 master weights. Using bitwise stochastic rounding, it supports hardware execution across CPU SIMD (AVX-512, AVX2), Vulkan compute shaders, and Triton GPU kernels, with buffer interoperability for NumPy and PyTorch.

Features

  • Contiguous Bucket Fusion & JIT Pre-Warmup: Optionally coalesces parameter groups into 1D contiguous allocations (fuse_tensors=True) partitioned into bounded chunks (bucket_cap_mb=500.0) to reduce kernel launch overhead on models with many tensors, and pre-warms Triton JIT kernels during initialization (prewarm_jit=True).
  • Dual-Buffer CLM, Single-Buffer TD-CLM & Triangle Family (T-Family, T1 / T2): Implements Convex Lookahead Momentum (CLM) (buffer=2), Triangle-Defect Convex Lookahead Momentum (TD-CLM) (buffer=1), and the tanh-free Triangle Family (buffer="T2" 2-Buffer Triangle Family Mode and buffer="T1" 1-Buffer Triangle Family Mode with the Triangle Inequality envelope, triangle-defect penalty, and rational update bounds). Single-buffer modes (1 and "T1") reduce optimizer state memory by 50% compared to two-buffer modes (2 and "T2").
  • Optional Framework Dependencies: The core package has dependencies = [] with a compiled binary under 1 MB. PyTorch, NumPy, and Triton are optional extras.
  • Hardware Acceleration:
    • x86_64 CPU: AVX-512 and AVX2 SIMD kernels parallelized across cores via Rayon.
    • Vulkan 1.3/1.4: Unified Memory Architecture (UMA) SPIR-V compute shaders for integrated and discrete GPUs.
    • Triton AOT / JIT: Compiled kernels for NVIDIA CUDA (SM75–SM120) and AMD ROCm (RDNA3/CDNA).
    • Scalar Core: Portable Rust fallback for other architectures (ARM/aarch64).
  • Framework Integration:
    • NumPy 2.x: Direct buffer access via the Python buffer protocol and __array_interface__.
    • PyTorch: Standard torch.optim.Optimizer interface with FSDP collective synchronization and optional FP16/BF16 master weights.
  • Runtime Compatibility: Built with CPython Stable ABI (abi3 and abi3t), supporting Python 3.10 through 3.15+ including free-threaded execution (PEP 703).

Architecture & Dispatch

The runtime dispatcher selects the execution backend based on tensor device, memory size, and available hardware:

Workspace Crate Structure 

optimizer/
├── Cargo.toml
├── src/
│   ├── lib.rs              # Rust crate entry point
│   ├── optimizer.rs        # Core PyO3 optimizer implementation
│   └── python_ffi/         # Python-Rust FFI bridge (mod.rs, config_parser.rs, extract.rs, worker.rs)
├── crates/
│   ├── luminav-core/       # Mathematical formulas, PRNG, and state structs
│   ├── luminav-simd/       # AVX-512 and AVX2 explicit SIMD intrinsics
│   ├── luminav-vulkan/     # Vulkan 1.3/1.4 UMA compute pipeline (Ash)
│   ├── luminav-triton-aot/ # Triton AOT runner for CUDA and ROCm HIP
│   └── luminav-dispatch/   # Hardware capability detection and routing
├── tests/                  # Standalone NumPy and PyTorch verification tests
└── tools/                  # AOT compilers for SPIR-V and Triton

Installation & Build

From Wheel (PyPI) 

pip install luminamoon

From Source (Maturin) 

git clone https://github.com/cloverx-id/luminamoon.git
cd luminamoon
pip install maturin
maturin develop --release

Optional Dependencies 

# PyTorch integration
pip install "luminamoon[torch]"

# Triton GPU acceleration (CUDA / ROCm)
pip install "luminamoon[gpu]"

# All optional dependencies
pip install "luminamoon[all]"

Quickstart

Standalone NumPy (No PyTorch Required) 

import numpy as np
from luminamoon import LuminaV

w = np.random.randn(64, 10).astype(np.float32)
b = np.zeros(10, dtype=np.float32)

optimizer = LuminaV(
    [w, b],
    lr=1e-2,
    betas=(0.9, 0.999),
    weight_decay=1e-2,
    buffer=2,  # 2 = Dual-Buffer CLM, 1 = Single-Buffer TD-CLM
)

grad_w = np.random.randn(64, 10).astype(np.float32)
grad_b = np.random.randn(10).astype(np.float32)

optimizer.step(grads=[grad_w, grad_b])

PyTorch Integration 

import torch
import torch.nn as nn
from luminamoon import LuminaV

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = nn.Sequential(
    nn.Linear(128, 256),
    nn.ReLU(),
    nn.Linear(256, 10),
).to(device)

criterion = nn.CrossEntropyLoss()
optimizer = LuminaV(
    model.parameters(),
    lr=8e-4,
    betas=(0.9, 0.999),
    weight_decay=8e-2,
    buffer=2,
    bound=True,
    bound_type="coordinate",
    cautious=True,
    fuse_tensors=False,    # Coalesce parameter groups into contiguous buckets
    bucket_cap_mb=500.0,   # Maximum chunk allocation size in MB
    prewarm_jit=True,      # Pre-compile Triton JIT kernels at initialization
)

inputs = torch.randn(64, 128, device=device)
targets = torch.randint(0, 10, (64,), device=device)

optimizer.zero_grad()
loss = criterion(model(inputs), targets)
loss.backward()
optimizer.step()

Hardware Verification

To inspect detected hardware backends and active dispatch targets:

from luminamoon import LuminaV

print(LuminaV.detect_hardware())

Documentation & Library Audits

Running Tests 

# Lint workspace
cargo clippy --workspace --all-targets -- -D warnings

# Rust workspace unit and parity tests
cargo test --workspace

# Standalone NumPy test
python optimizer/tests/test_standalone_numpy.py

# PyTorch integration test
python optimizer/tests/test_end_to_end_training.py

References

  • Triangle Inequality Defect & Lookahead: L. Maligranda, "Some remarks on the triangle inequality for norms," Banach J. Math. Anal., vol. 2, no. 2, pp. 31–41, 2008; M. R. Zhang et al., "Lookahead optimizer: k steps forward, 1 step back," NeurIPS, 2019.
  • Belief Variance & Decoupled Weight Decay: J. Zhuang et al., "AdaBelief optimizer: Adapting stepsizes by the belief in observed gradients," NeurIPS, 2020; S. Taniguchi et al., "ADOPT: Modified Adam can converge with any β₂ with the optimal rate," NeurIPS, 2024; I. Loshchilov and F. Hutter, "Decoupled weight decay regularization," ICLR, 2019.
  • Cautious Alignment & Trust-Region Clipping: K. Liang et al., "Cautious optimizers: Improving training with one line of code," ICLR, 2026; A. Brock et al., "High-performance large-scale image recognition without normalization," ICML, 2021.
  • Padé Rational Approximants & Stochastic Rounding: A. Molina et al., "Padé activation units: End-to-end learning of flexible activation functions in deep networks," ICLR, 2020; S. Gupta et al., "Deep learning with limited numerical precision," ICML, 2015; P. Zamirai et al., "Revisiting BFloat16 training," arXiv:2010.06192, 2021.

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.

LSO-Eval v1.1.0: Formal Verification & 4B LLM Qualification (Qwen3.5-4B-Base)

LuminaV (v1.5.3) is formally verified and empirically qualified under the LSO-Eval (v1.1.0) standard across Stage I (12/12 Z3 SMT solver UNSAT safety proofs & 9/9 SymPy/TOST precision checks), Stage II (B1–B5 non-convex surfaces, λ* = 0.3008 equilibrium, and 1-Pass EMA parity), Stage III (MProxyLLM F₁ ∧ F₂ ∧ F₃ screening), and Stage IV (Qwen/Qwen3.5-4B-Base 100% from-scratch pretraining in native torch.bfloat16 with cryptographic XXH3-64 zero-leakage validation).

Empirical Version Attribution (LuminaV v1.5.3): The Stage II–IV empirical qualification tables below were evaluated on LuminaV v1.5.3 under native torch.bfloat16 (dtype_mode=1, centered_defect=False). In the v1.5.3 evaluation data loader script, parallel dataset records were sampled by filtering uniqueness based on joint (source, domain) pairs rather than allocating unique records evenly per domain up to each domain's total target allocation; because all candidate and baseline optimizers shared this identical deterministic stream schedule, relative cross-optimizer comparisons in the v1.5.3 tables remain strictly controlled.

Stage IV-B: 2048 Context Full Schedule (20,000 Steps, Grad Accum 1x, 450,560 Held-Out Tokens, SHA256=108e517e1c181cb6)

LSO-Eval Stage IV-B Qwen3.5-4B Qualification (ctx=2048)
Optimizer Configuration Class Final EMA Val PPL ΔPPL VRAMpeak VRAMopt Max Rt,l Step / Opt Time LSO-Eval Status
LuminaV_Buffer2_Dual (8e-4) Class I (Dual) 4.5549 89.14 -43.43 43.89 GB 15.86 GB 0.0189 297.1 / 48.31 ms PASSED (Class I)
LuminaV_Buffer1_TDCLM_3e4 (3e-4) Class II (Single) 4.9173 165.13 +32.56 35.97 GB 7.94 GB (-50.0%) 0.0087 283.5 / 36.99 ms PASSED (Class II)
AdamW (Native BF16, 3e-4) Baseline 4.7621 132.57 +0.00 43.89 GB 15.86 GB 0.0037 288.0 / 41.48 ms PASSED
AdamW_FP32_Master (3e-4) Master Ref 4.8116 124.43 -8.14 80.11 GB 52.08 GB 0.0045 370.1 / 123.33 ms PASSED

Stage IV-A: 512 Context Qualification (4,800 Steps, Grad Accum 8x, 450,560 Held-Out Tokens, SHA256=4d278cb87e4abbe1)

LSO-Eval Stage IV-A Qwen3.5-4B Qualification (ctx=512)
Optimizer Configuration Class Final EMA Val PPL ΔPPL VRAMpeak VRAMopt Max Rt,l Step / Opt Time LSO-Eval Status
LuminaV_Buffer2_Dual (8e-4) Class I (Dual) 4.7621 141.96 -39.21 36.75 GB 15.88 GB 0.0047 841.6 / 48.37 ms PASSED (Class I)
LuminaV_Buffer1_TDCLM_3e4 (3e-4) Class II (Single) 5.2569 195.58 +14.41 28.81 GB 7.94 GB (-50.0%) 0.0120 829.4 / 36.77 ms PASSED (Class II)
LuminaV_Buffer1_TDCLM (8e-4) Class II (Single) 5.2953 221.07 +39.90 28.81 GB 7.94 GB (-50.0%) 0.0191 830.7 / 36.46 ms PASSED (Class II)
AdamW (Native BF16, 3e-4) Baseline 5.4497 181.17 +0.00 36.75 GB 15.88 GB 0.0034 835.4 / 41.58 ms PASSED
AdamW_FP32_Master (3e-4) Master Ref 5.0234 153.78 -27.39 75.36 GB 54.49 GB 0.0040 919.5 / 123.35 ms PASSED

VRAM Decomposition: VRAM_peak = VRAM_base + VRAM_opt, where the fixed non-optimizer baseline (VRAM_base = W_BF16 + G_BF16 + A_ctx) is 20.87 GB at ctx=512 and 28.03 GB at ctx=2048.

Citation 

@software{luminamoon2026luminav_code,
  author       = {{Silver Moon (cloverxion) and Lumina Moon Contributors}},
  organization = {Lumina Moon (cloverx-id)},
  title        = {{LuminaMoon & LuminaV Optimizer: Native Rust SIMD, Vulkan, and Triton Deep Learning Suite}},
  year         = {2026},
  publisher    = {GitHub / Hugging Face / PyPI},
  version      = {1.0.9},
  url          = {https://github.com/cloverx-id/luminamoon}
}
Downloads last month
881
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for cloverx-id/LuminaV-Optimizer