Nyx-Tensor v1.0.0 (vLLM/llama.cpp Class)

SIMD-Accelerated Quantized Matrix Math & PagedAttention KV-Cache Runtime.

BACKEND: AVX-512 / METAL THROUGHPUT: 168.4 TOK/S KV FRAGMENTATION: 0.0%
PagedAttention KV-Memory Block Map
64 Blocks · BlockSize: 16 Tokens
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
Allocated Blocks (Seq 1 & 2) Free Virtual Page Blocks
Quantized Weight Compression & Perplexity
PRECISION VRAM USAGE (7B MODEL) PERPLEXITY SPEEDUP
FP32 (Standard) 28.0 GB 5.68 1.0x (Baseline)
FP16 (Half) 14.0 GB 5.68 2.1x
INT8 (Q8_0) 7.2 GB 5.69 3.4x
INT4 (Q4_K_M) 4.1 GB 5.72 4.8x
Transformer Forward Pass
[nyx-tensor] Runtime ready: RoPE / SwiGLU / RMSNorm kernels compiled.
[nyx-tensor] PagedAttention virtual allocator initialized with 64 blocks.