Nyx Compiler Pipeline & Architecture
Exhaustive architectural breakdown of the Nyx dual-engine compilation pipeline: the high-throughput Bootstrap Compiler and the 100% Self-Hosted 7-Phase MLIR Compiler.
nyxc/main.exe) is fully operational and written 100% in Nyx syntax, successfully executing all 7 compilation phases from Lexing to MLIR IR generation. All 23/23 test suites and interactive domain showcases pass with 100% green verification.
1. Dual-Pipeline Compiler Architecture
Nyx employs a robust two-tier compiler strategy designed for rapid development iteration and maximum production performance:
┌─────────────────────────────────────────────────────────────────────────────┐
│ Tier 1: Bootstrap Compiler (Rust → C99) │
│ Source (.nyx) ➔ Lexer ➔ Pratt Parser ➔ Type Checker ➔ Region Analyzer │
│ ➔ C Codegen + rt_stdlib ➔ GCC/Clang ➔ Native Binary │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼ (Compiles self-hosted compiler)
┌─────────────────────────────────────────────────────────────────────────────┐
│ Tier 2: Self-Hosted Compiler (Nyx → MLIR/LLVM) │
│ Source (.nyx) ➔ nyxc/lexer ➔ nyxc/parser ➔ nyxc/typechecker │
│ ➔ nyxc/region_infer ➔ nyxc/monomorphize │
│ ➔ nyxc/closure_convert ➔ nyxc/codegen_mlir ➔ LLVM IR ➔ Native│
└─────────────────────────────────────────────────────────────────────────────┘2. Phase 1: Bootstrap Compiler (nyx-bootstrap)
The bootstrap compiler provides rapid turnaround times with source-level C99 emission and incremental compilation caching:
| Stage | Input / Output | Core Subsystems | Status |
|---|---|---|---|
| 1. Lexer | Source code ➔ Token stream | Unicode UTF-8 decoding, numeric suffixes (f64, i64, u32), byte literals (b'A'), raw strings |
✅ 100% Complete |
| 2. Parser (Pratt) | Tokens ➔ Abstract Syntax Tree (AST) | Precedence climbing, pattern matching destructuring, nominal structs, sum types, traits, closures, slice syntax (s[a..b]) |
✅ 100% Complete |
| 3. Type Checker | AST ➔ Typed AST (TIR) | Bidirectional Hindley-Milner inference, generic unification, method resolution, exhaustiveness checking | ✅ 100% Complete |
| 4. Region Analysis | Typed AST ➔ Ownership Scopes | Escape analysis, arena allocation scoping, zero-overhead bulk deallocation | ✅ 100% Complete |
| 5. C Codegen | Typed AST ➔ C99 + Header Linkage | Direct C emission linking native domain engines (rt_arena, rt_cloud, rt_gis, rt_ml, rt_chain, rt_robotics) |
✅ 100% Complete |
3. Phase 2: Self-Hosted Compiler (nyxc)
Written entirely in native Nyx syntax (nyxc/*.nyx), the self-hosted compiler features a modular 7-stage compilation pipeline targeting Multi-Level Intermediate Representation (MLIR):
1. Self-Hosted Lexer (nyxc/token.nyx & nyxc/lexer.nyx)
High-throughput scanner recognizing all Nyx keywords, string interpolation, escape sequences, and multi-character operators with detailed line and column source mapping.
Status: ✅ 100% Complete & Verified
2. Self-Hosted Parser (nyxc/ast.nyx & nyxc/parser.nyx)
Recursive descent Pratt parser handling top-level module declarations, function definitions, generic type parameters, pattern matching arms, expressions, and statements.
Status: ✅ 100% Complete & Verified
3. Self-Hosted Type Checker (nyxc/typechecker.nyx)
Full bidirectional type inference supporting generic type arguments, algebraic sum types (enums), nominal struct validation, and static method resolution.
Status: ✅ 100% Complete & Verified
4. Region Inference Engine (nyxc/region_infer.nyx)
Automated Type-and-Effect constraint solver that infers lexical arena lifetimes, detects variable escapes across function boundaries, and ensures zero-GC deterministic memory reclamation.
Status: ✅ 100% Complete & Verified
5. Monomorphization Pass (nyxc/monomorphize.nyx)
Specializes generic structs and generic functions into concrete, zero-cost native representations with name mangling.
Status: ✅ 100% Complete & Verified
6. Closure Conversion (nyxc/closure_convert.nyx)
Transforms closures with captured lexical variables into explicit environment structs and plain function pointers.
Status: ✅ 100% Complete & Verified
7. MLIR Dialect Code Generator (nyxc/codegen_mlir.nyx)
Emits structured MLIR representation using the custom nyx.* dialect (nyx.func, nyx.struct, nyx.region, nyx.async, nyx.if, nyx.while, nyx.for, nyx.match) for downstream optimization passes.
Status: ✅ 100% Complete & Verified
4. Self-Hosted CLI Interface (nyxc/main.nyx)
The compiled self-hosted CLI supports granular pipeline inspection flags:
# Run full self-hosted 7-phase compilation pipeline
nyxc/main.exe source.nyx --all
# Inspect specific compiler phase outputs
nyxc/main.exe source.nyx --tokens # Emit token stream
nyxc/main.exe source.nyx --ast # Emit AST
nyxc/main.exe source.nyx --types # Emit Typed AST
nyxc/main.exe source.nyx --regions # Emit Region-Annotated IR
nyxc/main.exe source.nyx --generics # Emit Monomorphized AST
nyxc/main.exe source.nyx --closures # Emit Closure-Converted AST
nyxc/main.exe source.nyx --mlir # Emit MLIR Dialect IR5. Incremental Compilation & Build Performance
- Source Hash Caching (
.nyx_cache/): Automatically hashes module dependencies to skip recompilation for unchanged AST trees. - Linker Optimization: Skips GCC/LLVM linking when target binaries are newer than all object dependencies.
- Sub-Millisecond Type Checking: Optimized symbol tables and region constraint unification deliver instant IDE feedback through the Nyx Language Server Protocol (LSP).