Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
177 commits
Select commit Hold shift + click to select a range
cb3d5ef
✨ Add ndc_vm crate skeleton for bytecode VM
timfennis Mar 2, 2026
0cb0105
📚 Chapter 14 + 15
timfennis Mar 2, 2026
7e92961
🚧 WIP bytecode VM compiler and chunk
timfennis Mar 3, 2026
5073062
New Value type for the VM, and constant compilation
timfennis Mar 3, 2026
90b6f9d
WIP
timfennis Mar 3, 2026
a1ded53
Fix warnings
timfennis Mar 3, 2026
d22e4d8
Basic conditions implemented
timfennis Mar 3, 2026
2cd2f90
Make function definition expression contain a signature instead of ex…
timfennis Mar 4, 2026
b565f58
Add tests for the compiler output
timfennis Mar 4, 2026
6d7ac15
Add disassembling functionality
timfennis Mar 4, 2026
4285fda
🌍 Add GetGlobal opcode and wire up Call instruction in the VM
timfennis Mar 4, 2026
e9a66a4
🔧 Bridge stdlib globals into VM and fix locals to use the stack
timfennis Mar 4, 2026
6919a39
🔀 Add runtime dynamic dispatch for overloaded functions in the VM
timfennis Mar 4, 2026
6ae41bb
Fix compilation of return opcodes
timfennis Mar 5, 2026
7f4c9d7
✅ Fix Statement/Block compilation and add compiler tests
timfennis Mar 5, 2026
d8a2229
♻️ Add Value::unit() constructor and use it in the compiler
timfennis Mar 5, 2026
fb4548a
♻️ Extract dispatch_call to remove duplicated Call logic
timfennis Mar 5, 2026
991735b
✅ Run tests against both tree-walk and VM interpreter
timfennis Mar 5, 2026
8cf5b14
🔌 Wire VM print output to the interpreter's captured buffer
timfennis Mar 5, 2026
ceb4481
🏗️ Make declarations pure statements and fix VM local variable stack …
timfennis Mar 5, 2026
bf4a2c7
🔍 Add vm-trace feature for source-annotated instruction tracing
timfennis Mar 5, 2026
7089192
💥 Add todo!() panics for unimplemented compiler branches
timfennis Mar 5, 2026
6a25c8b
🧹 Move vm-trace ip capture inside cfg block, suppress unused param wa…
timfennis Mar 5, 2026
7e1c4cf
Benchmark tool + tuple and list support
timfennis Mar 5, 2026
e234d51
⚡ Pre-allocate stack and avoid to_vec() on native calls
timfennis Mar 5, 2026
3af5082
Upvalue WIP + cargo fmt
timfennis Mar 6, 2026
7f74623
🔒 Implement closure capture and upvalue resolution in the VM
timfennis Mar 6, 2026
47fbe07
📦 Make VM lists reference-counted and sync mutations through the bridge
timfennis Mar 6, 2026
36c88f5
🏗️ Pre-allocate locals at function entry to fix stack corruption
timfennis Mar 6, 2026
9d4f5e6
Make [] indexing a function call (#94)
timfennis Mar 8, 2026
2b270da
Cargo fmt
timfennis Mar 8, 2026
302dd33
🔄 Add FunctionBody::Opaque to round-trip VM functions through the int…
timfennis Mar 8, 2026
9b4e355
WIP
timfennis Mar 9, 2026
ee0d577
WIP on upvalue hoisting in the analyzer
timfennis Mar 9, 2026
15618cc
🚧 Analyser-side upvalue hoisting (steps 1-4 of PLAN_upvalue_mutation.md)
timfennis Mar 10, 2026
ca055d8
✅ Fix upvalue mutation and iteration scope transparency
timfennis Mar 11, 2026
6e48b0e
♻️ Split compiler tests into separate crate
timfennis Mar 11, 2026
c0262de
✅ Implement analyser-side upvalue hoisting in the VM (steps 5 & 6)
timfennis Mar 11, 2026
895ce10
Remove completed plan
timfennis Mar 11, 2026
6543788
🔍 Replace unreachable todo in FunctionDeclaration compiler
timfennis Mar 11, 2026
3ef12aa
Add nested break/continue test cases
timfennis Mar 13, 2026
b45e9e7
Refactor VM compiler for readability
timfennis Mar 13, 2026
75ea6cc
Implement break/continue in VM compiler and add vm-ready markers
timfennis Mar 13, 2026
2a9ea38
Simplify compile_while by returning loop_start from new_loop_context
timfennis Mar 13, 2026
4245283
Implement op-assignment for identifier lvalues
timfennis Mar 13, 2026
e4eca05
✅ Fix VM return-outside-function error and function equality
timfennis Mar 13, 2026
8f5b12e
✅ Implement index assignment via stdlib []= function call
timfennis Mar 14, 2026
c367632
✅ Mark string index assignment tests as vm-ready
timfennis Mar 14, 2026
403f2c5
✅ Mark 3 more passing VM tests as vm-ready
timfennis Mar 14, 2026
fa8c04f
✅ Implement for-loops, ranges, and iterators in the bytecode VM
timfennis Mar 14, 2026
345dcec
✅ Fix compiler test + mark 14 more tests as vm-ready
timfennis Mar 14, 2026
7ce3867
♻️ Refactor iteration_local_slot helper in analyser
timfennis Mar 14, 2026
0df6a72
✅ Implement op-assign index (x[i] += v) in the bytecode VM
timfennis Mar 14, 2026
b932be4
✅ Implement tuple destructuring (Unpack opcode) in the bytecode VM
timfennis Mar 15, 2026
60cb1d8
✅ Enable VM closure callbacks from stdlib HOFs (map, filter, fold, al…
timfennis Mar 15, 2026
b96cca9
✅ Mark 6 more tests as vm-ready
timfennis Mar 15, 2026
d38e020
🐛 Fix two regressions introduced by HOF callback commit
timfennis Mar 15, 2026
3e6d18a
🐛 Fix sequence op-assignment error handling
timfennis Mar 15, 2026
d4041b8
✅ Implement index assignment in destructuring (compile_lvalue)
timfennis Mar 15, 2026
bf46571
✅ Implement map literals in VM compiler
timfennis Mar 15, 2026
c44cc05
✅ Add proper error handling for non-iterable values in VM
timfennis Mar 15, 2026
50042b7
🔄 Narrow VM error spans for for-loop sequences
timfennis Mar 15, 2026
0650d47
⬜ Narrow spans on synthetic VM opcodes
timfennis Mar 15, 2026
79f8408
✅ Propagate stdlib errors from VM native functions
timfennis Mar 15, 2026
838b795
✅ Implement destructuring and map iteration in VM for-loops
timfennis Mar 15, 2026
8e416d4
🔧 Fix compiler warnings and mark 11 more tests vm-ready
timfennis Mar 15, 2026
6c7e8b5
✅ Support string destructuring in VM and improve unpack error messages
timfennis Mar 15, 2026
b5f3303
✅ Add test for invalid string destructuring and unify unpack error me…
timfennis Mar 15, 2026
3079ed0
🔧 Refactor vm.rs for readability
timfennis Mar 15, 2026
7648856
✅ Type-check bool conditions in VM JumpIfFalse/JumpIfTrue
timfennis Mar 15, 2026
8e1e37b
✅ Mark 038_invalid_if_gaurd_in_comprehension as vm-ready
timfennis Mar 15, 2026
c541ece
✅ Fix closure-over-loop-variable and mark 041/014 as vm-ready
timfennis Mar 15, 2026
6d510aa
✅ Fix tuple range slicing in VM (042_unit_is_tuple)
timfennis Mar 15, 2026
381c979
✅ Fix Hash/PartialEq mismatch for VM-bridged function values (007/005)
timfennis Mar 15, 2026
24bd8c2
✅ Mark 009_slicing 003/004/006 as vm-ready
timfennis Mar 15, 2026
03e9207
✅ Fix op-assign on map values; mark 007/018 as vm-ready
timfennis Mar 15, 2026
bd4bf82
✅ Implement map/set comprehensions in VM; fix accumulator slot with d…
timfennis Mar 15, 2026
64d075a
✅ Implement MinHeap, MaxHeap, Deque in VM; fix None identifier in com…
timfennis Mar 15, 2026
24cf176
Fix int/float comparison tiebreaker in VM; add ordering edge-case tests
timfennis Mar 15, 2026
f3c475c
✅ Cover cross-type numeric ordering gaps in VM OrdValue
timfennis Mar 16, 2026
ac39892
Expand CLAUDE.md with commands and architecture overview
timfennis Mar 16, 2026
1cfcdbe
Implement unbounded range (start..) in bytecode VM
timfennis Mar 16, 2026
682eaba
Mark slicing and deque indexing tests as vm-ready
timfennis Mar 16, 2026
bffac36
Fix upvalue access in transparent iteration scopes
timfennis Mar 16, 2026
f2c5c0b
✅ VM op-assign: use in-place path when resolved_assign_operation matc…
timfennis Mar 16, 2026
8336451
Implement memoization for pure fn in the bytecode VM
timfennis Mar 16, 2026
53c6616
Implement vectorized tuple math in the bytecode VM
timfennis Mar 16, 2026
b60f021
Fix unbalanced if as expression: always yields a value (unit when false)
timfennis Mar 16, 2026
09747f0
fix indentation in test
timfennis Mar 16, 2026
ac44e3b
VM: return error instead of panicking when calling a non-function value
timfennis Mar 16, 2026
2e710f2
Fix big_int_ranges: return error instead of panicking, improve message
timfennis Mar 16, 2026
91a9e53
Fix weird_iterator_cloning VM test; remove borrow_mut test
timfennis Mar 16, 2026
308c95c
Remove // vm-ready markers; all VM tests run by default
timfennis Mar 16, 2026
4d9e43e
Fix two VM bugs: open upvalues and destructive list unpack (#104)
timfennis Mar 16, 2026
047231b
Phase 0: update CLAUDE.md and remove stale // vm-ready comments
timfennis Mar 16, 2026
d142122
Phase 1: add vm_native field to Function for bridge-free dispatch
timfennis Mar 16, 2026
d062bba
Migrate arithmetic operators to VM-native dispatch
timfennis Mar 16, 2026
bc4f8bb
Migrate comparison and boolean operators to VM-native dispatch
timfennis Mar 16, 2026
acbdfbf
Add VM-native dispatch for trig/math functions; add trig test coverage
timfennis Mar 16, 2026
aab9e65
export_module: auto-generate vm_native for simple-typed functions
timfennis Mar 16, 2026
96f4a59
Extend export_module macro: auto vm_native for &str, String, i64 types
timfennis Mar 16, 2026
4c088eb
Inverted bridge: VmNative body for stdlib; migrate list/index/value/s…
timfennis Mar 17, 2026
2d1a4e4
Refactor: Rc<Object> replaces Box<Object>; flatten inner Rc from muta…
timfennis Mar 17, 2026
7f0f5ea
Migrate deque/heap/map stdlib to VmNative; add VecDeque/BinaryHeap/Ha…
timfennis Mar 17, 2026
dad2098
Migrate list ++/++=, is_some/is_none, max/min to VmNative
timfennis Mar 17, 2026
af967da
Fix VM slot collision in nested list/map comprehensions
timfennis Mar 17, 2026
845e94a
Change NativeFunction error type from String to VmError
timfennis Mar 17, 2026
89f3033
Fix sort() being a no-op in the tree-walk interpreter
timfennis Mar 17, 2026
b3245f1
Migrate stdlib Sequence params to VM-native SeqValue; fix VM Int/Floa…
timfennis Mar 17, 2026
17dcb46
Optimize VM HOF dispatch with two-tier NativeFunc enum
timfennis Mar 17, 2026
719ed47
Migrate cmp/math/regex stdlib to VM-native types; fix cross-type nume…
timfennis Mar 18, 2026
fea0350
Migrate rand/sequence/value stdlib to VM-native types; add VM RepeatI…
timfennis Mar 18, 2026
2c51c29
Migrate serde stdlib to VM-native types
timfennis Mar 18, 2026
5adeb0c
Fix -= operator display for sets and add test
timfennis Mar 18, 2026
93d269b
Fix op-assign dynamic dispatch for maps, strings, and lists
timfennis Mar 18, 2026
961625e
Migrate string ++= to VM-native return type; add Value::from_string_rc
timfennis Mar 18, 2026
d79973f
Migrate hash_map stdlib to VM-native types; add MapValue type alias
timfennis Mar 18, 2026
856106b
add git info to CLAUDE.md
timfennis Mar 18, 2026
6e2ceb8
Migrate string.rs: replace ndc_interpreter::sequence::StringRepr import
timfennis Mar 18, 2026
734f36b
Add VM output support and wire print/dbg to VM-native path
timfennis Mar 18, 2026
21b52c8
Migrate list stdlib inner module to VM-native types
timfennis Mar 19, 2026
d9b4e9e
Replace ndc_interpreter::num/int with ndc_core in math stdlib
timfennis Mar 19, 2026
f9d07d0
Replace ndc_interpreter::num/compare with ndc_core in rand and sequence
timfennis Mar 19, 2026
fad0300
Remove tree-walk interpreter; make VM the only execution path
timfennis Mar 19, 2026
b2f96f5
Update bench.sh to reflect VM-only execution
timfennis Mar 19, 2026
3c8b567
Add error type prefix to diagnostic messages
timfennis Mar 19, 2026
a122929
Add persistent REPL VM with resume-from-halt support
timfennis Mar 19, 2026
f075047
Print REPL expression results; silence statements and unit
timfennis Mar 19, 2026
da6c83c
Delete GenericFunction interpreter paths from stdlib
timfennis Mar 19, 2026
f68b48a
Move StaticType from ndc_parser to ndc_core
timfennis Mar 19, 2026
e532c67
Fix build after moving StaticType to ndc_core
timfennis Mar 19, 2026
39471e4
Fix missing Value import in ndc_vm/src/value/function.rs
timfennis Mar 19, 2026
dc1c190
Move ndc_vm/src/value.rs to ndc_vm/src/value/mod.rs
timfennis Mar 19, 2026
4533136
Apply formatting from cargo fmt
timfennis Mar 19, 2026
da43313
🔧 Adjust imports and add FunctionRegistry skeleton
timfennis Mar 19, 2026
619c291
Replace Environment-based function registry with FunctionRegistry<Rc<…
timfennis Mar 19, 2026
4c9b357
Fix docs command and improve function return type annotations
timfennis Mar 19, 2026
b47e18c
Add 10k recursion depth test
timfennis Mar 19, 2026
90a278d
Fix pi_approx benchmark: move float conversion inside function
timfennis Mar 19, 2026
e9119a3
Rewrite docs with pretty output and --no-color flag, migrate to yansi
timfennis Mar 19, 2026
0121199
Remove duplicate tree-walk test variants; update CLAUDE.md
timfennis Mar 19, 2026
338f991
Perf: add int/float fast paths in PartialOrd, use with_context in rand
timfennis Mar 19, 2026
0f9250c
Perf: cache StaticType on CompiledFunction, inline vectorized calls, …
timfennis Mar 19, 2026
cc9bb78
Perf: zero-alloc overload matching, skip upvalue materialization for …
timfennis Mar 19, 2026
0c5acfd
Bench: increase fibonacci to fib(26), sieve to 200k
timfennis Mar 20, 2026
7d89a41
Refactor: remove tree-walk evaluator, Environment, and dead macro inf…
timfennis Mar 20, 2026
83a7158
Refactor: remove vm_bridge, interpreter Value type, and all bridge mo…
timfennis Mar 20, 2026
0652644
Cleanup: remove as_any from VmIterator, Vm::stub, stale bridge comments
timfennis Mar 20, 2026
4bf9f77
Refactor: extract ndc_analyser crate, remove WithStdlib trait
timfennis Mar 20, 2026
3472306
Bump version to 0.3.0
timfennis Mar 20, 2026
361b9f8
Use crate version for --version flag
timfennis Mar 20, 2026
fa99f81
Refactor: simplify output, decouple ndc_lsp from ndc_stdlib
timfennis Mar 20, 2026
fc51adb
Add benchmarks, fix quicksort/sieve, update bench.sh for version comp…
timfennis Mar 20, 2026
ccc4917
Fix: upvalue dispatch in overload sets, precise container static types
timfennis Mar 20, 2026
915ebc9
Perf: add Value::is_number, avoid O(n) static_type in hot path
timfennis Mar 20, 2026
89c7366
Perf: borrow opcode by ref in dispatch loop, ~4-8% speedup
timfennis Mar 20, 2026
e8a751a
Fix: map default function called when key is missing
timfennis Mar 20, 2026
0c5a3dd
Perf: skip static_type() for Any params in overload dispatch (~6-8%)
timfennis Mar 20, 2026
96c0737
Docs: document memoization key cost and upvalue pattern
timfennis Mar 20, 2026
c482fd8
Fix: B1/B2/B3 — convert panics and overflows to VmErrors
timfennis Mar 20, 2026
3f76f86
Fix: B5 — delete materialize_upvalues_in_args
timfennis Mar 20, 2026
9c8611d
Perf: P3/P4/P5 — close_upvalues single pass, vectorization deferred a…
timfennis Mar 20, 2026
96a0bec
Fix: O(N) dynamic dispatch for container params in overload resolution
timfennis Mar 20, 2026
f7479e8
Docs: fix manual spelling, grammar, and factual errors
timfennis Mar 20, 2026
3e0a0a1
Perf: P7 — MapIter clones keys only, borrows values on demand
timfennis Mar 20, 2026
88d2058
Feat: lazy combinations and take iterators
timfennis Mar 21, 2026
59ba9cd
Refactor: eval() returns Value, re-export NativeFunction
timfennis Mar 21, 2026
195b01f
Fix: B4 — UnboundedRangeIter overflow at i64::MAX
timfennis Mar 21, 2026
4aa16d9
Fix: B6 — resolve_callee returns VmError instead of panicking
timfennis Mar 21, 2026
456e406
Fix: analyser rejects calling non-function types
timfennis Mar 21, 2026
418ae09
Refactor: R5 — remove RefCell from VmCallable
timfennis Mar 21, 2026
276d25c
Refactor: M1, M2, RD1–RD4 from code review
timfennis Mar 21, 2026
8f81f96
Add documentation to all manually registered stdlib functions
timfennis Mar 21, 2026
65f2af7
Add docs to all undocumented stdlib functions, fix spelling, add docs…
timfennis Mar 21, 2026
341e9bf
Remove obsolete design documents
timfennis Mar 21, 2026
ef27001
Refactor ndc_macros: rename match→types, add NdcType enum, remove ful…
timfennis Mar 21, 2026
2e1e72a
Collect TODOs into TODO.md, remove from source, add CLAUDE.md rule
timfennis Mar 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Before commit
- Run `cargo fmt`
- Run `cargo check` and fix warnings and errors
- Ensure all tests pass (`cargo test`)
- Do not leave `TODO` comments in code — either fix the issue immediately or open a GitHub issue and record it in `TODO.md`

## Common Commands

```bash
# Build
cargo build

# Run all tests
cargo test

# Run a single test by name (substring match on the filename)
cargo test test_001_math_001_addition

# Run benchmarks
cargo bench -p benches

# Start REPL
cargo run --bin ndc

# Run a .ndc script
cargo run --bin ndc -- script.ndc

# Disassemble bytecode
cargo run --bin ndc -- disassemble script.ndc

# Show documentation (optionally filtered by query)
cargo run --bin ndc -- docs [query] [--no-color]

# Profile with perf (requires release-with-debug profile in Cargo.toml)
cargo build --profile release-with-debug
hyperfine --warmup 3 './target/release-with-debug/ndc script.ndc'
perf stat ./target/release-with-debug/ndc script.ndc
perf record -g --call-graph=dwarf -o /tmp/out.perf ./target/release-with-debug/ndc script.ndc
perf report -i /tmp/out.perf --stdio --no-children --percent-limit=1
```

## Manual

User-facing language documentation lives in `manual/src/`. It is an mdBook project. The entry point is `manual/src/SUMMARY.md`.

When making changes that affect language behaviour or runtime semantics, update the relevant manual page.

## Architecture

This is a custom language interpreter ("Andy C++") with a bytecode VM backend:

```
Source → [Lexer] → Tokens → [Parser] → AST → [Analyser] → Annotated AST
[Compiler]
[Bytecode VM] → Value
```

### Git Workflow
- Prefer short commit messages, only use multiple lines in case of unrelated changes
- Pull request titles must start with an emoji

### Crate Layout

| Crate | Role |
|---|---|
| `ndc_lexer` | Tokenisation, `Span` (offset+length) |
| `ndc_parser` | AST (`Expression`, `ExpressionLocation`), parser |
| `ndc_core` | `Number` (BigInt/Rational/Complex), `StaticType`, `FunctionRegistry`, ordering, hashing |
| `ndc_interpreter` | Semantic analyser, `Interpreter` facade (compile + run via VM) |
| `ndc_vm` | Bytecode `Compiler` and stack-based `Vm` |
| `ndc_stdlib` | Built-in functions registered via `FunctionRegistry` |
| `ndc_lsp` | LSP backend (hover, inlay hints) |
| `ndc_bin` | CLI entry point, REPL, syntax highlighting |

### Key Concepts

**Single execution path** — The bytecode VM in `ndc_vm` is the only execution path. `ndc_interpreter` acts as a facade: it runs the semantic analyser, compiles to bytecode via `ndc_vm::Compiler`, and executes via `ndc_vm::Vm`. `vm_bridge.rs` handles value conversion between `ndc_interpreter::Value` and `ndc_vm::Value`.

**Value types** — `ndc_interpreter/src/value.rs` and `ndc_vm/src/value.rs` are separate enums. The VM `Value` is constrained to 16 bytes (`Int(i64)`, `Float(f64)`, `Bool`, `None`, `Object(Box<Object>)`).

**Function overloading** — Functions are matched by name and arity. The semantic analyser produces `Binding::Resolved` (exact compile-time match) or `Binding::Dynamic(Vec<ResolvedVar>)` (runtime dispatch among candidates). Binary operators like `+` are parsed as `Expression::Call`.

**Semantic analyser** — `ndc_interpreter/src/semantic/analyser.rs` infers `StaticType` and resolves function bindings. `StaticType::Any` is the fallback when inference fails.

**`FunctionRegistry`** — Lives in `ndc_core`. Holds all registered built-in functions as `Rc<NativeFunction>`. Replaces the old `Environment`-based function registry. At runtime, natives are passed to the VM as global slots.

**Persistent REPL** — The `Interpreter` keeps `repl_state: Option<(Vm, Compiler)>` so variables declared on one REPL line are visible on subsequent lines (resume-from-halt pattern).

### Test Infrastructure

The `tests` crate auto-generates one test function per `.ndc` file at build time via `tests/build.rs`. For every `.ndc` file under `tests/programs/`, a single Rust test function is generated:
- `test_<path>` — runs via `Interpreter::run_str` (VM)

Test directives are comments inside `.ndc` files:
```ndc
// expect-output: 42 ← assert stdout equals this
// expect-error: divide ← assert error message contains this substring
```

### Compiler Tests

`compiler_tests/` validates the bytecode compiler by asserting exact `OpCode` sequences. Use these when adding new VM instructions.
Loading
Loading