forked from rivet-dev/secure-exec
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathprogress.txt
More file actions
2107 lines (1963 loc) · 176 KB
/
progress.txt
File metadata and controls
2107 lines (1963 loc) · 176 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# Ralph Progress Log
Started: 2026-03-17
PRD: ralph/kernel-hardening (46 stories)
## Codebase Patterns
- Claude binary at ~/.claude/local/claude — not on PATH by default; skip helpers must check this fallback location
- Claude Code --output-format stream-json requires --verbose flag; uses ANTHROPIC_BASE_URL natively (no fetch interceptor)
- Python WORKER_SOURCE is String.raw — use array.join("\n") for multiline Python code; f-strings with escaped quotes break
- @secure-exec/python must be rebuilt (`pnpm run build` in packages/secure-exec-python/) before tests pick up changes
- openShell() controller must close its slave FD after spawn (close-after-fork) — otherwise slave refCount stays >0 and master reads never get EOF
- fd_pipe/fd_dup (host_process) return kernel FDs — MUST register in local FDTable + localToKernelFd map for WASI fd_renumber/fd_close to work
- KernelFDTable.renumber closes kernel FD of overwritten target (POSIX close-on-renumber) — critical for pipe write end closure during stdout restore
- Kernel fdWrite is async for VFS files (read-modify-write at cursor), sync for pipes/PTYs — return type is number | Promise<number>
- proc_spawn must convert local FDs to kernel FDs via localToKernelFd before passing to kernel spawn RPC
- WASI oflags: 0x1=CREAT, 0x2=DIRECTORY, 0x4=EXCL, 0x8=TRUNC — different from POSIX; kernel-worker.ts must map correctly
- Worker local FD table has preopen FD 3 ("/") absent from kernel FD table — use localToKernelFd map for all file I/O RPCs (kernel-worker.ts)
- WASI path_open with OFLAG_DIRECTORY bypasses kernel fdOpen — verifies via vfsStat and creates preopen-style local FD for fd_readdir
- WASI _resolveWasiPath joins base+relative then normalizes (normalizePath removes "." and ".." segments)
- @secure-exec/python package at packages/secure-exec-python/ owns PyodideRuntimeDriver (driver.ts) — deps: @secure-exec/core, pyodide
- @secure-exec/browser package at packages/secure-exec-browser/ owns browser Web Worker runtime (driver.ts, runtime-driver.ts, worker.ts, worker-protocol.ts) — deps: @secure-exec/core, sucrase
- @secure-exec/node package at packages/secure-exec-node/ owns V8-specific execution engine (execution.ts, isolate.ts, bridge-loader.ts, polyfills.ts) — deps: @secure-exec/core, isolated-vm, esbuild, node-stdlib-browser
- @secure-exec/core package at packages/secure-exec-core/ owns shared types, utilities, bridge guest code, generated sources, and build scripts — build it first (turbo ^build handles this)
- When adding exports to shared modules in core, update BOTH core/src/index.ts AND the corresponding re-export file in secure-exec/src/shared/
- Bridge source is in core/src/bridge/, build scripts in core/scripts/, isolate-runtime source in core/isolate-runtime/
- build:bridge, build:polyfills, build:isolate-runtime scripts all live in core's package.json — secure-exec's build is just tsc
- bridge-loader.ts in secure-exec resolves core package root via createRequire(import.meta.url).resolve("@secure-exec/core") to find bridge.js and source
- Source-grep tests use readCoreSource() helper to read files from core's source tree
- Kernel errors use `KernelError(code, message)` from types.ts — always use structured codes, not plain Error with embedded code in message
- Pipe buffers are bounded by MAX_PIPE_BUFFER_BYTES (64KB) — writes to full buffer throw EAGAIN
- PTY buffers are bounded by MAX_PTY_BUFFER_BYTES (64KB) — writes to full buffer throw EAGAIN
- FD tables are bounded by MAX_FDS_PER_PROCESS (256) — excess allocations throw EMFILE
- Constants exported from pipe-manager.ts, pty.ts, and fd-table.ts respectively
- ERRNO_MAP in wasmvm/src/wasi-constants.ts is the single source of truth for POSIX→WASI errno mapping
- Bridge ServerResponseBridge.write/end must treat null as no-op (Node.js convention: res.end(null) ends without writing; Fastify's sendTrailer calls res.end(null, null, null))
- Use `pnpm run check-types` (turbo) for typecheck, not bare `tsc`
- Bridge readFileSync error.code is lost crossing isolate boundary — bridge must detect error patterns in message and re-create proper Node.js errors
- Node driver creates system driver with `permissions: { ...allowAllChildProcess }` only — no fs permissions → deny-by-default → EACCES for all fs reads
- Bridge fs.ts `createFsError` uses Node.js syscall conventions: readFileSync → "open", statSync → "stat", etc.
- WasmVM driver.ts exports createWasmVmRuntime() — worker-based with SAB RPC for sync/async bridge
- Kernel fdSeek is async (Promise<bigint>) — SEEK_END needs VFS readFile for file size; WasmVM driver awaits it in _handleSyscall
- Kernel VFS uses removeFile/removeDir (not unlink/rmdir), and VirtualStat has isDirectory/isSymbolicLink (not type)
- WasiFiletype must be re-exported from wasi-types.ts since polyfill imports it from there
- turbo task is `check-types` — add this script to package.json alongside `typecheck`
- pnpm-workspace.yaml includes `packages/os/*` and `packages/runtime/*` globs
- Adding a VFS method requires updating: interface (vfs.ts), all implementations (TestFileSystem, NodeFileSystem, InMemoryFileSystem), device-layer.ts, permissions.ts
- WASI polyfill file I/O goes through WasiFileIO bridge (wasi-file-io.ts); stdio/pipe handling stays in the polyfill
- WASI polyfill process/FD-stat goes through WasiProcessIO bridge (wasi-process-io.ts); proc_exit exception still thrown by polyfill
- WASI error precedence: check filetype before rights (e.g., ESPIPE before EBADF in fd_seek)
- WasmVM src/ has NO standalone OS-layer code; WASI constants in wasi-constants.ts, interfaces in wasi-types.ts
- WasmVM polyfill constructor requires { fileIO, processIO } in options — callers must provide bridge implementations
- Concrete VFS/FDTable/bridge implementations live in test/helpers/ (test infrastructure only)
- WasmVM package name is `@secure-exec/runtime-wasmvm` (not `@secure-exec/wasmvm`)
- WasmVM tests use vitest (describe/it/expect); vitest.config.ts in package root, test script is `vitest run`
- Kernel ProcessTable.allocatePid() atomically allocates PIDs; register() takes a pre-allocated PID
- Kernel ProcessContext has optional onStdout/onStderr for data emitted during spawn (before DriverProcess callbacks)
- Kernel fdRead is async (returns Promise<Uint8Array>) — reads from VFS at cursor position
- Use createTestKernel({ drivers: [...] }) and MockRuntimeDriver for kernel integration tests
- fixture.json supports optional `packageManager` field ("pnpm" | "npm") — defaults to pnpm; use "npm" for flat node_modules layout testing
- Node RuntimeDriver package is `@secure-exec/runtime-node` at packages/runtime/node/
- createNodeRuntime() wraps NodeExecutionDriver behind kernel RuntimeDriver interface
- KernelCommandExecutor adapter converts kernel.spawn() ManagedProcess to CommandExecutor SpawnedProcess
- npm/npx entry scripts resolved from host Node installation (walks up from process.execPath)
- Kernel spawnManaged forwards onStdout/onStderr from SpawnOptions to InternalProcess callbacks
- NodeExecutionDriver.exec() captures process.exit(N) via regex on error message — returns { code: N }
- Python RuntimeDriver package is `@secure-exec/runtime-python` at packages/runtime/python/
- createPythonRuntime() wraps Pyodide behind kernel RuntimeDriver interface with single shared Worker
- Inside String.raw template literals, use `\n` (not `\\n`) for newlines in embedded JS string literals
- Cannot add runtime packages as devDeps of secure-exec (cyclic dep via runtime-node → secure-exec); use relative imports in tests
- KernelInterface.spawn must forward all ProcessContext callbacks (onStdout/onStderr) to SpawnOptions
- Integration test helpers at packages/secure-exec/tests/kernel/helpers.ts — createIntegrationKernel(), skipUnlessWasmBuilt(), skipUnlessPyodide()
- SpawnOptions has stdinFd/stdoutFd/stderrFd for pipe wiring — reference FDs in caller's table, resolved via callerPid
- KernelInterface.pipe(pid) installs pipe FDs in the process's table (returns actual FD numbers)
- FDTableManager.fork() copies parent's FD table for child — child inherits all open FDs with shared cursors
- fdClose is refcount-aware for pipes: only calls pipeManager.close() when description.refCount drops to 0
- Pipe descriptions start with refCount=0 (not 1); openWith() provides the real reference count
- fdRead for pipes routes through PipeManager.read()
- When stdout/stderr is piped, spawnInternal skips callback buffering — data flows through kernel pipe
- Rust FFI proc_spawn takes argv_ptr+len, envp_ptr+len, stdin/stdout/stderr FDs, cwd_ptr+len, ret_pid (10 params)
- fd_pipe host import packs read+write FDs: low 16 bits = readFd, high 16 bits = writeFd in intResult
- WasmVM stdout writer redirected through fdWrite RPC when stdout is piped
- WasmVM stdin pipe: kernel.pipe(pid) + fdDup2(pid, readFd, 0) + polyfill.setStdinReader()
- Node driver stdin: buffer writeStdin data, closeStdin resolves Promise passed to exec({ stdin })
- Permission-wrapped VFS affects mount() via populateBin() — fs deny tests must skip driver mounting; childProcess deny tests must include allowAllFs
- Bridge process.stdin does NOT emit 'end' for empty stdin ("") — pass undefined for no-stdin case
- E2E fixture tests: use NodeFileSystem({ root: projectDir }) for real npm package resolution
- npm/npx in V8 isolate need host filesystem fallback — createHostFallbackVfs wraps kernel VFS
- WasmVM _handleSyscall fdRead case MUST call data.set(result, 0) to write to SAB — without this, worker reads garbage
- SAB overflow guard: check responseData.length > DATA_BUFFER_BYTES before writing, return errno 76 (EIO)
- Bridge execSync wraps as `bash -c 'cmd'`; spawnSync passes command/args directly — use spawnSync for precise routing tests
- PtyManager description IDs start at 200,000 (pipes at 100,000, regular FDs at 1) — avoid collisions between managers
- PTY is bidirectional: master write→slave read (input), slave write→master read (output); isatty() is true only for slave FDs
- Adding a new FD-managed resource (like PTY) requires updating: fdRead, fdWrite, fdClose, fdSeek, isStdioPiped, cleanupProcessFDs in kernel.ts
- PTY default termios: icanon=true, echo=true, isig=true (POSIX standard); tests wanting raw mode must explicitly set via tcsetattr or ptySetDiscipline
- PTY setDiscipline/setForegroundPgid take description ID internally but KernelInterface methods take (pid, fd) and resolve through FD table
- Termios API: tcgetattr/tcsetattr/tcsetpgrp/tcgetpgrp in KernelInterface; PtyManager stores Termios per PTY with configurable cc (control characters)
- tcgetattr returns a deep copy — callers cannot mutate internal state
- /dev/fd/N in fdOpen → dup(N); VFS-level readDir/stat for /dev/fd are PID-unaware; use devFdReadDir(pid) and devFdStat(pid, fd) on KernelInterface for PID-aware operations
- Device layer has DEVICE_DIRS set (/dev/fd, /dev/pts) for pseudo-directories — stat returns directory mode 0o755, readDir returns empty (PID context required for dynamic content)
- ResourceBudgets (maxOutputBytes, maxBridgeCalls, maxTimers, maxChildProcesses) flow: NodeRuntimeOptions → RuntimeDriverOptions → NodeExecutionDriver constructor
- Bridge-side timer budget: inject `_maxTimers` number as global, bridge checks `_timers.size + _intervals.size >= _maxTimers` synchronously — host-side enforcement doesn't work because `_scheduleTimer.apply()` is async (Promise)
- Bridge `_scheduleTimer.apply(undefined, [delay], { result: { promise: true } })` is async — host throws become unhandled Promise rejections, not catchable try/catch
- Console output (logRef/errorRef) should NOT count against maxBridgeCalls — output has its own maxOutputBytes budget; counting it would exhaust the budget during error reporting
- Per-execution budget state: `budgetState` object reset via `resetBudgetState()` before each context creation (executeInternal and __unsafeCreateContext)
- Kernel maxProcesses: check `processTable.runningCount() >= maxProcesses` in spawnInternal before PID allocation; throws EAGAIN
- Cross-runtime spawn (WasmVM→Node, WasmVM→Python): kernel.spawnInternal registers child PID in parent's driver PID set AND inherits parent's onStdout/onStderr callbacks when stdout is not piped
- kernel.exec() always runs `sh -c command` through brush-shell — all cross-runtime commands go through WasmVM proc_spawn
- ERR_RESOURCE_BUDGET_EXCEEDED is the error code for all bridge resource budget violations
- maxBuffer enforcement: host-side for sync paths (spawnSyncRef tracks bytes, kills, returns maxBufferExceeded flag), bridge-side for async paths (exec/execFile track bytes, kill child); default 1MB for exec/execSync/execFile/execFileSync, unlimited for spawnSync
- Adding a new bridge fs operation requires 10+ file changes: types.ts, all 4 VFS impls, permissions.ts, bridge-contract.ts, global-exposure.ts, setup-fs-facade.ts, runtime-globals.d.ts, execution-driver.ts, bridge/fs.ts, and runtime-node adapters
- Bridge fs.ts `bridgeCall()` helper wraps applySyncPromise calls with ENOENT/EACCES/EEXIST error re-creation — use it for ALL new bridge fs methods
- runtime-node has two VFS adapters (createKernelVfsAdapter, createHostFallbackVfs) that both need new VFS methods forwarded
- diagnostics_channel is Tier 4 (deferred) with a custom no-op stub in require-setup.ts — channels report no subscribers, publish is no-op; needed for Fastify compatibility
- Fastify fixture uses `app.routing(req, res)` for programmatic dispatch — avoids light-my-request's deep ServerResponse dependency; `app.server.emit("request")` won't work because sandbox Server lacks full EventEmitter
- Sandbox Server class needs `setTimeout`, `keepAliveTimeout`, `requestTimeout` properties for framework compatibility — added as no-ops
- Moving a module from Unsupported (Tier 5) to Deferred (Tier 4) requires changes in: module-resolver.ts, require-setup.ts, node-stdlib.md contract, and adding BUILTIN_NAMED_EXPORTS entry
- `declare module` for untyped npm packages must live in a `.d.ts` file (not `.ts`) — TypeScript treats it as augmentation in `.ts` files and fails with TS2665
- Host httpRequest adapter must use `http` or `https` transport based on URL protocol — always using `https` breaks localhost HTTP requests from sandbox
- To test sandbox http.request() client behavior, create an external nodeHttp server in the test code and have the sandbox request to it
- NodeExecutionDriver split into 5 modules in src/node/: isolate-bootstrap.ts (types+utilities), module-resolver.ts, esm-compiler.ts, bridge-setup.ts, execution-lifecycle.ts; facade is execution-driver.ts (<300 lines)
- Source policy tests (isolate-runtime-injection-policy, bridge-registry-policy) read specific source files by path — update them when moving code between files
- esmModuleCache has a sibling esmModuleReverseCache (Map<ivm.Module, string>) for O(1) module→path lookup — both must be updated together and cleared together in execution.ts
---
## 2026-03-17 - US-001
- Already implemented in prior iteration (fdTableManager.remove(pid) in kernel onExit handler)
- Marked passes: true in prd.json
---
## 2026-03-17 - US-002
- What was implemented: EIO guard for SharedArrayBuffer 1MB overflow in WasmVM syscall RPC
- Files changed:
- packages/runtime/wasmvm/src/driver.ts — fixed fdRead to write data to SAB via data.set(), added overflow guard returning EIO (errno 76) for responses >1MB
- packages/runtime/wasmvm/test/driver.test.ts — added SAB overflow protection tests
- prd.json — marked US-001 and US-002 as passes: true
- **Learnings for future iterations:**
- fdRead in _handleSyscall was missing data.set(result, 0) — data was never written to SAB, only length was stored
- vfsReadFile/vfsReaddir/etc already call data.set() which throws RangeError on overflow, caught as EIO by mapErrorToErrno fallback
- General overflow guard after try/catch provides belt-and-suspenders protection for all data-returning syscalls
- WASM-gated tests (describe.skipIf(!hasWasmBinary)) skip in CI when binary isn't built — see US-014
---
## 2026-03-17 - US-003
- What was implemented: Replaced fake negative assertion test with 3 real boundary tests proving host filesystem access is blocked
- Files changed:
- packages/runtime/node/test/driver.test.ts — replaced 'cannot access host filesystem directly' with 3 tests: direct /etc/passwd, symlink traversal, relative path traversal
- packages/secure-exec/src/bridge/fs.ts — fixed readFileSync error conversion to detect ENOENT and EACCES patterns in error messages, added EACCES errno mapping
- prd.json — marked US-003 as passes: true
- **Learnings for future iterations:**
- Error `.code` property is stripped when crossing the V8 isolate boundary via `applySyncPromise` — only `.message` survives
- Bridge must detect error codes in the message string (e.g., "EACCES", "ENOENT") and reconstruct proper Node.js errors with `.code`
- Node driver's deny-by-default fs permissions mean `/etc/passwd` returns EACCES (not ENOENT) — the permission layer blocks before VFS lookup
- Bridge `readFileSync` was inconsistent with `statSync` — statSync already checked for "ENOENT" in messages, readFileSync did not
- `tests/runtime-driver/node/index.test.ts` has flaky ECONNREFUSED failures (pre-existing, not related to this change)
---
## 2026-03-17 - US-004
- What was implemented: Replaced fake child_process routing test with spy driver that records { command, args, callerPid }
- Files changed:
- packages/runtime/node/test/driver.test.ts — replaced 'child_process.spawn routes through kernel to other drivers' with spy-based test that wraps MockRuntimeDriver.spawn to record calls
- **Learnings for future iterations:**
- execSync wraps commands as `bash -c 'cmd'` — use spawnSync to test direct command routing since it passes command/args through unchanged
- Spy pattern: wrap the existing MockRuntimeDriver.spawn with a recording layer rather than creating a separate class — keeps mock behavior and adds observability
- ProcessContext.ppid is the caller's PID (parent), ProcessContext.pid is the spawned child's PID
---
## 2026-03-17 - US-005
- What was implemented: Replaced placeholder "spawning multiple child processes each gets unique kernel PID" test with honest "concurrent child process spawning assigns unique PIDs" test
- Files changed:
- packages/runtime/node/test/driver.test.ts — replaced test: spawns 12 children via spawnSync, spy driver records ctx.pid for each, asserts all 12 PIDs are unique
- **Learnings for future iterations:**
- Reusing the spy driver pattern from US-004 (wrap MockRuntimeDriver.spawn) works well for PID tracking — ctx.pid gives the kernel-assigned child PID
- spawnSync is better than execSync for these tests since it doesn't wrap as bash -c
- 12 processes is comfortably above the 10+ requirement and fast enough (~314ms for all tests)
---
## 2026-03-17 - US-006
- What was implemented: Added echoStdin config to MockRuntimeDriver and two new tests verifying full stdin→process→stdout pipeline
- Files changed:
- packages/kernel/test/helpers.ts — added echoStdin option to MockCommandConfig; writeStdin echoes data via proc.onStdout, closeStdin triggers exit
- packages/kernel/test/kernel-integration.test.ts — added 2 tests: single writeStdin echo and multi-chunk writeStdin concatenation
- prd.json — marked US-006 as passes: true
- **Learnings for future iterations:**
- onStdout is wired to a buffer callback at kernel.ts:237 immediately after driver.spawn() returns, so echoing in writeStdin works synchronously
- echoStdin processes use neverExit-like behavior (no auto-exit) and resolve on closeStdin — this mirrors real process stdin semantics
- spawnManaged replays buffered stdout when options.onStdout is set, ensuring no data loss between spawn and callback attachment
---
## 2026-03-17 - US-007
- What was implemented: Fixed fdSeek to properly handle SEEK_SET, SEEK_CUR, SEEK_END, and pipe rejection (ESPIPE). Added 5 tests.
- Files changed:
- packages/kernel/src/types.ts — changed fdSeek return type to Promise<bigint>
- packages/kernel/src/kernel.ts — implemented proper whence-based seek logic with VFS readFile for SEEK_END, added pipe rejection (ESPIPE), EINVAL for negative positions and invalid whence
- packages/runtime/wasmvm/src/driver.ts — added await to fdSeek call in _handleSyscall
- packages/kernel/test/kernel-integration.test.ts — added 5 tests: SEEK_SET reset+read, SEEK_CUR relative advance, SEEK_END EOF, SEEK_END with negative offset, pipe ESPIPE rejection
- prd.json — marked US-007 as passes: true
- **Learnings for future iterations:**
- fdSeek was a stub that ignored whence and had no pipe rejection — just set cursor = offset directly
- Making fdSeek async was required because SEEK_END needs VFS.readFile (async) to get file size
- The WasmVM _handleSyscall is already async, so adding await to the fdSeek case was straightforward
- KernelInterface.fdSeek callers: kernel.ts implementation, WasmVM driver.ts _handleSyscall, WasmVM kernel-worker.ts (sync RPC — blocked by SAB, unaffected by async driver side)
---
## 2026-03-17 - US-008
- What was implemented: Added permission deny scenario tests covering fs deny-all, fs path-based filtering, childProcess deny-all, childProcess selective, and filterEnv (deny, allow-all, restricted keys)
- Files changed:
- packages/kernel/src/permissions.ts — added checkChildProcess() function for spawn-time permission enforcement
- packages/kernel/src/kernel.ts — stored permissions, added checkChildProcess call in spawnInternal before PID allocation
- packages/kernel/src/index.ts — exported checkChildProcess
- packages/kernel/test/helpers.ts — added Permissions type import, added permissions option to createTestKernel
- packages/kernel/test/kernel-integration.test.ts — added 8 permission deny scenario tests
- prd.json — marked US-008 as passes: true
- **Learnings for future iterations:**
- Permissions wrap the VFS at kernel construction time — mount() calls populateBin() which goes through the permission-wrapped VFS, so fs deny-all tests can't mount drivers
- For fs deny tests, skip driver mounting (test VFS directly). For childProcess deny tests, include fs: () => ({ allow: true }) so mount succeeds
- childProcess permission was defined in types but never enforced — added checkChildProcess in spawnInternal between command resolution and PID allocation
- filterEnv returns {} when no env permission is set (deny-by-default for missing permission checks)
---
## 2026-03-17 - US-009
- What was implemented: Added 4 tests verifying stdio FD override wiring during spawn with stdinFd/stdoutFd/stderrFd
- Files changed:
- packages/kernel/test/kernel-integration.test.ts — added "stdio FD override wiring" describe block with 4 tests: stdinFd→pipe, stdoutFd→pipe, all three overrides, parent table unchanged
- prd.json — marked US-009 as passes: true
- **Learnings for future iterations:**
- KernelInterface.spawn() uses ctx.ppid as callerPid for FD table forking — stdinFd/stdoutFd/stderrFd reference FDs in the caller's (ppid) table
- applyStdioOverride closes inherited FD and installs the caller's description at the target FD number — child gets a new reference (refCount++) to the same FileDescription
- fdStat(pid, fd).filetype can verify FD type (FILETYPE_PIPE vs FILETYPE_CHARACTER_DEVICE) without needing internal table access
- Pipe data flow tests (write→read across pid boundaries) are the strongest verification that wiring is correct — filetype alone doesn't prove the right description was installed
---
## 2026-03-17 - US-010
- What was implemented: Added concurrent PID stress tests spawning 100 processes — verifies PID uniqueness and exit code capture under high concurrency
- Files changed:
- packages/kernel/test/kernel-integration.test.ts — added "concurrent PID stress (100 processes)" describe block with 2 tests: PID uniqueness and exit code correctness
- prd.json — marked US-010 as passes: true
- **Learnings for future iterations:**
- 100 concurrent mock processes complete in ~30ms — MockRuntimeDriver's queueMicrotask-based exit is effectively instant
- Exit codes can be varied per command via configs (i % 256) to verify each process's exit is captured individually, not just "all exited 0"
- ProcessTable.allocatePid() handles 100+ concurrent spawns without PID collision — atomic allocation works correctly
---
## 2026-03-17 - US-011
- What was implemented: Added 3 pipe refcount edge case tests verifying multi-writer EOF semantics via fdDup
- Files changed:
- packages/kernel/test/kernel-integration.test.ts — added "pipe refcount edge cases (multi-writer EOF)" describe block with 3 tests
- prd.json — marked US-011 as passes: true
- **Learnings for future iterations:**
- ki.fdDup(pid, fd) creates a new FD sharing the same FileDescription — refCount increments, both FDs can write to the same pipe
- Pipe EOF (empty Uint8Array from fdRead) only triggers when ALL write-end references are closed (refCount drops to 0)
- Single-process pipe tests (create pipe + dup in same process) are simpler than multi-process tests and sufficient for testing refcount mechanics
- Pipe buffer concatenates writes from any reference to the same write description — order preserved within each call
---
## 2026-03-17 - US-012
- What was implemented: Added 2 tests verifying the full process exit FD cleanup chain: exit → FD table removed → refcounts decremented → pipe EOF / FD table gone
- Files changed:
- packages/kernel/test/kernel-integration.test.ts — added "process exit FD cleanup chain" describe block with 2 tests: pipe write end EOF on exit, 10-FD cleanup on exit
- prd.json — marked US-012 as passes: true
- **Learnings for future iterations:**
- The cleanup chain is: driverProcess.onExit → processTable.markExited → onProcessExit callback → cleanupProcessFDs → fdTableManager.remove(pid) → table.closeAll() → pipe refcounts drop → pipeManager.close() signals EOF
- Testing the chain end-to-end (process exit → pipe reader gets EOF) is more valuable than unit-testing individual links, since the chain is wired via callbacks
- Existing US-001 tests already verify FD table removal; US-012 adds chain verification (exit causes downstream effects like pipe EOF)
- fdOpen throwing ESRCH is the observable proxy for "FDTableManager has no entry" since has()/size aren't exposed through KernelInterface
---
## 2026-03-17 - US-013
- What was implemented: Track zombie cleanup timer IDs and clear them on kernel dispose to prevent post-dispose timer firings
- Files changed:
- packages/kernel/src/process-table.ts — added zombieTimers Map, store timer IDs in markExited, clear all in terminateAll
- packages/kernel/test/kernel-integration.test.ts — added 2 tests: single zombie dispose and 10-zombie batch dispose
- prd.json — marked US-013 as passes: true
- **Learnings for future iterations:**
- ProcessTable.markExited schedules `setTimeout(() => this.reap(pid), 60_000)` — these timers can fire after kernel.dispose() if not tracked
- terminateAll() is the natural place to clear zombie timers since it's called by KernelImpl.dispose()
- The fix is minimal: zombieTimers Map<number, ReturnType<typeof setTimeout>>, set in markExited, clearTimeout + clear() in terminateAll
- Timer callback also deletes from the map to avoid retaining references to already-fired timers
---
## 2026-03-17 - US-014
- What was implemented: CI WASM build pipeline and CI-only guard test ensuring WASM binary availability
- Files changed:
- .github/workflows/ci.yml — added Rust nightly toolchain setup, wasm-opt/binaryen install, build artifact caching, `make wasm` step before Node.js tests
- packages/runtime/wasmvm/test/driver.test.ts — added CI-only guard test that fails if hasWasmBinary is false when CI=true
- CLAUDE.md — added "WASM Binary" section documenting build instructions and CI behavior
- prd.json — marked US-014 as passes: true
- **Learnings for future iterations:**
- CI needs Rust nightly (pinned in wasmvm/rust-toolchain.toml), wasm32-wasip1 target, rust-src component, and wasm-opt (binaryen)
- Install binaryen via apt (fast) rather than `cargo install wasm-opt` (slow compilation)
- Cache key should include Cargo.lock and rust-toolchain.toml to invalidate on dependency or toolchain changes
- Guard test uses `if (process.env.CI)` to only run in CI — locally, WASM-gated tests continue to skip gracefully
- The guard test validates the build step worked; the skipIf tests remain unchanged so local dev without WASM still works
---
## 2026-03-17 - US-015
- What was implemented: Replaced WasmVM error string matching with structured error codes
- Files changed:
- packages/kernel/src/types.ts — added KernelError class with typed `.code: KernelErrorCode` field and KernelErrorCode union type (15 POSIX codes)
- packages/kernel/src/kernel.ts — all `throw new Error("ECODE: ...")` replaced with `throw new KernelError("ECODE", "...")`
- packages/kernel/src/fd-table.ts — same KernelError migration for EBADF throws
- packages/kernel/src/pipe-manager.ts — same KernelError migration for EBADF/EPIPE throws
- packages/kernel/src/process-table.ts — same KernelError migration for ESRCH throws
- packages/kernel/src/device-layer.ts — same KernelError migration for EPERM throws
- packages/kernel/src/permissions.ts — replaced manual `err.code = "EACCES"` with KernelError
- packages/kernel/src/index.ts — exported KernelError and KernelErrorCode
- packages/runtime/wasmvm/src/wasi-constants.ts — added complete WASI errno table (15 codes) and ERRNO_MAP lookup object
- packages/runtime/wasmvm/src/driver.ts — rewrote mapErrorToErrno() to check `.code` first, fallback to ERRNO_MAP string matching; exported for testing
- packages/runtime/wasmvm/test/driver.test.ts — added 13 tests covering structured code mapping, fallback string matching, non-Error values, and exhaustive KernelErrorCode coverage
- **Learnings for future iterations:**
- KernelError extends Error with `.code` field — same pattern as VfsError in wasi-types.ts but for kernel-level errors
- mapErrorToErrno now checks `(err as { code?: string }).code` first — works for KernelError, VfsError, and NodeJS.ErrnoException alike
- ERRNO_MAP in wasi-constants.ts is the single source of truth for POSIX→WASI errno mapping; eliminates magic numbers
- The message format `"CODE: description"` is preserved for backward compatibility with bridge string matching
- permissions.ts previously set `.code` manually via cast — KernelError makes this cleaner with typed constructor
---
## 2026-03-17 - US-016
- What was implemented: Kernel quickstart guide already existed from prior docs commit (10bb4f9); verified all acceptance criteria met and marked passes: true
- Files changed:
- prd.json — marked US-016 as passes: true
- **Learnings for future iterations:**
- docs/kernel/quickstart.mdx was committed as part of the initial docs scaffolding in 10bb4f9
- The guide covers all required topics: install, createKernel+VFS, mount drivers, exec(), spawn() streaming, cross-runtime example, VFS read/write, dispose()
- Follows Mintlify MDX style with Steps, Tabs, Info components and 50-70% code ratio
- docs.json already has the Kernel group with all 4 pages registered
---
## 2026-03-17 - US-017, US-018, US-019, US-020
- What was implemented: All four docs stories were already scaffolded in prior commit (10bb4f9). Verified acceptance criteria met. Moved Kernel group in docs.json to between Features and Reference per US-020 AC.
- Files changed:
- docs/docs.json — moved Kernel group from between System Drivers and Features to between Features and Reference
- prd.json — marked US-017, US-018, US-019, US-020 as passes: true
- **Learnings for future iterations:**
- All kernel docs (quickstart, api-reference, cross-runtime, custom-runtime) were scaffolded in the initial docs commit
- docs.json navigation ordering matters — acceptance criteria specified "between Features and Reference"
- Mintlify MDX uses Steps, Tabs, Info, CardGroup components for rich layout
---
## 2026-03-17 - US-021
- What was implemented: Process group (pgid) and session ID (sid) tracking in kernel process table with setpgid/setsid/getpgid/getsid syscalls and process group kill
- Files changed:
- packages/kernel/src/types.ts — added pgid/sid to ProcessEntry/ProcessInfo, added setpgid/getpgid/setsid/getsid to KernelInterface, added SIGQUIT/SIGTSTP/SIGWINCH signals
- packages/kernel/src/process-table.ts — register() inherits pgid/sid from parent, added setpgid/setsid/getpgid/getsid methods, kill() supports negative pid for process group signals
- packages/kernel/src/kernel.ts — wired setpgid/getpgid/setsid/getsid in createKernelInterface()
- packages/kernel/src/index.ts — exported SIGQUIT/SIGTSTP/SIGWINCH
- packages/kernel/test/kernel-integration.test.ts — added 8 tests covering pgid/sid inheritance, group kill, setsid, setpgid, EPERM/ESRCH error cases
- prd.json — marked US-021 as passes: true
- **Learnings for future iterations:**
- Processes without a parent (ppid=0 or parent not found) default to pgid=pid, sid=pid (session leader)
- Child inherits parent's pgid/sid at register() time — matches POSIX fork() semantics
- kill(-pgid, signal) iterates all entries; only sends to running processes in the group
- setsid fails with EPERM if process is already a group leader (pgid === pid) — POSIX constraint
- setpgid validates target group exists (at least one running process with that pgid)
- MockRuntimeDriver.killSignals config is essential for verifying signal delivery in process group tests
---
## 2026-03-17 - US-022
- What was implemented: PTY device layer with master/slave FD pairs and bidirectional I/O
- Files changed:
- packages/kernel/src/pty.ts — new PtyManager class following PipeManager pattern: createPty(), createPtyFDs(), read/write/close, isPty/isSlave
- packages/kernel/src/types.ts — added openpty() and isatty() to KernelInterface
- packages/kernel/src/kernel.ts — wired PtyManager into fdRead/fdWrite/fdClose/fdSeek, added openpty/isatty implementations, PTY cleanup in cleanupProcessFDs
- packages/kernel/src/index.ts — exported PtyManager
- packages/kernel/test/kernel-integration.test.ts — added 9 PTY tests: master→slave, slave→master, isatty, multiple PTYs, master close hangup, slave close hangup, bidirectional multi-chunk, path format, ESPIPE rejection
- prd.json — marked US-022 as passes: true
- **Learnings for future iterations:**
- PtyManager follows same FileDescription/refCount pattern as PipeManager — description IDs start at 200,000 (pipes at 100,000, regular FDs at 1)
- PTY is bidirectional unlike pipes: master write→slave read (input buffer), slave write→master read (output buffer)
- isatty() returns true only for slave FDs — master FDs are not terminals (matches POSIX: master is the controlling side)
- PTY FDs use FILETYPE_CHARACTER_DEVICE (same as /dev/stdin) since terminals are character devices
- Hangup semantics: closing one end causes reads on the other to return null (mapped to empty Uint8Array by kernel fdRead)
- isStdioPiped() check was extended to include PTY FDs so kernel skips callback buffering for PTY-backed stdio
- cleanupProcessFDs needed updating to handle PTY descriptions alongside pipe descriptions
---
## 2026-03-17 - US-023
- What was implemented: PTY line discipline with canonical mode, raw mode, echo, and signal generation (^C→SIGINT, ^Z→SIGTSTP, ^\→SIGQUIT, ^D→EOF)
- Files changed:
- packages/kernel/src/pty.ts — added LineDisciplineConfig interface, discipline/lineBuffer/foregroundPgid to PtyState, onSignal callback in PtyManager constructor, processInput/deliverInput/echoOutput/signalForByte methods, setDiscipline/setForegroundPgid public methods
- packages/kernel/src/types.ts — added ptySetDiscipline/ptySetForegroundPgid to KernelInterface
- packages/kernel/src/kernel.ts — PtyManager now initialized with signal callback (kill -pgid), wired ptySetDiscipline/ptySetForegroundPgid in createKernelInterface
- packages/kernel/src/index.ts — exported LineDisciplineConfig type
- packages/kernel/test/kernel-integration.test.ts — added 9 PTY line discipline tests: raw mode, canonical backspace, canonical line buffering, echo mode, ^C/^Z/^\/^D, ^C clears line buffer
- prd.json — marked US-023 as passes: true
- **Learnings for future iterations:**
- Default PTY mode is raw (no processing) to preserve backward compat with US-022 tests — canonical/echo/isig are opt-in via ptySetDiscipline
- Signal chars (^C/^Z/^\) are handled by isig flag; ^D (EOF) is handled by canonical mode — these are independent as in POSIX
- PtyManager.onSignal callback wraps processTable.kill(-pgid, signal) with try/catch since pgid may be gone
- Master writes go through processInput; slave writes bypass discipline entirely (they're program output)
- Fast path: when all discipline flags are off, data is passed directly to inputBuffer without byte-by-byte scanning
---
## 2026-03-17 - US-024
- What was implemented: Termios support with tcgetattr/tcsetattr/tcsetpgrp/tcgetpgrp syscalls; Termios interface with configurable control characters; default PTY mode changed to canonical+echo+isig on (POSIX standard)
- Files changed:
- packages/kernel/src/types.ts — added Termios, TermiosCC interfaces and defaultTermios() factory; added tcgetattr/tcsetattr/tcsetpgrp/tcgetpgrp to KernelInterface
- packages/kernel/src/pty.ts — replaced internal LineDisciplineConfig with Termios; signalForByte now uses cc values; added getTermios/setTermios/getForegroundPgid methods; default changed to canonical+echo+isig on
- packages/kernel/src/kernel.ts — wired tcgetattr/tcsetattr/tcsetpgrp/tcgetpgrp through FD table resolution to PtyManager
- packages/kernel/src/index.ts — exported Termios, TermiosCC types and defaultTermios function
- packages/kernel/test/kernel-integration.test.ts — fixed 3 US-022 tests to explicitly set raw mode (previously relied on raw default); added 8 termios tests
- prd.json — marked US-024 as passes: true
- **Learnings for future iterations:**
- Changing PTY default from raw to canonical+echo+isig broke US-022 tests that wrote data without newline — fix is to add explicit raw mode setup
- Termios stored per PtyState, not per FD — both master and slave FDs on the same PTY share the same termios
- tcgetattr must return a deep copy to prevent callers from mutating internal state
- setDiscipline (backward compat API) maps canonical→icanon internally; both APIs modify the same Termios object
- signalForByte uses termios.cc values (vintr/vquit/vsusp) rather than hardcoded constants, allowing custom signal characters
- openShell allocates a controller PID+FD table to hold the PTY master, spawns shell with slave as stdin/stdout/stderr
- Mock readStdinFromKernel config: process reads from stdin FD via KernelInterface and echoes to stdout FD — simulates real process FD I/O through PTY
- Mock survivableSignals config: signals that are recorded but don't cause exit — needed for SIGINT/SIGWINCH in shell tests
---
## 2026-03-17 - US-025
- What was implemented: kernel.openShell() convenience method wiring PTY + process groups + termios for interactive shell use
- Files changed:
- packages/kernel/src/types.ts — added OpenShellOptions, ShellHandle interfaces; added openShell() to Kernel interface
- packages/kernel/src/kernel.ts — implemented openShell() in KernelImpl: allocates controller PID+FD table, creates PTY, spawns shell with slave FDs, sets up process groups and foreground pgid, starts read pump, returns ShellHandle
- packages/kernel/src/index.ts — exported OpenShellOptions, ShellHandle types
- packages/kernel/test/helpers.ts — added readStdinFromKernel (process reads stdin FD via KernelInterface, echoes to stdout FD) and survivableSignals (signals that don't cause exit) to MockCommandConfig
- packages/kernel/test/kernel-integration.test.ts — added 5 openShell tests: echo data, ^C survives, ^D exits, resize SIGWINCH, isatty(0) true
- prd.json — marked US-025 as passes: true
- **Learnings for future iterations:**
- openShell needs a "controller" process (PID + FD table) to hold the PTY master — the controller isn't a real running process, just an FD table owner
- createChildFDTable with callerPid forks the controller's table (inheriting master FD into child), but refcounting handles cleanup correctly
- readStdinFromKernel mock pattern is essential for PTY testing — the mock reads from FD 0 via ki.fdRead() and writes to FD 1 via ki.fdWrite(), simulating how a real runtime would use the PTY slave
- survivableSignals must include SIGINT(2), SIGTSTP(20), and SIGWINCH(28) for shell-like processes that handle these without dying
- The PTY read pump (master → onData) uses ptyManager.read() directly instead of going through KernelInterface, since we're inside KernelImpl
---
## 2026-03-17 - US-026
- What was implemented: kernel.connectTerminal() method and scripts/shell.ts CLI entry point
- Files changed:
- packages/kernel/src/types.ts — added ConnectTerminalOptions interface extending OpenShellOptions with onData override; added connectTerminal() to Kernel interface
- packages/kernel/src/kernel.ts — implemented connectTerminal(): wires openShell() to process.stdin/stdout, sets raw mode (if TTY), forwards resize, restores terminal on exit
- packages/kernel/src/index.ts — exported ConnectTerminalOptions type
- scripts/shell.ts — CLI entry point: creates kernel with InMemoryFileSystem, mounts WasmVM and optionally Node, calls kernel.connectTerminal(), accepts --wasm-path and --no-node flags
- packages/kernel/test/kernel-integration.test.ts — added 4 tests: exit code 0, custom exit code, command/args forwarding, onData override with PTY data flow
- **Learnings for future iterations:**
- connectTerminal guards setRawMode behind isTTY check — in test/CI environments stdin is a pipe, not a TTY
- process.stdin.emit('data', ...) works in tests to simulate user input without a real TTY — useful for testing PTY data flow end-to-end
- stdin.resume() is needed after attaching the data listener to ensure data events fire; stdin.pause() in finally to avoid keeping event loop alive
- The onData override is the key testing seam — tests capture output chunks without needing a real terminal
- scripts/shell.ts uses relative imports (../packages/...) since it's not a workspace package; tsx handles TS execution from the repo root
---
## 2026-03-17 - US-027
- What was implemented: /dev/fd pseudo-directory — fdOpen('/dev/fd/N') → dup(N), devFdReadDir/devFdStat on KernelInterface, device layer /dev/fd and /dev/pts directory support
- Files changed:
- packages/kernel/src/types.ts — added devFdReadDir and devFdStat to KernelInterface
- packages/kernel/src/device-layer.ts — added DEVICE_DIRS set (/dev/fd, /dev/pts), isDeviceDir helper; updated stat/readDir/readDirWithTypes/exists/lstat/createDir/mkdir/removeDir for device pseudo-directories
- packages/kernel/src/kernel.ts — fdOpen intercepts /dev/fd/N → dup(pid, N); implemented devFdReadDir (iterates FD table entries) and devFdStat (stats underlying file, synthetic stat for pipe/PTY)
- packages/kernel/test/kernel-integration.test.ts — added 9 tests: file dup via /dev/fd, pipe read via /dev/fd, devFdReadDir lists 0/1/2, devFdReadDir includes opened FDs, devFdStat on file, devFdStat on pipe, EBADF for bad /dev/fd/N, stat('/dev/fd') directory, readDir('/dev/fd') empty, exists checks
- prd.json — marked US-027 as passes: true
- **Learnings for future iterations:**
- /dev/fd/N open → dup is the primary mechanism; once dup'd, fdRead/fdWrite work naturally through existing pipe/PTY/file routing
- VFS-level readDir/stat for /dev/fd can't have PID context — the VFS is shared across all processes. PID-aware operations need dedicated KernelInterface methods (devFdReadDir, devFdStat)
- Device layer pseudo-directories (/dev/fd, /dev/pts) need separate handling from device nodes (/dev/null, /dev/stdin) — they have isDirectory:true stat and empty readDir
- devFdStat for pipe/PTY FDs returns a synthetic stat (mode 0o666, size 0, ino = description.id) since there's no underlying file to stat
- isDevicePath now also matches /dev/pts/* prefix (needed for PTY paths from US-022)
---
## 2026-03-17 - US-028
- What was implemented: fdPread and fdPwrite (positional I/O) on KernelInterface — reads/writes at a given offset without moving the FD cursor
- Files changed:
- packages/kernel/src/types.ts — added fdPread/fdPwrite to KernelInterface
- packages/kernel/src/kernel.ts — implemented fdPread (VFS read at offset, no cursor change) and fdPwrite (VFS read-modify-write at offset, file extension with zero-fill, no cursor change); ESPIPE for pipes/PTYs
- packages/runtime/wasmvm/src/kernel-worker.ts — wired fdPread/fdPwrite to pass offset through RPC (previously ignored `_offset` param)
- packages/runtime/wasmvm/src/driver.ts — added fdPread/fdPwrite cases in _handleSyscall to route to kernel.fdPread/fdPwrite
- packages/kernel/test/kernel-integration.test.ts — added 7 tests: pread at offset 0, pread at middle offset, pwrite at offset, pwrite file extension, ESPIPE on pipe, pread at EOF, combined pread+pwrite cursor independence
- prd.json — marked US-028 as passes: true
- **Learnings for future iterations:**
- fdPwrite requires read-modify-write pattern: read existing content, create larger buffer if needed, write data at offset, writeFile back to VFS
- fdPwrite extending past file end fills gap with zeros (same as POSIX pwrite behavior)
- WasmVM kernel-worker was ignoring offset for fdPread/fdPwrite — just delegated to regular fdRead/fdWrite RPC. Fixed by adding dedicated fdPread/fdPwrite RPC calls with offset param
- Both fdPread and fdPwrite are async (return Promise) since they need VFS readFile which is async
- Existing tests use `driver.kernelInterface!` pattern to get KernelInterface, not the createTestKernel return value
---
## 2026-03-17 - US-029
- What was implemented: PTY and interactive shell documentation page (docs/kernel/interactive-shell.mdx)
- Files changed:
- docs/kernel/interactive-shell.mdx — new doc covering openShell(), connectTerminal(), PTY internals, termios config, process groups/job control, terminal UI wiring, CLI example
- docs/docs.json — added "kernel/interactive-shell" to Kernel navigation group
- prd.json — marked US-029 as passes: true
- **Learnings for future iterations:**
- Mintlify MDX docs use Tabs, Steps, Info, CardGroup, Card components — follow existing pattern in quickstart.mdx
- docs.json navigation pages are paths without extension (e.g., "kernel/interactive-shell" not "kernel/interactive-shell.mdx")
- Documentation-only stories don't need test runs — only typecheck is required per acceptance criteria
---
## 2026-03-17 - US-030
- What was implemented: Updated kernel API reference with all P4 syscalls
- Files changed:
- docs/kernel/api-reference.mdx — added: kernel.openShell()/connectTerminal() with OpenShellOptions/ShellHandle/ConnectTerminalOptions, ShellHandle type reference, fdPread/fdPwrite positional I/O, process group/session syscalls (setpgid/getpgid/setsid/getsid), PTY operations (openpty/isatty/ptySetDiscipline/ptySetForegroundPgid), termios operations (tcgetattr/tcsetattr/tcsetpgrp/tcgetpgrp), /dev/fd pseudo-directory operations (devFdReadDir/devFdStat), device layer notes (device nodes + pseudo-directories), Termios/TermiosCC type reference, KernelError/KernelErrorCode reference, signal constants table
- prd.json — marked US-030 as passes: true
- **Learnings for future iterations:**
- API reference should mirror KernelInterface in types.ts — iterate all methods and ensure each has a corresponding doc entry
- Mintlify Info component useful for calling out PID context limitations on VFS-level device paths
- fdSeek is async (Promise<bigint>) — the prior doc showed it as sync; fixed to include await
- FDStat has `rights` (not `rightsBase`/`rightsInheriting`) — fixed stale comment in doc
---
## 2026-03-17 - US-031
- What was implemented: Global host resource budgets — maxOutputBytes, maxBridgeCalls, maxTimers, maxChildProcesses on NodeRuntimeOptions, and maxProcesses on KernelOptions
- Files changed:
- packages/kernel/src/types.ts — added EAGAIN to KernelErrorCode, maxProcesses to KernelOptions
- packages/kernel/src/kernel.ts — stored maxProcesses, enforce in spawnInternal before PID allocation
- packages/kernel/src/process-table.ts — added runningCount() method
- packages/secure-exec/src/runtime-driver.ts — added ResourceBudgets interface, resourceBudgets to RuntimeDriverOptions
- packages/secure-exec/src/runtime.ts — added resourceBudgets to NodeRuntimeOptions, pass through to factory
- packages/secure-exec/src/index.ts — exported ResourceBudgets type
- packages/secure-exec/src/node/execution-driver.ts — stored budget limits, added budgetState/resetBudgetState/checkBridgeBudget; enforced maxOutputBytes in logRef/errorRef, maxChildProcesses in spawnStartRef/spawnSyncRef, maxBridgeCalls in all fs/network/timer/child_process References; injected _maxTimers global for bridge-side timer enforcement
- packages/secure-exec/src/bridge/process.ts — added _checkTimerBudget() function, called from setTimeout and setInterval before creating timer entries
- packages/kernel/test/helpers.ts — added maxProcesses option to createTestKernel
- packages/kernel/test/kernel-integration.test.ts — added 4 kernel maxProcesses tests
- packages/secure-exec/tests/test-utils.ts — added resourceBudgets to LegacyNodeRuntimeOptions
- packages/secure-exec/tests/runtime-driver/node/resource-budgets.test.ts — new test file with 8 tests covering all 4 bridge budgets
- prd.json — marked US-031 as passes: true
- **Learnings for future iterations:**
- Bridge _scheduleTimer.apply() is async — host-side throws become unhandled Promise rejections. Timer budget enforcement must be bridge-side (inject _maxTimers global, check _timers.size + _intervals.size synchronously)
- Console logRef/errorRef should NOT count against maxBridgeCalls — it would prevent error reporting after budget exhaustion
- Per-execution budget state must be reset before each context creation (both executeInternal and __unsafeCreateContext paths)
- Timer budget uses concurrent count (_timers.size + _intervals.size) — setTimeout entries are removed when they fire, setInterval entries persist until clearInterval
- Kernel maxProcesses uses processTable.runningCount() which counts only "running" status entries — exited processes don't consume slots
---
## 2026-03-17 - US-032
- What was implemented: maxBuffer enforcement on child-process output buffering for execSync, spawnSync, exec, execFile, and execFileSync
- Files changed:
- packages/secure-exec/src/node/execution-driver.ts — spawnSyncRef now accepts maxBuffer in options, tracks stdout/stderr bytes, kills process and returns maxBufferExceeded flag when exceeded
- packages/secure-exec/src/bridge/child-process.ts — exec() tracks output bytes with default 1MB maxBuffer, kills child on exceed; execSync() passes maxBuffer through RPC, checks maxBufferExceeded in response; spawnSync() passes maxBuffer through RPC, returns error in result; execFile() same pattern as exec(); execFileSync() passes maxBuffer to spawnSync, throws on exceed
- packages/secure-exec/tests/runtime-driver/node/maxbuffer.test.ts — new test file with 10 tests: execSync within/exceeding/small/default maxBuffer, spawnSync stdout/stderr independent enforcement and no-enforcement-when-unset, execFileSync within/exceeding limits
- prd.json — marked US-032 as passes: true
- **Learnings for future iterations:**
- Host-side spawnSyncRef is where maxBuffer enforcement must happen for sync paths — the host buffers all output before returning to bridge
- maxBuffer passed through JSON options in the RPC call ({cwd, env, maxBuffer}); host returns {maxBufferExceeded: true} flag
- Default maxBuffer 1MB applies to execSync/execFileSync (Node.js convention); spawnSync has no default (unlimited unless explicitly set)
- Async exec/execFile maxBuffer enforcement happens bridge-side — data arrives via _childProcessDispatch, bridge tracks bytes and kills child via host kill reference
- Async exec tests timeout in mock executor setup because streaming dispatch (host→isolate applySync) requires real kernel integration; sync paths are fully testable with mock executors
- ERR_CHILD_PROCESS_STDIO_MAXBUFFER is the standard Node.js error code for this condition
---
## 2026-03-17 - US-033
- What was implemented: Added fs.cp/cpSync, fs.mkdtemp/mkdtempSync, fs.opendir/opendirSync to bridge
- Files changed:
- packages/secure-exec/src/bridge/fs.ts — added cpSync (recursive directory copy with force/errorOnExist), mkdtempSync (random suffix temp dir), opendirSync (Dir class with readSync/read/async iteration), plus callback and promise forms
- packages/secure-exec/tests/runtime-driver/node/index.test.ts — added 12 tests covering all three APIs in sync, callback, and promise forms
- prd.json — marked US-033 passes: true
- **Learnings for future iterations:**
- All three APIs can be implemented purely on the isolate side using existing bridge references (readFile, writeFile, readDir, mkdir, stat) — no new host bridge globals needed
- Dir class needs Symbol.asyncIterator for `for await (const entry of dir)` — standard async generator pattern works
- cpSync for directories requires explicit `{ recursive: true }` to match Node.js semantics — without it, throws ERR_FS_EISDIR
- mkdtempSync uses Math.random().toString(36).slice(2, 8) for suffix — good enough for VFS uniqueness, no crypto needed
---
## 2026-03-17 - US-034
- What was implemented: Added glob, statfs, readv, fdatasync, fsync APIs to the bridge fs module
- Files changed:
- packages/secure-exec/src/bridge/fs.ts — added fsyncSync/fdatasyncSync (no-op, validate FD), readvSync (scatter-read using readSync), statfsSync (synthetic TMPFS stats), globSync (VFS pattern matching with glob-to-regex), plus async callback and promise forms for all
- packages/secure-exec/tests/runtime-driver/node/index.test.ts — added 20 tests covering sync, callback, and promise forms for all 5 APIs
- prd.json — marked US-034 passes: true
- **Learnings for future iterations:**
- All five APIs implemented purely on isolate side — no new host bridge globals needed (glob walks VFS via readdirSync/statSync, statfs returns synthetic values, readv uses readSync, fsync/fdatasync are no-ops)
- StatsFs type in Node.js @types expects number fields (not bigint) — use `as unknown as nodeFs.StatsFs` cast for synthetic return
- Glob implementation uses late-bound references (`_globReadDir`, `_globStat`) assigned after `fs` object definition to avoid circular reference issues
- readvSync follows writev pattern: iterate buffers, call readSync per buffer, advance position, stop on partial read (EOF)
---
## 2026-03-17 - US-035
- What was implemented: Wired deferred fs APIs (chmod, chown, link, symlink, readlink, truncate, utimes) through the bridge to VFS
- Files changed:
- packages/secure-exec/src/types.ts — Added new VFS methods + FsAccessRequest ops
- packages/secure-exec/src/shared/in-memory-fs.ts — Added symlink/readlink/lstat/link/chmod/chown/utimes/truncate implementations with symlink resolution
- packages/secure-exec/src/node/driver.ts (NodeFileSystem) — Delegated to node:fs/promises
- packages/secure-exec/src/node/module-access.ts (ModuleAccessFileSystem) — Delegated to base VFS with read-only projection guards
- packages/secure-exec/src/browser/driver.ts (OpfsFileSystem) — Added stubs (ENOSYS for unsupported, no-op for metadata)
- packages/secure-exec/src/shared/permissions.ts — Added permission wrappers, fsOpToSyscall cases, stubs for new ops
- packages/secure-exec/src/shared/bridge-contract.ts — Added 8 new host bridge keys, types, facade interface members
- packages/secure-exec/src/shared/global-exposure.ts — Added inventory entries
- packages/secure-exec/isolate-runtime/src/inject/setup-fs-facade.ts — Added refs to facade
- packages/secure-exec/isolate-runtime/src/common/runtime-globals.d.ts — Added global type declarations
- packages/secure-exec/src/node/execution-driver.ts — Wired 8 new ivm References to VFS methods
- packages/secure-exec/src/bridge/fs.ts — Replaced "not supported" throws with real sync/async/callback/promises implementations; updated watch/watchFile message to include "use polling"
- packages/runtime/node/src/driver.ts — Added new methods to kernel VFS adapters
- .agent/contracts/node-stdlib.md — Updated deferred API classification
- tests/runtime-driver/node/index.test.ts — Added 12 tests covering sync/async/callback/promises/permissions
- **Learnings for future iterations:**
- Adding a new bridge fs operation requires changes in 10+ files: types.ts (VFS+FsAccessRequest), all 4 VFS implementations, permissions.ts, bridge-contract.ts, global-exposure.ts, setup-fs-facade.ts, runtime-globals.d.ts, execution-driver.ts, bridge/fs.ts, and runtime-node adapter
- Bridge errors that cross the isolate boundary lose their .code property — new bridge methods MUST use bridgeCall() wrapper for ENOENT/EACCES/EEXIST error re-creation
- InMemoryFileSystem needs explicit symlink tracking (Map<string, string>) and a resolveSymlink() helper with max-depth loop detection
- VirtualStat.isSymbolicLink must be optional (?) since older code doesn't set it
- runtime-node has two VFS adapters (createKernelVfsAdapter, createHostFallbackVfs) that both need updating for new VFS methods
- Project-matrix sandbox has no NetworkAdapter — http.createServer().listen() throws; pass useDefaultNetwork to createNodeDriver to enable HTTP server fixtures
- Express/Fastify fixtures can dispatch mock requests via `app(req, res, cb)` with EventEmitter-based req/res; emit req 'end' synchronously (not nextTick) to avoid sandbox async errors
---
## 2026-03-17 - US-036
- What was implemented: Express project-matrix fixture that loads Express, creates an app with 3 routes, dispatches mock requests through the app handler, and verifies JSON responses
- Files changed:
- packages/secure-exec/tests/projects/express-pass/package.json — new fixture with express@4.21.2
- packages/secure-exec/tests/projects/express-pass/fixture.json — pass expectation
- packages/secure-exec/tests/projects/express-pass/src/index.js — Express app with programmatic dispatch
- prd.json — marked US-036 as passes: true
- **Learnings for future iterations:**
- Express can be tested programmatically without HTTP server by passing mock req/res objects through `app(req, res, callback)` — Express's `setPrototypeOf` adds its methods (json, send, etc.) to the mock
- Mock req/res must have own properties for `end`, `setHeader`, `getHeader`, `removeHeader`, `writeHead`, `write` since Express's prototype chain expects them
- Mock res needs `socket` and `connection` objects with `writable: true`, `on()`, `end()`, `destroy()` to prevent crashes from `on-finished` and `finalhandler` packages
- Do NOT emit req 'end' event via `process.nextTick` — causes async error in sandbox's EventEmitter; emit synchronously after `app()` call instead
- Sandbox project-matrix has NO NetworkAdapter, so `http.createServer().listen()` throws; `useDefaultNetwork: true` on createNodeDriver would enable it
- Kernel e2e project-matrix tests skip locally when WASM binary is not built (skipUnlessWasmBuilt)
---
## 2026-03-17 - US-037
- What was implemented: Fastify project-matrix fixture with programmatic request dispatch
- Files changed:
- packages/secure-exec/tests/projects/fastify-pass/ — new fixture (package.json, fixture.json, src/index.js, pnpm-lock.yaml)
- packages/secure-exec/src/module-resolver.ts — moved diagnostics_channel from Unsupported to Deferred tier, added BUILTIN_NAMED_EXPORTS
- packages/secure-exec/isolate-runtime/src/inject/require-setup.ts — moved diagnostics_channel to deferred, added custom no-op stub with channel/tracingChannel/hasSubscribers
- packages/secure-exec/src/bridge/network.ts — added Server.setTimeout/keepAliveTimeout/requestTimeout/headersTimeout/timeout properties, added ServerResponseCallable function constructor for .call() compatibility
- .agent/contracts/node-stdlib.md — updated module tier assignment (diagnostics_channel → Tier 4)
- prd.json — marked US-037 passes: true
- **Learnings for future iterations:**
- Fastify requires diagnostics_channel (Node.js built-in) — was Tier 5 (throw on require), needed promotion to Tier 4 with custom stub
- light-my-request (Fastify's inject lib) calls http.ServerResponse.call(this, req) — ES6 classes can't be called without new; use app.routing(req, res) instead
- Sandbox project-matrix has no NetworkAdapter — http.createServer().listen() throws ENOSYS; use programmatic dispatch for fixture testing
- Fastify's app.routing(req, res) is available after app.ready() and routes requests through the full Fastify pipeline without needing a server
- Mock req for Fastify needs: setEncoding, read, destroy, pipe, isPaused, _readableState (stream interface) plus httpVersion/httpVersionMajor/httpVersionMinor
- Mock res for Fastify needs: assignSocket, detachSocket, writeContinue, hasHeader, getHeaderNames, getHeaders, cork, uncork, setTimeout, addTrailers, flushHeaders
---
## 2026-03-17 - US-038
- What was implemented
- Created pnpm-layout-pass fixture: require('left-pad') through pnpm's symlinked .pnpm/ structure
- Created bun-layout-pass fixture: require('left-pad') through npm/bun flat node_modules layout
- Added `packageManager` field support to fixture.json schema ("pnpm" | "npm")
- Updated project-matrix.test.ts: metadata validation, install command selection, cache key with PM version
- Updated e2e-project-matrix.test.ts: same packageManager support for kernel tests
- bun-layout fixture uses `"packageManager": "npm"` to create flat layout (same structure as bun)
- Files changed
- packages/secure-exec/tests/projects/pnpm-layout-pass/ — new fixture (package.json, fixture.json, src/index.js)
- packages/secure-exec/tests/projects/bun-layout-pass/ — new fixture (package.json, fixture.json, src/index.js)
- packages/secure-exec/tests/project-matrix.test.ts — PackageManager type, validation, install command routing, cache key
- packages/secure-exec/tests/kernel/e2e-project-matrix.test.ts — same packageManager support
- prd.json — marked US-038 passes: true
- **Learnings for future iterations:**
- fixture.json schema is strict — new keys must be added to allowedTopLevelKeys set in parseFixtureMetadata
- Both project-matrix.test.ts and e2e-project-matrix.test.ts have parallel prep logic that must be kept in sync
- npm creates flat node_modules (same structure as bun) — good proxy for testing bun layout without requiring bun installed
- Cache key must include the package manager name and version to avoid cross-PM cache collisions
---
## 2026-03-17 - US-039
- Removed @ts-nocheck from polyfills.ts and os.ts
- Files changed:
- packages/secure-exec/src/bridge/polyfills.ts — removed @ts-nocheck, module declaration moved to .d.ts
- packages/secure-exec/src/bridge/text-encoding-utf-8.d.ts — NEW: type declaration for untyped text-encoding-utf-8 package
- packages/secure-exec/src/bridge/os.ts — removed @ts-nocheck, used type assertions for partial polyfill types
- **Learnings for future iterations:**
- `declare module` for untyped packages cannot go in `.ts` files (treated as augmentation, fails TS2665); must use separate `.d.ts` file
- os.ts is a polyfill providing a Linux subset — Node.js types include Windows WSA* errno constants and RTLD_DEEPBIND that don't apply; cast sub-objects rather than adding unused constants
- userInfo needs `nodeOs.UserInfoOptions` parameter type (not raw `{ encoding: BufferEncoding }`) to match overloaded signatures
---
## 2026-03-17 - US-040
- Removed @ts-nocheck from packages/secure-exec/src/bridge/child-process.ts
- Only 2 type errors: `(code: number)` callback params in `.on("close", ...)` didn't match `EventListener = (...args: unknown[]) => void`
- Fixed by changing to `(...args: unknown[])` with `const code = args[0] as number` inside
- Files changed: packages/secure-exec/src/bridge/child-process.ts (2 callbacks on lines 374 and 696)
- **Learnings for future iterations:**
- child-process.ts was nearly type-safe already — only event listener callbacks needed parameter type fixes
- The `EventListener = (...args: unknown[]) => void` type used by the ChildProcess polyfill means all `.on()` callbacks must accept `unknown` params
---
## 2026-03-17 - US-041
- Removed @ts-nocheck from packages/secure-exec/src/bridge/process.ts and packages/secure-exec/src/bridge/network.ts
- process.ts had ~24 type errors: circular self-references in stream objects (_stdout/_stderr/_stdin returning `typeof _stdout`), `Partial<typeof nodeProcess>` causing EventEmitter return type mismatches, missing `_maxTimers` declaration, `./polyfills` import missing `.js` extension, `whatwg-url` missing type declarations
- network.ts had ~16 type errors: `satisfies Partial<typeof nodeDns>` requiring `__promisify__` on all dns functions, `Partial<typeof nodeHttp>` return type requiring full overload sets, `this` not assignable in clone() methods, implicit `any` params
- Files changed:
- packages/secure-exec/src/bridge/process.ts — removed @ts-nocheck, added StdioWriteStream/StdinStream interfaces, changed process type to `Record<string, unknown> & {...}`, cast export to `typeof nodeProcess`, fixed import path, added `_maxTimers` declaration, made StdinListener param optional
- packages/secure-exec/src/bridge/network.ts — removed @ts-nocheck, removed `satisfies Partial<typeof nodeDns>`, changed `createHttpModule` return to `Record<string, unknown>`, fixed clone() casts, added explicit types on callback params
- packages/secure-exec/src/bridge/whatwg-url.d.ts — new module declaration for whatwg-url
- **Learnings for future iterations:**
- Bridge polyfill objects that self-reference (`return this`) need explicit interface types to break circular inference — TypeScript can't infer `typeof x` while `x` is being defined
- `Partial<typeof nodeModule>` and `satisfies Partial<typeof nodeModule>` are too strict for bridge polyfills — they require matching all Node.js overloads and subproperties like `__promisify__`. Use `Record<string, unknown>` internally and cast at export boundaries
- The `whatwg-url` package (v15) has no built-in types — needs a local `.d.ts` module declaration
- For `_addListener`/`_removeListener` helper functions that return `process` (forward reference), use `unknown` return type to break the cycle
---
## 2026-03-17 - US-042
- What was implemented: Replaced JSON-based v8.serialize/deserialize with structured clone serializer supporting Map, Set, RegExp, Date, BigInt, circular refs, undefined, NaN, ±Infinity, ArrayBuffer, and typed arrays
- Files changed:
- packages/secure-exec/isolate-runtime/src/inject/bridge-initial-globals.ts — added __scEncode/__scDecode functions implementing tagged JSON structured clone format; serialize wraps in {$v8sc:1,d:...} envelope, deserialize detects envelope and falls back to legacy JSON
- packages/secure-exec/src/generated/isolate-runtime.ts — rebuilt by build-isolate-runtime.mjs
- packages/secure-exec/tests/runtime-driver/node/index.test.ts — added 7 roundtrip tests: Map, Set, RegExp, Date, circular refs, special primitives (undefined/NaN/Infinity/-Infinity/BigInt), ArrayBuffer and typed arrays
- prd.json — marked US-042 as passes: true
- **Learnings for future iterations:**
- isolate-runtime code is compiled by esbuild into IIFE and stored in src/generated/isolate-runtime.ts — run `node scripts/build-isolate-runtime.mjs` from packages/secure-exec after modifying any file in isolate-runtime/src/inject/
- To avoid ambiguity in the tagged JSON format, all non-primitive values (including plain objects and arrays) must be tagged — prevents confusion between a tagged type `{t:"map",...}` and a plain object that happens to have a `t` key
- Legacy JSON format fallback in deserialize ensures backwards compatibility if older serialized buffers exist
- v8.serialize tests must roundtrip inside the isolate (serialize + deserialize in same run) since the Buffer format is sandbox-specific, not compatible with real V8 wire format
---
## 2026-03-17 - US-043
- What was implemented: HTTP Agent pooling (maxSockets), upgrade event (101), trailer headers, socket event on ClientRequest, protocol-aware httpRequest host adapter
- Files changed:
- packages/secure-exec/src/bridge/network.ts — replaced no-op Agent with full pooling implementation (per-host maxSockets queue with acquire/release), added FakeSocket class for socket events, updated ClientRequest to use agent pooling + emit 'socket' event + fire 'upgrade' on 101 + populate trailers, updated IncomingMessage to populate trailers from response
- packages/secure-exec/src/node/driver.ts — fixed httpRequest to use http/https based on URL protocol (was always https), added 'upgrade' event handler for 101 responses, added trailer forwarding from res.trailers
- packages/secure-exec/src/types.ts — added optional `trailers` field to NetworkAdapter.httpRequest return type
- packages/secure-exec/tests/runtime-driver/node/index.test.ts — added Agent maxSockets=1 serialization test (external HTTP server with concurrency tracking), added upgrade event test (external HTTP server with 'upgrade' handler)
- prd.json — marked US-043 as passes: true
- **Learnings for future iterations:**
- Host httpRequest adapter was always using `https.request` regardless of URL protocol — sandbox http.request to localhost HTTP servers requires `http.request` on the host side
- Agent pooling is purely bridge-side: ClientRequest acquires/releases slots from the Agent, no host-side changes needed for the pooling logic
- For testing sandbox's http.request() behavior, create an external HTTP server in the test code (outside sandbox) — the sandbox's request goes through bridge → host adapter → real request to external server
- Node.js HTTP parser fires 'upgrade' event (not response callback) for 101 status — host adapter must handle this explicitly
- FakeSocket class satisfies `request.on('socket', cb)` API — libraries like got/axios use this to detect socket assignment
---
## 2026-03-17 - US-044
- What was implemented: Codemod example project demonstrating safe code transformations in secure-exec sandbox
- Files changed:
- examples/codemod/package.json (new) — @secure-exec/example-codemod package with tsx dev script
- examples/codemod/src/index.ts (new) — reads source → writes to VFS → executes codemod in sandbox → reads transformed result → prints diff
- **Learnings for future iterations:**
- esbuild (used by tsx) cannot parse template literal backticks or `${` inside String.raw templates — use `String.fromCharCode(96)` and split `'$' + '{'` to work around
- Examples don't need tsconfig.json — they inherit from the workspace and use tsx for runtime TS execution
- Example naming convention: `@secure-exec/example-<name>` with `"private": true` and `"type": "module"`
- InMemoryFileSystem methods (readTextFile, writeFile) are async (return Promises) — must await them on the host side
---
## 2026-03-17 - US-045
- What was implemented: Split 1903-line NodeExecutionDriver monolith into 5 focused modules + 237-line facade
- Files changed:
- packages/secure-exec/src/node/isolate-bootstrap.ts (new, 206 lines) — types (DriverDeps, BudgetState), constants, PayloadLimitError, payload/budget utility functions, host builtin helpers
- packages/secure-exec/src/node/module-resolver.ts (new, 191 lines) — getNearestPackageType, getModuleFormat, shouldRunAsESM, resolveESMPath, resolveReferrerDirectory
- packages/secure-exec/src/node/esm-compiler.ts (new, 367 lines) — compileESMModule, createESMResolver, runESM, dynamic import resolution, setupDynamicImport
- packages/secure-exec/src/node/bridge-setup.ts (new, 779 lines) — setupRequire (fs/child_process/network ivm.References), setupConsole, setupESMGlobals, timing mitigation
- packages/secure-exec/src/node/execution-lifecycle.ts (new, 136 lines) — applyExecutionOverrides, CommonJS globals, global exposure policy, awaitScriptResult, stdin/env/cwd overrides
- packages/secure-exec/src/node/execution-driver.ts (rewritten, 237 lines) — facade class owning DriverDeps state, delegating to extracted modules
- packages/secure-exec/tests/isolate-runtime-injection-policy.test.ts — updated to read all node/ source files instead of just execution-driver.ts
- packages/secure-exec/tests/bridge-registry-policy.test.ts — updated to read bridge-setup.ts and esm-compiler.ts for HOST_BRIDGE_GLOBAL_KEYS checks
- prd.json — marked US-045 as passes: true
- **Learnings for future iterations:**
- Source policy tests (isolate-runtime-injection-policy, bridge-registry-policy) assert that specific strings appear in execution-driver.ts — when splitting files, update these tests to read all relevant source files
- DriverDeps interface centralizes mutable state shared across extracted modules — modules use Pick<DriverDeps, ...> for narrow dependency declarations
- Bridge-setup is the largest extracted module (779 lines) because all ivm.Reference creation for fs/child_process/network is a single cohesive unit
- The execution.ts ExecutionRuntime interface already existed as a delegation pattern — the facade wires extracted functions into this interface via executeInternal
---
## 2026-03-17 - US-046
- Replaced O(n) ESM module reverse lookup with O(1) Map-based bidirectional cache
- Added `esmModuleReverseCache: Map<ivm.Module, string>` to DriverDeps, CompilerDeps, and ExecutionRuntime
- Updated esm-compiler.ts to populate reverse cache on every esmModuleCache.set() and use Map.get() instead of for-loop
- Updated execution.ts to clear reverse cache alongside forward cache
- Files changed:
- packages/secure-exec/src/node/isolate-bootstrap.ts — added esmModuleReverseCache to DriverDeps
- packages/secure-exec/src/node/esm-compiler.ts — O(1) reverse lookup, populate reverse cache on set
- packages/secure-exec/src/node/execution-driver.ts — initialize and pass reverse cache
- packages/secure-exec/src/execution.ts — add to ExecutionRuntime type, clear on reset
- packages/secure-exec/tests/runtime-driver/node/index.test.ts — added deep chain (50-module) and wide (1000-module) ESM tests
- prd.json — marked US-046 as passes: true
- **Learnings for future iterations:**
- esmModuleCache flows through 4 interfaces: DriverDeps, CompilerDeps (Pick), ExecutionRuntime, and the execution-driver executeInternal passthrough — adding a sibling cache requires updating all 4
- ivm.Module instances work as Map keys (reference identity)
- The reverse cache must be cleared in execution.ts executeWithRuntime alongside the forward cache
---
## 2026-03-17 - US-047
- Implemented resolver memoization with positive/negative caches in package-bundler.ts
- Added ResolutionCache interface with 4 cache maps: resolveResults (top-level), packageJsonResults, existsResults, statResults
- Threaded cache through all resolution functions: resolveModule, resolvePath, readPackageJson, resolveNodeModules, etc.
- Added cachedSafeExists() and cachedStat() wrappers that check cache before VFS probes
- Added resolutionCache to DriverDeps, initialized in NodeExecutionDriver constructor
- Cache cleared per-execution in executeWithRuntime() alongside other caches
- Wired cache through bridge-setup.ts (require resolution) and module-resolver.ts (ESM resolution)
- Files changed:
- packages/secure-exec/src/package-bundler.ts — ResolutionCache type, createResolutionCache(), cached wrappers, threading
- packages/secure-exec/src/node/isolate-bootstrap.ts — added resolutionCache to DriverDeps
- packages/secure-exec/src/node/execution-driver.ts — initialize cache in constructor, pass through to ExecutionRuntime
- packages/secure-exec/src/execution.ts — add ResolutionCache to ExecutionRuntime type, clear per-execution
- packages/secure-exec/src/node/bridge-setup.ts — pass cache to resolveModule(), added to BridgeDeps
- packages/secure-exec/src/node/module-resolver.ts — pass cache to resolveModule() in resolveESMPath()
- packages/secure-exec/src/node/esm-compiler.ts — added resolutionCache to CompilerDeps
- packages/secure-exec/tests/runtime-driver/node/resolver-memoization.test.ts — 9 tests
- prd.json — marked US-047 as passes: true
- **Learnings for future iterations:**
- Adding a new cache to the resolution pipeline requires updating: DriverDeps, BridgeDeps (Pick), CompilerDeps (Pick), ResolverDeps (Pick), ExecutionRuntime, and execution-driver passthrough
- The cache parameter is optional on resolveModule() to avoid breaking browser/worker.ts which doesn't share DriverDeps
- Mid-level caches (exists, stat, packageJson) benefit multiple modules in the same tree; top-level cache (resolveResults) gives O(1) for repeated identical lookups
- Using `?.` optional chaining on cache writes (e.g., `cache?.existsResults.set()`) keeps the uncached path clean
---
## 2026-03-17 - US-048
- What was implemented
- Added `zombieTimerCount` getter to ProcessTable for test observability
- Exposed `zombieTimerCount` on the Kernel interface and KernelImpl
- Rewrote zombie timer cleanup tests with vi.useFakeTimers() to actually verify timer state:
- process exit → zombieTimerCount > 0
- kernel.dispose() → zombieTimerCount === 0
- advance 60s after dispose → no callbacks fire (process entry still exists)
- multiple zombie processes → all N timers cleared on dispose
- Files changed
- packages/kernel/src/process-table.ts — added zombieTimerCount getter
- packages/kernel/src/types.ts — added zombieTimerCount to Kernel interface
- packages/kernel/src/kernel.ts — added zombieTimerCount getter forwarding to processTable
- packages/kernel/test/kernel-integration.test.ts — rewrote 2 vacuous tests into 4 assertive tests with fake timers
- prd.json — marked US-048 as passes: true
- **Learnings for future iterations:**
- vi.useFakeTimers() must be wrapped in try/finally with vi.useRealTimers() to avoid polluting other tests
- Tests that only assert "no throw" are vacuous for cleanup verification — always assert observable state changes
- ProcessTable.zombieTimers is private Map; exposing count via getter avoids leaking the timer IDs
---
## 2026-03-17 - US-049
- Added `packageManager: "pnpm"` to fixture.json
- Generated pnpm-lock.yaml via `pnpm install --ignore-workspace --prefer-offline`
- pnpm creates real symlink structure: node_modules/left-pad → .pnpm/left-pad@0.0.3/node_modules/left-pad
- All 14 project matrix tests pass including pnpm-layout-pass
- Files changed:
- packages/secure-exec/tests/projects/pnpm-layout-pass/fixture.json
- packages/secure-exec/tests/projects/pnpm-layout-pass/pnpm-lock.yaml (new)
- **Learnings for future iterations:**
- node_modules are never committed — only lock files; the test framework copies source (excluding node_modules) to a staging dir and runs install
- pnpm install in fixture dirs needs `--ignore-workspace` flag to avoid being treated as workspace package
- validPackageManagers in project-matrix.test.ts is Set(["pnpm", "npm", "bun"])
---
## 2026-03-17 - US-050
- Fixed bun fixture: changed fixture.json packageManager from "npm" to "bun"
- Generated bun.lock via `bun install` (bun 1.3.10 uses text-based bun.lock, not binary bun.lockb)
- Added "bun" as valid packageManager in both project-matrix.test.ts and e2e-project-matrix.test.ts
- Added getBunVersion() helper for cache key calculation in both test files
- Added bun install command branch in prepareFixtureProject in both test files
- All 14 project matrix tests pass including bun-layout-pass
- Files changed:
- packages/secure-exec/tests/projects/bun-layout-pass/fixture.json
- packages/secure-exec/tests/projects/bun-layout-pass/bun.lock (new)
- packages/secure-exec/tests/project-matrix.test.ts
- packages/secure-exec/tests/kernel/e2e-project-matrix.test.ts
- **Learnings for future iterations:**
- Bun 1.3.10 creates text-based bun.lock (not binary bun.lockb from v0)
- Bun install doesn't need --prefer-offline or --ignore-workspace flags
- Both project-matrix.test.ts and kernel/e2e-project-matrix.test.ts must be updated in sync for new package managers
---
## 2026-03-17 - US-051
- Fixed Express and Fastify fixtures to use real HTTP servers
- Root cause: bridge ServerResponseBridge.write/end did not handle null chunks — Fastify's sendTrailer calls res.end(null, null, null) which pushed null into _chunks, causing Buffer.concat to fail with "Cannot read properties of null (reading 'length')"
- Fix: updated write() and end() in bridge/network.ts to treat null as no-op (matching Node.js behavior)
- Updated Fastify fixture to use app.listen() instead of manual http.createServer + app.routing
- All 14 project matrix tests pass, all 149 node runtime driver tests pass, typecheck passes
- Files changed:
- packages/secure-exec/src/bridge/network.ts (null-safe write/end)
- packages/secure-exec/tests/projects/fastify-pass/src/index.js (use app.listen)
- prd.json (US-051 passes: true)
- **Learnings for future iterations:**
- Node.js res.end(null) is valid and means "end without writing data" — bridge must match this convention
- Fastify v5 calls res.end(null, null, null) in sendTrailer to avoid V8's ArgumentsAdaptorTrampoline — this is a common Node.js pattern
- When debugging sandbox HTTP failures, check the bridge's ServerResponseBridge.write/end for type handling gaps
- Express fixture passes with basic http bridge; Fastify needs null-safe write/end due to internal stream handling
---
## 2026-03-17 - US-052
- Created @secure-exec/core package (packages/secure-exec-core/) with shared types, utilities, and constants
- Moved types.ts, runtime-driver.ts, and all shared/* files to core/src/
- Extracted TIMEOUT_EXIT_CODE and TIMEOUT_ERROR_MESSAGE from isolate.ts into core/src/shared/constants.ts
- Replaced secure-exec originals with re-export shims from @secure-exec/core
- Added @secure-exec/core workspace dependency to secure-exec package.json
- Updated build-isolate-runtime.mjs to sync generated manifest to core package
- Updated isolate-runtime-injection-policy test to read require-setup.ts from core's source
- Files changed: 32 files (16 new in core, 16 modified in secure-exec)
- **Learnings for future iterations:**
- pnpm-workspace.yaml `packages/*` glob automatically picks up packages/secure-exec-core/
- turbo.json `^build` dependency automatically builds upstream workspace deps — no config changes needed
- TypeScript can't resolve `@secure-exec/core` until core's dist/ exists — must build core first
- Re-export files must include ALL exports from the original module (check for missing exports by running tsc)
- Source-grep tests that read shared files must be updated to point to core's canonical source location
- The generated/isolate-runtime.ts must exist in core for require-setup.ts to compile — copy it during build
---
## 2026-03-17 - US-053
- Moved bridge/ directory (11 files) from secure-exec/src/bridge/ to core/src/bridge/
- Moved generated/polyfills.ts to core/src/generated/ (isolate-runtime.ts already in core)
- Moved isolate-runtime/ source directory (19 files) to core/isolate-runtime/
- Moved build-polyfills.mjs and build-isolate-runtime.mjs to core/scripts/
- Moved tsconfig.isolate-runtime.json to core
- Updated core package.json: added build:bridge, build:polyfills, build:isolate-runtime, build:generated scripts; added esbuild and node-stdlib-browser deps; added "default" export condition
- Simplified secure-exec package.json: removed all build:* scripts (now in core), simplified build to just tsc, simplified check-types, removed build:generated prefixes from test scripts
- Updated 7 files in secure-exec to import getIsolateRuntimeSource/POLYFILL_CODE_MAP from @secure-exec/core instead of local generated/
- Updated bridge-loader.ts to resolve core package root via createRequire and find bridge source/bundle in core's directory
- Updated 6 type conformance tests to import bridge modules from core's source
- Updated bridge-registry-policy.test.ts with readCoreSource() helper for reading core-owned files
- Updated isolate-runtime-injection-policy.test.ts to read build script from core/scripts/
- Removed dual-sync code from build-isolate-runtime.mjs (no longer needed — script is now in core)
- Added POLYFILL_CODE_MAP export to core's index.ts barrel
- Files changed: 53 files (moves + import updates)
- **Learnings for future iterations:**
- core's exports map needs a "default" condition (not just "import") for createRequire().resolve() to work — ESM-only exports break require.resolve
- bridge-loader.ts uses createRequire(import.meta.url) to find @secure-exec/core package root, then derives dist/bridge.js and src/bridge/index.ts paths from there
- Generated files (polyfills.ts, isolate-runtime.ts) are gitignored and must be built before tsc — turbo task dependencies handle this automatically
- Kernel integration tests (tests/kernel/) have pre-existing failures unrelated to package restructuring — they use a different code path through runtime-node
- build:bridge produces dist/bridge.js in whichever package owns the bridge source — bridge-loader.ts must know where to find it
---
## 2026-03-17 - US-054
- What was implemented: Moved runtime facades (runtime.ts, python-runtime.ts), filesystem helpers (fs-helpers.ts), ESM compiler (esm-compiler.ts), module resolver (module-resolver.ts), package bundler (package-bundler.ts), and bridge setup (bridge-setup.ts) from secure-exec/src/ to @secure-exec/core
- Files changed:
- packages/secure-exec-core/src/runtime.ts — NEW: NodeRuntime facade (imports from core-local paths)
- packages/secure-exec-core/src/python-runtime.ts — NEW: PythonRuntime facade
- packages/secure-exec-core/src/fs-helpers.ts — NEW: VFS helper functions
- packages/secure-exec-core/src/esm-compiler.ts — NEW: ESM wrapper generator for built-in modules
- packages/secure-exec-core/src/module-resolver.ts — NEW: module classification/resolution with inlined hasPolyfill
- packages/secure-exec-core/src/package-bundler.ts — NEW: VFS module resolution (resolveModule, loadFile, etc.)
- packages/secure-exec-core/src/bridge-setup.ts — NEW: bridge globals setup code loader
- packages/secure-exec-core/src/index.ts — added exports for all 7 new modules
- packages/secure-exec/src/{runtime,python-runtime,fs-helpers,esm-compiler,module-resolver,package-bundler,bridge-setup}.ts — replaced with re-exports from @secure-exec/core
- packages/secure-exec/tests/isolate-runtime-injection-policy.test.ts — updated bridgeSetup source path to read from core
- prd.json — marked US-054 as passes: true
- **Learnings for future iterations:**
- module-resolver.ts depended on hasPolyfill from polyfills.ts — inlined it in core since core already has node-stdlib-browser dependency
- Source policy tests (isolate-runtime-injection-policy) read source files by path and must be updated when moving code to core
- Re-export pattern: replace moved file with `export { X } from "@secure-exec/core"` — all consumers using relative imports from secure-exec keep working unchanged
- Existing consumers in node/, browser/, tests/ that import `../module-resolver.js` etc. don't need changes since the re-export files forward to core
---
## 2026-03-17 - US-055
- What was implemented
- Added subpath exports to @secure-exec/core package.json with `./internal/*` prefix convention
- Subpaths cover all root-level modules (bridge-setup, esm-compiler, fs-helpers, module-resolver, package-bundler, runtime, python-runtime, runtime-driver, types), generated modules (isolate-runtime, polyfills), and shared/* wildcard
- Each subpath export includes types, import, and default conditions
- Skipped bridge-loader subpath since it hasn't been moved to core yet (still in secure-exec)
- Files changed
- packages/secure-exec-core/package.json — added 12 internal subpath exports + shared/* wildcard
- prd.json — marked US-055 as passes: true
- **Learnings for future iterations:**
- Subpath exports with `types` condition require matching `.d.ts` files in dist — tsc already generates these when `declaration: true`
- Wildcard subpath exports (`./internal/shared/*`) map to `./dist/shared/*.js` — Node resolves the `*` placeholder
- `./internal/` prefix is a convention signal, not enforced — runtime packages can import but external consumers should not
- bridge-loader.ts is in secure-exec (not core) — future stories (US-056) will move it to @secure-exec/node
- Pre-existing WasmVM/kernel test failures are unrelated to package config changes — they require the WASM binary built locally
---
## 2026-03-17 - US-056
- What was implemented: Created @secure-exec/node package and moved V8 execution engine files
- Files changed:
- packages/secure-exec-node/package.json — new package with deps: @secure-exec/core, isolated-vm, esbuild, node-stdlib-browser
- packages/secure-exec-node/tsconfig.json — standard ES2022/NodeNext config
- packages/secure-exec-node/src/index.ts — barrel exporting all moved modules
- packages/secure-exec-node/src/execution.ts — V8 execution loop (moved from secure-exec, imports updated to @secure-exec/core)
- packages/secure-exec-node/src/isolate.ts — V8 isolate utilities (moved, imports updated)
- packages/secure-exec-node/src/bridge-loader.ts — esbuild bridge compilation (moved, imports unchanged since already used @secure-exec/core)
- packages/secure-exec-node/src/polyfills.ts — esbuild stdlib bundling (moved, no import changes needed)
- packages/secure-exec/src/execution.ts — replaced with re-export stub from @secure-exec/node
- packages/secure-exec/src/isolate.ts — replaced with re-export stub from @secure-exec/node
- packages/secure-exec/src/bridge-loader.ts — replaced with re-export stub from @secure-exec/node
- packages/secure-exec/src/polyfills.ts — replaced with re-export stub from @secure-exec/node
- packages/secure-exec/src/python/driver.ts — updated to import TIMEOUT_* constants from @secure-exec/core directly
- packages/secure-exec/tests/isolate-runtime-injection-policy.test.ts — updated source-grep test to read bridge-loader.ts from canonical location (@secure-exec/node)
- packages/secure-exec/package.json — added @secure-exec/node workspace dependency
- pnpm-lock.yaml — updated for new package
- prd.json — marked US-056 as passes: true
- **Learnings for future iterations:**
- turbo.json ^build handles workspace dependency ordering automatically — no turbo.json changes needed when adding new workspace packages
- Re-export stubs in secure-exec preserve backward compatibility for internal consumers (node/*, python/*) while the canonical code moves to @secure-exec/node
- Source-grep policy tests (isolate-runtime-injection-policy.test.ts) must be updated when source files move — they read source by path
- python/driver.ts only needed TIMEOUT_ERROR_MESSAGE and TIMEOUT_EXIT_CODE from isolate.ts — these are already in @secure-exec/core, so direct import avoids dependency on @secure-exec/node
- @secure-exec/node uses internal/* subpath exports (./internal/execution, ./internal/isolate, etc.) matching the pattern established by @secure-exec/core
- pnpm-workspace.yaml `packages/*` glob auto-discovers packages/secure-exec-node/ — no workspace config changes needed
---
## 2026-03-17 - US-057
- Moved 8 node/ source files (execution-driver, isolate-bootstrap, module-resolver, execution-lifecycle, esm-compiler, bridge-setup, driver, module-access) from secure-exec/src/node/ to @secure-exec/node (packages/secure-exec-node/src/)
- Updated all imports in moved files: `../shared/*` → `@secure-exec/core/internal/shared/*`, `../isolate.js` → `./isolate.js`, `../types.js` → `@secure-exec/core`, etc.
- Added 8 new subpath exports to @secure-exec/node package.json
- Updated @secure-exec/node index.ts to export public API (NodeExecutionDriver, createNodeDriver, createNodeRuntimeDriverFactory, NodeFileSystem, createDefaultNetworkAdapter, ModuleAccessFileSystem)
- Replaced original files in secure-exec/src/node/ with thin re-export stubs pointing to @secure-exec/node
- Updated secure-exec barrel (index.ts) to re-export from @secure-exec/node instead of ./node/driver.js
- Updated source-grep policy tests (isolate-runtime-injection-policy, bridge-registry-policy) to read from canonical @secure-exec/node location
- Files changed: 21 files (8 new in secure-exec-node, 8 replaced in secure-exec/src/node/, 1 barrel, 2 test files, 1 package.json, 1 index.ts)
- **Learnings for future iterations:**
- bridge compilation is already handled by @secure-exec/core's build:bridge step; @secure-exec/node just imports getRawBridgeCode() — no separate build:bridge needed in node package
- Source policy tests read source files by filesystem path, not by import — must update paths when moving code between packages
- @secure-exec/core/internal/shared/* wildcard export provides access to all shared modules, so moved files can use subpath imports
---
## 2026-03-17 - US-058
- Updated packages/runtime/node/ to depend on @secure-exec/node + @secure-exec/core instead of secure-exec
- Files changed:
- packages/runtime/node/package.json — replaced `secure-exec` dep with `@secure-exec/core` + `@secure-exec/node`
- packages/runtime/node/src/driver.ts — updated imports: NodeExecutionDriver/createNodeDriver from @secure-exec/node, allowAllChildProcess/types from @secure-exec/core
- pnpm-lock.yaml — regenerated
- Verified: no transitive dependency on pyodide or browser code; `pnpm why pyodide` and `pnpm why secure-exec` return empty
- All 24 tests pass, typecheck passes
- **Learnings for future iterations:**
- @secure-exec/core exports all shared types (CommandExecutor, VirtualFileSystem) and permissions (allowAllChildProcess) — use it for type-only and utility imports
- @secure-exec/node exports V8-specific code (NodeExecutionDriver, createNodeDriver) — use it for execution engine imports
- pnpm install (without --frozen-lockfile) is needed when changing workspace dependencies
---
## 2026-03-17 - US-059