Skip to content

fix: null-terminate syscall paths, fix poll/select PM deadlock, stop render thread competing with BWM#215

Merged
ryanbreen merged 1 commit intomainfrom
fix/pty-null-termination-and-poll-deadlock
Feb 14, 2026
Merged

fix: null-terminate syscall paths, fix poll/select PM deadlock, stop render thread competing with BWM#215
ryanbreen merged 1 commit intomainfrom
fix/pty-null-termination-and-poll-deadlock

Conversation

@ryanbreen
Copy link
Owner

Summary

  • CPath null-termination: All 11 path-taking libbreenix syscall wrappers now use write_volatile to ensure null-terminated paths. Rust &str is NOT null-terminated and the compiler can optimize away trailing zeroes, causing the kernel's copy_cstr_from_user() to read garbage.
  • sys_poll/sys_select PM fix: Snapshot fd entries under PROCESS_MANAGER (which masks all interrupts on ARM64), drop PM, then iterate. Eliminates holding PM across PTY buffer locks, TCP connections, etc.
  • Render thread display takeover: Add DISPLAY_TAKEN flag (standard fbcon/KD_GRAPHICS pattern) so the render thread stops competing with BWM for SHELL_FRAMEBUFFER during GPU busy-waits. Also move PM acquisition before FB lock in sys_fbdraw to prevent nested interrupt-masking inside long-held spinlocks.
  • Remove all diagnostic serial_println!/print! from previous debugging sessions.

Test plan

  • ARM64 boot test passes (run-aarch64-boot-test-native.sh)
  • Interactive mode: BWM terminal stays responsive after typing commands
  • x86-64 boot test passes (run-boot-parallel.sh 1)

🤖 Generated with Claude Code

…render thread competing with BWM

Three categories of fixes for ARM64 interactive mode lockups:

1. **Null-terminated paths for kernel syscalls** (libs/libbreenix/src/fs.rs)
   Add CPath struct that ensures all path strings passed to kernel syscalls
   are properly null-terminated with write_volatile. Rust &str is NOT null-
   terminated, and the compiler can optimize away zero bytes after the string
   data. Applied to all 11 path-taking functions: open, open_with_mode,
   access, unlink, mkdir, rmdir, rename, link, symlink, readlink, mkfifo.

2. **Drop PROCESS_MANAGER before I/O in sys_poll and sys_select**
   (kernel/src/syscall/handlers.rs)
   On ARM64, process::manager() disables ALL interrupts (DAIF mask 0xF).
   Previously sys_poll/sys_select held PM across the entire poll loop
   including PTY buffer locks, TCP connection locks, etc. Now they snapshot
   fd entries under PM (cheap Arc::clone), drop PM, then iterate. This
   follows the same extract-then-drop pattern used by sys_read/sys_write.

3. **Render thread stops competing with BWM for GPU** (render_task.rs,
   graphics.rs, handlers.rs)
   - Add DISPLAY_TAKEN flag set by sys_take_over_display(). Render thread
     skips framebuffer flush and cursor updates when BWM owns the display
     (standard pattern — same as Linux fbcon KD_GRAPHICS mode).
   - Move PROCESS_MANAGER acquisition in sys_fbdraw BEFORE the
     SHELL_FRAMEBUFFER lock. Previously PM (interrupts disabled) was
     acquired nested inside the framebuffer lock during GPU busy-waits.
   - Remove all diagnostic serial_println/print! from previous debugging.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ryanbreen ryanbreen merged commit 6501ed2 into main Feb 14, 2026
2 of 4 checks passed
@ryanbreen ryanbreen deleted the fix/pty-null-termination-and-poll-deadlock branch February 14, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant