fix(keyboard): prevent modifier key auto-release during typing (#1386)

[adamshiervani]

Summary

• Fix modifier keys (Shift, Ctrl, Alt) being dropped after the first key combination when held during typing
• Root cause: browser key-repeat events (~30Hz) cancelled and restarted the keepalive interval on every keydown, but the repeat rate (~33ms) was shorter than the keepalive period (50ms), so the keepalive tick could never fire. When a second key was pressed and the modifier's repeat stopped, the modifier's 100ms auto-release timer expired with no keepalive to extend it
• Fix: start the keepalive interval on first key press and leave it running undisturbed until all keys are released; track held keys client-side via a Set<number> to know when to start/stop
• Also fixes pre-existing oxlint warnings in useKeyboard.ts (no-floating-promises, restrict-template-expressions)

Test plan

• New e2e test: modifier held with key-repeat simulation (20 rapid presses)
• New e2e test: modifier held across rapid tap burst (20 keys at 50ms)
• New e2e test: multiple simultaneous modifiers (Ctrl+Shift+key)
• New e2e test: reversed release order (modifier up before non-modifier)
• New e2e test: AltGr (AltRight) held while tapping
• New e2e test: modifier held while tapping 10 keys over 10 seconds
• All 39 remote-agent e2e tests pass
• Full make test_e2e suite (47 tests) passes
• Manual verification: Shift+JSON types "JSON" correctly on deployed device

Closes #1386

---

Note

Medium Risk
Changes the client-side HID keepalive scheduling used to prevent device auto-release, which can affect all keyboard input timing and stuck-key behavior; adds broad e2e coverage to reduce regression risk.

Overview
Fixes a regression where held modifiers (e.g. Shift/Ctrl/Alt) could auto-release during typing by changing the keepalive logic in useKeyboard to keep a single interval running while any keys are held, instead of cancelling/restarting it on each press.

Adds client-side tracking of currently held keys and ensures the keepalive tick always uses the latest HID RPC sender across reconnects; also cleans up a few legacy RPC sends/logging to avoid floating promises. Expands ra-all.spec.ts with several new e2e scenarios covering long holds, key-repeat, burst typing, multiple modifiers, reversed release order, and AltGr to validate modifiers are not dropped.

^{Written by Cursor Bugbot for commit 6189cba. This will update automatically on new commits. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Stale keepalive closure after RPC channel reconnection
  • Added tracking of scheduleKeepAlive reference to detect and restart the keepalive timer when RPC channel reconnects, while preserving the key-repeat starvation fix.

Or push these changes by commenting:

@cursor push 3792219466

Preview (3792219466)

diff --git a/ui/src/hooks/useKeyboard.ts b/ui/src/hooks/useKeyboard.ts
--- a/ui/src/hooks/useKeyboard.ts
+++ b/ui/src/hooks/useKeyboard.ts
@@ -212,6 +212,9 @@
     return { modifier: modifiers, keys };
   }
 
+  // Track the current scheduleKeepAlive reference to detect reconnections
+  const scheduleKeepAliveRef = useRef(scheduleKeepAlive);
+
   const sendKeypress = useCallback(
     (key: number, press: boolean) => {
       if (press) {
@@ -227,7 +230,10 @@
         // interval — key-repeat events fire faster (~33ms) than the keepalive
         // period (50ms), so cancel+restart would prevent the tick from ever
         // firing, starving the device-side auto-release extension (issue #1386).
-        if (!keepAliveTimerRef.current) {
+        // However, if scheduleKeepAlive changed (e.g., RPC reconnection), restart
+        // the timer to pick up the new closure.
+        if (!keepAliveTimerRef.current || scheduleKeepAliveRef.current !== scheduleKeepAlive) {
+          scheduleKeepAliveRef.current = scheduleKeepAlive;
           scheduleKeepAlive();
         }
       } else {

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

[mbassalbioinformatics]

Issue is still present in 0.5.6 release. Bug not fixed.