feat: live macOS dictation — push-to-talk mic into the realtime ASR session #1

2026-09-09T19:07:35+03:00

iulianp commented

2026-09-09 19:07:35 +03:00

/dictate + ctrl+shift+d toggle one shared session: press to start, press again
to stop, flush and paste into the prompt (a terminal gives no key-release event
for hold-to-talk without the Kitty keyboard protocol)
client.recordMic(): spawns ffmpeg AVFoundation capture and appends s16le frames
as they arrive (no pacer — the mic is already 1x paced); stop() = commit + the
same quiescence drain transcribe() uses, since is_last_result never flips true
enumerates mics by slicing the "AVFoundation audio devices:" block off stderr
(ffmpeg prints the table there, then fails opening the dummy input)
--device takes an index or name, --lang / ASR_DICTATE_LANGUAGE, --submit sends
instead of pasting; asr_health now reports the mic inputs it enumerated
capture death surfaces via the emptyGraceMs path with the TCC hint (System
Settings → Privacy & Security → Microphone) when ffmpeg exits non-zero at 0 bytes
env.sh sources the gitignored .env so a shell gets ASR_NIM_URL / ASR_API_KEY

- /dictate + ctrl+shift+d toggle one shared session: press to start, press again to stop, flush and paste into the prompt (a terminal gives no key-release event for hold-to-talk without the Kitty keyboard protocol) - client.recordMic(): spawns ffmpeg AVFoundation capture and appends s16le frames as they arrive (no pacer — the mic is already 1x paced); stop() = commit + the same quiescence drain transcribe() uses, since is_last_result never flips true - enumerates mics by slicing the "AVFoundation audio devices:" block off stderr (ffmpeg prints the table there, then fails opening the dummy input) - --device takes an index or name, --lang / ASR_DICTATE_LANGUAGE, --submit sends instead of pasting; asr_health now reports the mic inputs it enumerated - capture death surfaces via the emptyGraceMs path with the TCC hint (System Settings → Privacy & Security → Microphone) when ffmpeg exits non-zero at 0 bytes - env.sh sources the gitignored .env so a shell gets ASR_NIM_URL / ASR_API_KEY

iulianp added 1 commit 2026-09-09 19:07:36 +03:00

feat: live macOS dictation — push-to-talk mic into the realtime ASR session 0b44d7e1bf

- /dictate + ctrl+shift+d toggle one shared session: press to start, press again
  to stop, flush and paste into the prompt (a terminal gives no key-release event
  for hold-to-talk without the Kitty keyboard protocol)
- client.recordMic(): spawns ffmpeg AVFoundation capture and appends s16le frames
  as they arrive (no pacer — the mic is already 1x paced); stop() = commit + the
  same quiescence drain transcribe() uses, since is_last_result never flips true
- enumerates mics by slicing the "AVFoundation audio devices:" block off stderr
  (ffmpeg prints the table there, then fails opening the dummy input)
- --device takes an index or name, --lang / ASR_DICTATE_LANGUAGE, --submit sends
  instead of pasting; asr_health now reports the mic inputs it enumerated
- capture death surfaces via the emptyGraceMs path with the TCC hint (System
  Settings → Privacy & Security → Microphone) when ffmpeg exits non-zero at 0 bytes
- env.sh sources the gitignored .env so a shell gets ASR_NIM_URL / ASR_API_KEY

iulianp added 1 commit 2026-09-09 20:31:12 +03:00

wip: incremental ASR 65855ace6c

iulianp added 1 commit 2026-09-10 22:00:25 +03:00

wip: incremental ASR outputs in realtime, few bugs ecd686073e

You are not authorized to merge this pull request.

This pull request can be merged automatically.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feat/macos-live-dictation:feat/macos-live-dictation

git checkout feat/macos-live-dictation

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: iulianp/pi-asr#1