Release the audio session when voice capture and playback go idle (#2866)

* Release the audio session when voice capture and playback go idle

Opening Paseo permanently paused music playing in other apps. Once the
mic had been used a single time, every subsequent foreground of the app
killed the user's audio again until the app was force-quit.

iOS AudioEngine.setupAudioSession() takes .playAndRecord/.voiceChat — a
non-mixing category — and calls setActive(true), but the only
setActive(false) was reachable from tearDown(). Nothing handed the
session back after a normal dictation turn, so iOS re-asserted the last
category the app set on every foreground. The iOS module had no
app-lifecycle handling at all, while Android already had a lifecycle
listener; that asymmetry is what made the platforms diverge. Android had
the same leak in a different shape: AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE
was requested per recording and never abandoned.

Add releaseAudioSession() to both platforms and call it once capture
stops and the playback queue drains. iOS pauses the engine, calls
setActive(false, .notifyOthersOnDeactivation) — this is what tells the
other app to resume — then drops to .ambient/.mixWithOthers so any
implicit reactivation mixes instead of interrupting. The session is
re-acquired lazily on toggleRecording(true)/playPCMData, so voice keeps
working; resumeRecordingAndPlayer() re-activates before restarting the
engine, which keeps post-interruption resume intact. Releasing is
guarded on isRecording/isPlaying, and checkEngineIsRunning() now refuses
to start the engine while the session is released, since starting it
would silently re-take the session.

The JS wrapper probes the native module before calling, because a JS
bundle shipped over EAS Update can land on a binary predating this
function.

* Correct the iOS signing diagnosis in mobile-testing docs

The "No Accounts" error does not mean no Apple ID is configured. The
account can be registered in Xcode while its Xcode-Token keychain item is
missing, which Xcode reports as No Accounts and then compounds by falling
back to a stale cached profile — surfacing as a certificate mismatch that
sends you chasing the wrong thing.

Document the DVTDeveloperAccountManager log line that names the real
cause, the two commands that check what is actually configured, and the
build-unsigned-then-sign-manually stopgap.

* Reacquire Android audio focus when playback restarts

releaseAudioSession() abandons the AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE
request and pauses the AudioTrack once capture and playback go idle, but
nothing took either back. The next playback turn therefore ran with no
focus held, so the assistant's response competed with whatever the user
was listening to instead of interrupting it.

Add acquireAudioSessionIfNeeded(), the mirror of the iOS
activateAudioSessionIfNeeded() already called from playPCMData, and use
it from both playPCMData() and resumePlayback(). A null audioFocusRequest
is what "we released" looks like, so it doubles as the guard against
re-requesting focus mid-turn.

Reported by Greptile on #2866.

* Make Android playback focus reacquisition non-fatal

The previous commit dropped PCM data when focus reacquisition did not
return AUDIOFOCUS_REQUEST_GRANTED. That is worse than playing unfocused:
playPCMData is fire-and-forget from JS, and playAudio() settles on a
duration timer rather than a native ack, so the runtime acknowledged and
discarded a chunk that was never queued.

It also broke delayed focus. The request is built with
setAcceptsDelayedFocusGain(true), so AUDIOFOCUS_REQUEST_DELAYED is a
normal outcome, but routing it through handleAudioFocusBlocked() abandoned
the pending request and the later grant could never arrive.

Request focus, then play regardless of the result, and leave the request
outstanding so a delayed grant still lands via the listener.

Reported by Greptile on #2866.

* Track scheduled buffers so iOS actually releases after playback

releaseAudioSession() guarded on !speechPlayer.isPlaying, but
AVAudioPlayerNode.isPlaying stays true from play() until an explicit
stop()/pause() — draining every scheduled buffer does not clear it. The
normal completion path is processQueue() -> releaseSessionIfIdle(), and
stopPlayback() is only reached from the explicit stop() API, so after the
first spoken response the guard blocked every release and the session
stayed active. The music-pausing bug returned in full for anyone using
voice responses.

Count buffers that are scheduled but not yet rendered, decrementing from
the .dataPlayedBack completion handler, and guard on that instead. The
handler runs on an AVAudioEngine-internal thread, so the count is behind a
lock. Both stop() paths reset it, since stop() discards what is scheduled
and a stale count would block every later release.

Found by Codex review on #2866.

* Stop re-taking Android audio focus on every foreground

ExpoTwoWayAudioLifeCycleListener calls resumeRecordingAndPlayer() from
onResume, and that requested AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE before
looking at isRecordingBeforePause. Foregrounding the app while nothing was
recording therefore took exclusive focus back and re-paused the user's
music — the same "every foreground kills my music" symptom this change set
exists to remove, just reached through the Android lifecycle instead of the
iOS category.

Return early unless capture or playback was actually live when we
backgrounded.

Found by Codex review on #2866.
This commit is contained in:
Kevin Aspesi
2026-08-05 22:53:23 +02:00
committed by GitHub
parent 0728148657
commit fb5cfb9fb9
7 changed files with 329 additions and 18 deletions
+124
View File
@@ -252,6 +252,25 @@ adb emu gsm call 5551234
Expected result: Paseo does not throw `RuntimeException: Audio focus request failed`; native audio reports an interruption and voice mode stops or pauses coherently.
### Releasing the audio session when idle
Paseo must not hold the OS audio session once it is neither capturing nor playing, or the user's
background music stays paused. On iOS this is not just a "while recording" problem: the
`.playAndRecord`/`.voiceChat` category is non-mixing and survives backgrounding, and iOS re-asserts
it every time the app returns to the foreground — so one dictation turn kills music for the life of
the process, on every open, until the app is force-quit. Android holds
`AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE` with the same effect.
`createAudioEngine` (`packages/app/src/voice/audio-engine.native.ts`) calls
`releaseAudioSession()` whenever capture stops and the playback queue drains, and the iOS module
also releases on `OnAppEntersBackground`. The native side re-guards on `isRecording` /
`speechPlayer.isPlaying`, because the native engine is a singleton shared by multiple JS engine
wrappers (voice provider + dictation) and only it knows the true state.
This cannot be validated by a JS test — verify on a device: play music, open Paseo, use dictation
once, stop, and confirm the music resumes; then background/foreground the app and confirm it keeps
playing.
## Unistyles + Reanimated
### The crash
@@ -320,6 +339,111 @@ Platform-specific stream edges belong on `StreamStrategy`:
If a chat footer looks duplicated or appears above the assistant message on mobile, start with `packages/app/src/agent-stream/layout.test.ts`. Do not add a React Native renderer test for this class of bug; make the pure layout invariant fail first.
## Local iOS device builds
Native changes (anything under `packages/expo-two-way-audio/ios/`, or any new Expo module) **cannot be
tested over Metro or EAS Update**. Those ship JS only. A stale dev-client binary silently lacks the new
native functions, so the feature no-ops and the test proves nothing. Rebuild the binary.
Symptom of a stale dev client: `Cannot find native module 'ExpoDocumentPicker'` at startup, followed by a
cascade of `Route "./X.tsx" is missing the required default export` warnings. The route warnings are not a
router bug — the module-level throw aborts `_layout.tsx` evaluation, so its default export never gets
assigned. One missing native module, N confusing warnings.
**Prerequisite: Xcode needs a signed-in Apple ID for the signing team _with a valid keychain token_.** An
account that merely appears in Xcode → Settings → Accounts is not enough — if its `Xcode-Token` is missing,
the build below fails at code signing and blames a _certificate_ instead. See
[Signing needs a working Apple ID token in Xcode](#signing-needs-a-working-apple-id-token-in-xcode) before
you start, not after it fails.
```bash
cd packages/app
npm --prefix ../.. run build:client
APP_VARIANT=development npx expo prebuild --platform ios
APP_VARIANT=development npx expo run:ios --device
```
`APP_VARIANT=development` is required. The `ios` npm script does not set it, and `app.config.js` defaults
to `production` — so a bare `npm run ios` builds `sh.paseo` and collides with the App Store install instead
of the `sh.paseo.debug` dev client. Ignore prebuild's `--non-interactive is not supported` warning; use
`CI=1` if you need non-interactive.
### Signing needs a working Apple ID token in Xcode
Device builds fail with a trio of errors that all point away from the real cause:
```
error: No Accounts: Add a new account in Accounts settings.
error: Provisioning profile "iOS Team Provisioning Profile: *" doesn't include
signing certificate "Apple Development: <name>".
error: Provisioning profile "iOS Team Provisioning Profile: *" doesn't include
the aps-environment entitlement.
```
**`No Accounts` does not reliably mean no account is configured.** Check the line _above_ it in the build
log before believing it:
```
DVTDeveloperAccountManager: Failed to load credentials for <apple-id>:
Invalid credentials in keychain for <apple-id>, missing Xcode-Token
```
That is the actual failure: the Apple ID is registered in Xcode, but its `Xcode-Token` keychain item is gone
(typically after an Apple ID password change or a keychain reset). Xcode reports it as `No Accounts`, then
falls back to whatever stale profile is cached — and if the machine holds a dev cert newer than that
profile, you get the _certificate_ mismatch as a second-order symptom. Chasing the certificate error is a
dead end.
Verify what is actually configured rather than trusting the message:
```bash
# Accounts Xcode knows about
defaults read com.apple.dt.Xcode DVTDeveloperAccountManagerAppleIDLists
# The credential that is actually missing (error => no token)
security find-generic-password -s "Xcode-Token"
```
Fix: Xcode → Settings → Accounts, re-sign-in to the listed Apple ID (removing and re-adding forces a fresh
token). That restores `-allowProvisioningUpdates`, profile regeneration, and push.
Signing an Xcode-_managed_ profile via `CODE_SIGN_STYLE=Manual` is not a workaround; Xcode rejects it with
`is Xcode managed, but signing settings require a manually managed profile`.
If you need a build on the device before the token is fixed, build unsigned and sign by hand with a cert the
cached profile already embeds (list them with `security cms -D -i <profile>.mobileprovision`, then compare
against `security find-identity -v -p codesigning`):
```bash
xcodebuild ... CODE_SIGNING_ALLOWED=NO CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY="" build
cp <profile>.mobileprovision "$APP/embedded.mobileprovision"
# sign nested frameworks/dylibs first, then the bundle, then:
xcrun devicectl device install app --device <udid> "$APP"
```
This is a stopgap, not a fix: `expo prebuild` regenerates `packages/app/ios/` and discards it. It also
requires stripping entitlements the cached profile lacks (e.g. `aps-environment`), so push is dead in such a
build.
To verify only that native Swift compiles, skip signing entirely — the pods have their own schemes:
```bash
cd packages/app/ios
xcodebuild -workspace PaseoDebug.xcworkspace -scheme ExpoTwoWayAudio \
-sdk iphoneos -destination 'generic/platform=iOS' CODE_SIGNING_ALLOWED=NO build
```
Confirm the log compiled _your_ file and not a cached copy: the `SwiftCompile` line should reference
`packages/expo-two-way-audio/ios/...`, not a path inside DerivedData.
### Headless launch cannot verify runtime behavior
`xcrun devicectl device process launch` reports `The app terminated with the exit code 0` about a second
after launch when the phone's screen is off — the app is suspended and killed before the JS bundle loads.
This is **not** a build defect. Confirm with a control: launch `com.apple.Preferences` the same way and
watch it exit identically. Headless install and launch prove the binary is valid and that startup reaches
RN init; anything about on-screen behavior needs a human holding the phone.
## iOS Simulator
```bash