Release the audio session when voice capture and playback go idle (#2866)
* Release the audio session when voice capture and playback go idle Opening Paseo permanently paused music playing in other apps. Once the mic had been used a single time, every subsequent foreground of the app killed the user's audio again until the app was force-quit. iOS AudioEngine.setupAudioSession() takes .playAndRecord/.voiceChat — a non-mixing category — and calls setActive(true), but the only setActive(false) was reachable from tearDown(). Nothing handed the session back after a normal dictation turn, so iOS re-asserted the last category the app set on every foreground. The iOS module had no app-lifecycle handling at all, while Android already had a lifecycle listener; that asymmetry is what made the platforms diverge. Android had the same leak in a different shape: AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE was requested per recording and never abandoned. Add releaseAudioSession() to both platforms and call it once capture stops and the playback queue drains. iOS pauses the engine, calls setActive(false, .notifyOthersOnDeactivation) — this is what tells the other app to resume — then drops to .ambient/.mixWithOthers so any implicit reactivation mixes instead of interrupting. The session is re-acquired lazily on toggleRecording(true)/playPCMData, so voice keeps working; resumeRecordingAndPlayer() re-activates before restarting the engine, which keeps post-interruption resume intact. Releasing is guarded on isRecording/isPlaying, and checkEngineIsRunning() now refuses to start the engine while the session is released, since starting it would silently re-take the session. The JS wrapper probes the native module before calling, because a JS bundle shipped over EAS Update can land on a binary predating this function. * Correct the iOS signing diagnosis in mobile-testing docs The "No Accounts" error does not mean no Apple ID is configured. The account can be registered in Xcode while its Xcode-Token keychain item is missing, which Xcode reports as No Accounts and then compounds by falling back to a stale cached profile — surfacing as a certificate mismatch that sends you chasing the wrong thing. Document the DVTDeveloperAccountManager log line that names the real cause, the two commands that check what is actually configured, and the build-unsigned-then-sign-manually stopgap. * Reacquire Android audio focus when playback restarts releaseAudioSession() abandons the AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE request and pauses the AudioTrack once capture and playback go idle, but nothing took either back. The next playback turn therefore ran with no focus held, so the assistant's response competed with whatever the user was listening to instead of interrupting it. Add acquireAudioSessionIfNeeded(), the mirror of the iOS activateAudioSessionIfNeeded() already called from playPCMData, and use it from both playPCMData() and resumePlayback(). A null audioFocusRequest is what "we released" looks like, so it doubles as the guard against re-requesting focus mid-turn. Reported by Greptile on #2866. * Make Android playback focus reacquisition non-fatal The previous commit dropped PCM data when focus reacquisition did not return AUDIOFOCUS_REQUEST_GRANTED. That is worse than playing unfocused: playPCMData is fire-and-forget from JS, and playAudio() settles on a duration timer rather than a native ack, so the runtime acknowledged and discarded a chunk that was never queued. It also broke delayed focus. The request is built with setAcceptsDelayedFocusGain(true), so AUDIOFOCUS_REQUEST_DELAYED is a normal outcome, but routing it through handleAudioFocusBlocked() abandoned the pending request and the later grant could never arrive. Request focus, then play regardless of the result, and leave the request outstanding so a delayed grant still lands via the listener. Reported by Greptile on #2866. * Track scheduled buffers so iOS actually releases after playback releaseAudioSession() guarded on !speechPlayer.isPlaying, but AVAudioPlayerNode.isPlaying stays true from play() until an explicit stop()/pause() — draining every scheduled buffer does not clear it. The normal completion path is processQueue() -> releaseSessionIfIdle(), and stopPlayback() is only reached from the explicit stop() API, so after the first spoken response the guard blocked every release and the session stayed active. The music-pausing bug returned in full for anyone using voice responses. Count buffers that are scheduled but not yet rendered, decrementing from the .dataPlayedBack completion handler, and guard on that instead. The handler runs on an AVAudioEngine-internal thread, so the count is behind a lock. Both stop() paths reset it, since stop() discards what is scheduled and a stale count would block every later release. Found by Codex review on #2866. * Stop re-taking Android audio focus on every foreground ExpoTwoWayAudioLifeCycleListener calls resumeRecordingAndPlayer() from onResume, and that requested AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE before looking at isRecordingBeforePause. Foregrounding the app while nothing was recording therefore took exclusive focus back and re-paused the user's music — the same "every foreground kills my music" symptom this change set exists to remove, just reached through the Android lifecycle instead of the iOS category. Return early unless capture or playback was actually live when we backgrounded. Found by Codex review on #2866.
This commit is contained in:
@@ -252,6 +252,25 @@ adb emu gsm call 5551234
|
||||
|
||||
Expected result: Paseo does not throw `RuntimeException: Audio focus request failed`; native audio reports an interruption and voice mode stops or pauses coherently.
|
||||
|
||||
### Releasing the audio session when idle
|
||||
|
||||
Paseo must not hold the OS audio session once it is neither capturing nor playing, or the user's
|
||||
background music stays paused. On iOS this is not just a "while recording" problem: the
|
||||
`.playAndRecord`/`.voiceChat` category is non-mixing and survives backgrounding, and iOS re-asserts
|
||||
it every time the app returns to the foreground — so one dictation turn kills music for the life of
|
||||
the process, on every open, until the app is force-quit. Android holds
|
||||
`AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE` with the same effect.
|
||||
|
||||
`createAudioEngine` (`packages/app/src/voice/audio-engine.native.ts`) calls
|
||||
`releaseAudioSession()` whenever capture stops and the playback queue drains, and the iOS module
|
||||
also releases on `OnAppEntersBackground`. The native side re-guards on `isRecording` /
|
||||
`speechPlayer.isPlaying`, because the native engine is a singleton shared by multiple JS engine
|
||||
wrappers (voice provider + dictation) and only it knows the true state.
|
||||
|
||||
This cannot be validated by a JS test — verify on a device: play music, open Paseo, use dictation
|
||||
once, stop, and confirm the music resumes; then background/foreground the app and confirm it keeps
|
||||
playing.
|
||||
|
||||
## Unistyles + Reanimated
|
||||
|
||||
### The crash
|
||||
@@ -320,6 +339,111 @@ Platform-specific stream edges belong on `StreamStrategy`:
|
||||
|
||||
If a chat footer looks duplicated or appears above the assistant message on mobile, start with `packages/app/src/agent-stream/layout.test.ts`. Do not add a React Native renderer test for this class of bug; make the pure layout invariant fail first.
|
||||
|
||||
## Local iOS device builds
|
||||
|
||||
Native changes (anything under `packages/expo-two-way-audio/ios/`, or any new Expo module) **cannot be
|
||||
tested over Metro or EAS Update**. Those ship JS only. A stale dev-client binary silently lacks the new
|
||||
native functions, so the feature no-ops and the test proves nothing. Rebuild the binary.
|
||||
|
||||
Symptom of a stale dev client: `Cannot find native module 'ExpoDocumentPicker'` at startup, followed by a
|
||||
cascade of `Route "./X.tsx" is missing the required default export` warnings. The route warnings are not a
|
||||
router bug — the module-level throw aborts `_layout.tsx` evaluation, so its default export never gets
|
||||
assigned. One missing native module, N confusing warnings.
|
||||
|
||||
**Prerequisite: Xcode needs a signed-in Apple ID for the signing team _with a valid keychain token_.** An
|
||||
account that merely appears in Xcode → Settings → Accounts is not enough — if its `Xcode-Token` is missing,
|
||||
the build below fails at code signing and blames a _certificate_ instead. See
|
||||
[Signing needs a working Apple ID token in Xcode](#signing-needs-a-working-apple-id-token-in-xcode) before
|
||||
you start, not after it fails.
|
||||
|
||||
```bash
|
||||
cd packages/app
|
||||
npm --prefix ../.. run build:client
|
||||
APP_VARIANT=development npx expo prebuild --platform ios
|
||||
APP_VARIANT=development npx expo run:ios --device
|
||||
```
|
||||
|
||||
`APP_VARIANT=development` is required. The `ios` npm script does not set it, and `app.config.js` defaults
|
||||
to `production` — so a bare `npm run ios` builds `sh.paseo` and collides with the App Store install instead
|
||||
of the `sh.paseo.debug` dev client. Ignore prebuild's `--non-interactive is not supported` warning; use
|
||||
`CI=1` if you need non-interactive.
|
||||
|
||||
### Signing needs a working Apple ID token in Xcode
|
||||
|
||||
Device builds fail with a trio of errors that all point away from the real cause:
|
||||
|
||||
```
|
||||
error: No Accounts: Add a new account in Accounts settings.
|
||||
error: Provisioning profile "iOS Team Provisioning Profile: *" doesn't include
|
||||
signing certificate "Apple Development: <name>".
|
||||
error: Provisioning profile "iOS Team Provisioning Profile: *" doesn't include
|
||||
the aps-environment entitlement.
|
||||
```
|
||||
|
||||
**`No Accounts` does not reliably mean no account is configured.** Check the line _above_ it in the build
|
||||
log before believing it:
|
||||
|
||||
```
|
||||
DVTDeveloperAccountManager: Failed to load credentials for <apple-id>:
|
||||
Invalid credentials in keychain for <apple-id>, missing Xcode-Token
|
||||
```
|
||||
|
||||
That is the actual failure: the Apple ID is registered in Xcode, but its `Xcode-Token` keychain item is gone
|
||||
(typically after an Apple ID password change or a keychain reset). Xcode reports it as `No Accounts`, then
|
||||
falls back to whatever stale profile is cached — and if the machine holds a dev cert newer than that
|
||||
profile, you get the _certificate_ mismatch as a second-order symptom. Chasing the certificate error is a
|
||||
dead end.
|
||||
|
||||
Verify what is actually configured rather than trusting the message:
|
||||
|
||||
```bash
|
||||
# Accounts Xcode knows about
|
||||
defaults read com.apple.dt.Xcode DVTDeveloperAccountManagerAppleIDLists
|
||||
|
||||
# The credential that is actually missing (error => no token)
|
||||
security find-generic-password -s "Xcode-Token"
|
||||
```
|
||||
|
||||
Fix: Xcode → Settings → Accounts, re-sign-in to the listed Apple ID (removing and re-adding forces a fresh
|
||||
token). That restores `-allowProvisioningUpdates`, profile regeneration, and push.
|
||||
|
||||
Signing an Xcode-_managed_ profile via `CODE_SIGN_STYLE=Manual` is not a workaround; Xcode rejects it with
|
||||
`is Xcode managed, but signing settings require a manually managed profile`.
|
||||
|
||||
If you need a build on the device before the token is fixed, build unsigned and sign by hand with a cert the
|
||||
cached profile already embeds (list them with `security cms -D -i <profile>.mobileprovision`, then compare
|
||||
against `security find-identity -v -p codesigning`):
|
||||
|
||||
```bash
|
||||
xcodebuild ... CODE_SIGNING_ALLOWED=NO CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY="" build
|
||||
cp <profile>.mobileprovision "$APP/embedded.mobileprovision"
|
||||
# sign nested frameworks/dylibs first, then the bundle, then:
|
||||
xcrun devicectl device install app --device <udid> "$APP"
|
||||
```
|
||||
|
||||
This is a stopgap, not a fix: `expo prebuild` regenerates `packages/app/ios/` and discards it. It also
|
||||
requires stripping entitlements the cached profile lacks (e.g. `aps-environment`), so push is dead in such a
|
||||
build.
|
||||
|
||||
To verify only that native Swift compiles, skip signing entirely — the pods have their own schemes:
|
||||
|
||||
```bash
|
||||
cd packages/app/ios
|
||||
xcodebuild -workspace PaseoDebug.xcworkspace -scheme ExpoTwoWayAudio \
|
||||
-sdk iphoneos -destination 'generic/platform=iOS' CODE_SIGNING_ALLOWED=NO build
|
||||
```
|
||||
|
||||
Confirm the log compiled _your_ file and not a cached copy: the `SwiftCompile` line should reference
|
||||
`packages/expo-two-way-audio/ios/...`, not a path inside DerivedData.
|
||||
|
||||
### Headless launch cannot verify runtime behavior
|
||||
|
||||
`xcrun devicectl device process launch` reports `The app terminated with the exit code 0` about a second
|
||||
after launch when the phone's screen is off — the app is suspended and killed before the JS bundle loads.
|
||||
This is **not** a build defect. Confirm with a control: launch `com.apple.Preferences` the same way and
|
||||
watch it exit identically. Headless install and launch prove the binary is valid and that startup reaches
|
||||
RN init; anything about on-screen behavior needs a human holding the phone.
|
||||
|
||||
## iOS Simulator
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user