* Release the audio session when voice capture and playback go idle Opening Paseo permanently paused music playing in other apps. Once the mic had been used a single time, every subsequent foreground of the app killed the user's audio again until the app was force-quit. iOS AudioEngine.setupAudioSession() takes .playAndRecord/.voiceChat — a non-mixing category — and calls setActive(true), but the only setActive(false) was reachable from tearDown(). Nothing handed the session back after a normal dictation turn, so iOS re-asserted the last category the app set on every foreground. The iOS module had no app-lifecycle handling at all, while Android already had a lifecycle listener; that asymmetry is what made the platforms diverge. Android had the same leak in a different shape: AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE was requested per recording and never abandoned. Add releaseAudioSession() to both platforms and call it once capture stops and the playback queue drains. iOS pauses the engine, calls setActive(false, .notifyOthersOnDeactivation) — this is what tells the other app to resume — then drops to .ambient/.mixWithOthers so any implicit reactivation mixes instead of interrupting. The session is re-acquired lazily on toggleRecording(true)/playPCMData, so voice keeps working; resumeRecordingAndPlayer() re-activates before restarting the engine, which keeps post-interruption resume intact. Releasing is guarded on isRecording/isPlaying, and checkEngineIsRunning() now refuses to start the engine while the session is released, since starting it would silently re-take the session. The JS wrapper probes the native module before calling, because a JS bundle shipped over EAS Update can land on a binary predating this function. * Correct the iOS signing diagnosis in mobile-testing docs The "No Accounts" error does not mean no Apple ID is configured. The account can be registered in Xcode while its Xcode-Token keychain item is missing, which Xcode reports as No Accounts and then compounds by falling back to a stale cached profile — surfacing as a certificate mismatch that sends you chasing the wrong thing. Document the DVTDeveloperAccountManager log line that names the real cause, the two commands that check what is actually configured, and the build-unsigned-then-sign-manually stopgap. * Reacquire Android audio focus when playback restarts releaseAudioSession() abandons the AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE request and pauses the AudioTrack once capture and playback go idle, but nothing took either back. The next playback turn therefore ran with no focus held, so the assistant's response competed with whatever the user was listening to instead of interrupting it. Add acquireAudioSessionIfNeeded(), the mirror of the iOS activateAudioSessionIfNeeded() already called from playPCMData, and use it from both playPCMData() and resumePlayback(). A null audioFocusRequest is what "we released" looks like, so it doubles as the guard against re-requesting focus mid-turn. Reported by Greptile on #2866. * Make Android playback focus reacquisition non-fatal The previous commit dropped PCM data when focus reacquisition did not return AUDIOFOCUS_REQUEST_GRANTED. That is worse than playing unfocused: playPCMData is fire-and-forget from JS, and playAudio() settles on a duration timer rather than a native ack, so the runtime acknowledged and discarded a chunk that was never queued. It also broke delayed focus. The request is built with setAcceptsDelayedFocusGain(true), so AUDIOFOCUS_REQUEST_DELAYED is a normal outcome, but routing it through handleAudioFocusBlocked() abandoned the pending request and the later grant could never arrive. Request focus, then play regardless of the result, and leave the request outstanding so a delayed grant still lands via the listener. Reported by Greptile on #2866. * Track scheduled buffers so iOS actually releases after playback releaseAudioSession() guarded on !speechPlayer.isPlaying, but AVAudioPlayerNode.isPlaying stays true from play() until an explicit stop()/pause() — draining every scheduled buffer does not clear it. The normal completion path is processQueue() -> releaseSessionIfIdle(), and stopPlayback() is only reached from the explicit stop() API, so after the first spoken response the guard blocked every release and the session stayed active. The music-pausing bug returned in full for anyone using voice responses. Count buffers that are scheduled but not yet rendered, decrementing from the .dataPlayedBack completion handler, and guard on that instead. The handler runs on an AVAudioEngine-internal thread, so the count is behind a lock. Both stop() paths reset it, since stop() discards what is scheduled and a stale count would block every later release. Found by Codex review on #2866. * Stop re-taking Android audio focus on every foreground ExpoTwoWayAudioLifeCycleListener calls resumeRecordingAndPlayer() from onResume, and that requested AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE before looking at isRecordingBeforePause. Foregrounding the app while nothing was recording therefore took exclusive focus back and re-paused the user's music — the same "every foreground kills my music" symptom this change set exists to remove, just reached through the Android lifecycle instead of the iOS category. Return early unless capture or playback was actually live when we backgrounded. Found by Codex review on #2866.
224 lines
7.1 KiB
Swift
224 lines
7.1 KiB
Swift
import ExpoModulesCore
|
|
|
|
let ON_MIC_DATA_EVENT_NAME = "onMicrophoneData"
|
|
let ON_INPUT_VOLUME_LEVEL_EVENT_NAME = "onInputVolumeLevelData"
|
|
let ON_OUTPUT_VOLUME_LEVEL_EVENT_NAME = "onOutputVolumeLevelData"
|
|
let ON_RECORDING_CHANGE_EVENT_NAME = "onRecordingChange"
|
|
let ON_AUDIO_INTERRUPTION_EVENT_NAME = "onAudioInterruption"
|
|
|
|
public class ExpoTwoWayAudioModule: Module {
|
|
private var audioEngine: AudioEngine?
|
|
public func definition() -> ModuleDefinition {
|
|
Name("ExpoTwoWayAudio")
|
|
|
|
OnCreate {
|
|
let permissionsManager = self.appContext?.permissions
|
|
EXPermissionsMethodsDelegate.register(
|
|
[
|
|
MicrophonePermissionRequester()
|
|
],
|
|
withPermissionsManager: permissionsManager
|
|
)
|
|
|
|
}
|
|
|
|
AsyncFunction("initialize") { () -> Bool in
|
|
do {
|
|
if self.audioEngine != nil {
|
|
return true
|
|
}
|
|
self.audioEngine = try AudioEngine()
|
|
self.setupMicrophoneCallback()
|
|
self.setupInputAudioLevelCallback()
|
|
self.setupOutputAudioLevelCallback()
|
|
self.setupAudioInterruptionCallback()
|
|
return true
|
|
} catch {
|
|
print("Failed to initialize AudioEngine: \(error)")
|
|
return false
|
|
}
|
|
}
|
|
|
|
// Safety net for the JS-driven release in `stopCapture`/`processQueue`: never leave
|
|
// the non-mixing `.playAndRecord` session active while backgrounded, or iOS
|
|
// re-asserts it on the next foreground and kills the user's music again.
|
|
OnAppEntersBackground {
|
|
guard self.audioEngine?.isRecording == false else { return }
|
|
self.audioEngine?.releaseAudioSession()
|
|
}
|
|
|
|
Function("releaseAudioSession") {
|
|
self.audioEngine?.releaseAudioSession()
|
|
}
|
|
|
|
Function("isRecording") { () -> Bool in
|
|
guard let audioEngine = self.audioEngine else {
|
|
print("AudioEngine not initialized")
|
|
return false
|
|
}
|
|
return audioEngine.isRecording
|
|
}
|
|
|
|
Function("toggleRecording") { (val: Bool) -> Bool in
|
|
guard let audioEngine = self.audioEngine else {
|
|
print("AudioEngine not initialized")
|
|
return false
|
|
}
|
|
let isRecording = audioEngine.toggleRecording(val)
|
|
self.sendEvent(
|
|
ON_RECORDING_CHANGE_EVENT_NAME,
|
|
[
|
|
"data": isRecording
|
|
])
|
|
return isRecording
|
|
}
|
|
|
|
Function("getMicrophoneModeIOS") { () -> String in
|
|
if #available(iOS 15.0, *) {
|
|
let mode = AVCaptureDevice.preferredMicrophoneMode.rawValue
|
|
var micMode = ""
|
|
|
|
switch mode {
|
|
case 1:
|
|
micMode = MicrophoneMode.wideSpectrum.rawValue
|
|
case 2:
|
|
micMode = MicrophoneMode.voiceIsolation.rawValue
|
|
default:
|
|
micMode = MicrophoneMode.standard.rawValue
|
|
}
|
|
return micMode
|
|
}
|
|
print("Please update your ios")
|
|
|
|
return ""
|
|
}
|
|
|
|
Function("setMicrophoneModeIOS") {
|
|
if #available(iOS 15, *) {
|
|
if AVCaptureDevice.preferredMicrophoneMode != .voiceIsolation {
|
|
AVCaptureDevice.showSystemUserInterface(.microphoneModes)
|
|
}
|
|
} else {
|
|
print("This code only runs on iOS 14 and lower")
|
|
}
|
|
}
|
|
|
|
Function("tearDown") {
|
|
self.audioEngine?.tearDown()
|
|
self.audioEngine = nil
|
|
}
|
|
|
|
Function("restart") {
|
|
self.audioEngine?.resumeRecordingAndPlayer()
|
|
self.sendEvent(
|
|
ON_RECORDING_CHANGE_EVENT_NAME,
|
|
[
|
|
"data": audioEngine?.isRecording
|
|
])
|
|
|
|
}
|
|
|
|
Function("playPCMData") { (pcmData: Data) in
|
|
self.audioEngine?.playPCMData(pcmData)
|
|
}
|
|
|
|
Function("bypassVoiceProcessing") { (bypass: Bool) in
|
|
self.audioEngine?.bypassVoiceProcessing(bypass)
|
|
}
|
|
|
|
Function("isPlaying") { () -> Bool in
|
|
return self.audioEngine?.isPlaying ?? false
|
|
}
|
|
|
|
Function("stopPlayback") {
|
|
self.audioEngine?.stopPlayback()
|
|
}
|
|
|
|
Function("pausePlayback") {
|
|
self.audioEngine?.pausePlayback()
|
|
}
|
|
|
|
Function("resumePlayback") {
|
|
self.audioEngine?.resumePlayback()
|
|
}
|
|
|
|
AsyncFunction("getMicrophonePermissionsAsync") { (promise: Promise) in
|
|
EXPermissionsMethodsDelegate.getPermissionWithPermissionsManager(
|
|
self.appContext?.permissions,
|
|
withRequester: MicrophonePermissionRequester.self,
|
|
resolve: promise.resolver,
|
|
reject: promise.legacyRejecter
|
|
)
|
|
}
|
|
|
|
AsyncFunction("requestMicrophonePermissionsAsync") { (promise: Promise) in
|
|
EXPermissionsMethodsDelegate.askForPermission(
|
|
withPermissionsManager: self.appContext?.permissions,
|
|
withRequester: MicrophonePermissionRequester.self,
|
|
resolve: promise.resolver,
|
|
reject: promise.legacyRejecter
|
|
)
|
|
}
|
|
|
|
// Define the events that can be emitted
|
|
Events([
|
|
ON_MIC_DATA_EVENT_NAME,
|
|
ON_INPUT_VOLUME_LEVEL_EVENT_NAME,
|
|
ON_OUTPUT_VOLUME_LEVEL_EVENT_NAME,
|
|
ON_RECORDING_CHANGE_EVENT_NAME,
|
|
ON_AUDIO_INTERRUPTION_EVENT_NAME,
|
|
])
|
|
}
|
|
|
|
private func setupMicrophoneCallback() {
|
|
audioEngine?.onMicDataCallback = { [weak self] data in
|
|
self?.sendEvent(
|
|
ON_MIC_DATA_EVENT_NAME,
|
|
[
|
|
"data": data
|
|
])
|
|
}
|
|
}
|
|
|
|
private func setupInputAudioLevelCallback() {
|
|
audioEngine?.onInputVolumeCallback = { [weak self] level in
|
|
self?.sendEvent(
|
|
ON_INPUT_VOLUME_LEVEL_EVENT_NAME,
|
|
[
|
|
"data": level
|
|
])
|
|
}
|
|
}
|
|
|
|
private func setupOutputAudioLevelCallback() {
|
|
audioEngine?.onOutputVolumeCallback = { [weak self] level in
|
|
self?.sendEvent(
|
|
ON_OUTPUT_VOLUME_LEVEL_EVENT_NAME,
|
|
[
|
|
"data": level
|
|
])
|
|
}
|
|
}
|
|
|
|
private func setupAudioInterruptionCallback() {
|
|
audioEngine?.onAudioInterruptionCallback = { [weak self] data in
|
|
self?.sendEvent(
|
|
ON_AUDIO_INTERRUPTION_EVENT_NAME,
|
|
[
|
|
"data": data
|
|
])
|
|
self?.sendEvent(
|
|
ON_RECORDING_CHANGE_EVENT_NAME,
|
|
[
|
|
"data": self?.audioEngine?.isRecording
|
|
])
|
|
}
|
|
}
|
|
|
|
enum MicrophoneMode: String, Enumerable {
|
|
case standard
|
|
case voiceIsolation
|
|
case wideSpectrum
|
|
}
|
|
}
|