Informal Notes on AVAudioSession

Audio Session (AVAudioSession) Overview

An audio session (AVAudioSession) acts as an interface between your app and the operating system (excluding macOS, including iOS, iPadOS, tvOS, visionOS, watchOS) for audio usage.

Key Features of Audio Session

1. Determine how audio is used (behaves) in the app
   • Setting categories, modes, options, and route sharing policies
2. Activating/deactivating the audio session
3. Responding to audio events
   • Audio interruption events, reset of audio/video services
4. Up to iOS 16: recording-related tasks, recording permission requests, etc. (From iOS 17 onward, use AVAudioApplication.recordPermission)
5. Configuring audio hardware
   • Selecting audio input/output
   • Setting sample rate and I/O buffer duration

When simply outputting audio (e.g., playing audio/video), the default behavior often requires no action. Especially when using AVFoundation (AVFAudio) classes, implementing steps 1 and 3 is usually sufficient and straightforward.
Here, I will describe aspects other than "5. Configuring audio hardware."

Environment

• Target OS: iOS 17+ (largely applicable to iPadOS, tvOS, visionOS, watchOS)
• Test device: iOS 26
• Xcode 26

1. Determine How Audio Is Used in the App

Based on the app's audio use case, configure:

• Category (AVAudioSession.Category)
• Mode (AVAudioSession.Mode)
• Category Options (AVAudioSession.CategoryOptions)
• Route Sharing Policy (AVAudioSession.RouteSharingPolicy)

Category (AVAudioSession.Category)

The category defines the broad behavior of the audio session.
For example, the category determines:

• Playback only or recording capable
• Whether to mix with other apps' audio
• Whether to mute with the silent switch or screen lock
• Whether background playback is possible

Category, Behavior, and Use Cases

There are six categories, with the following behaviors when set.
The default category is soloAmbient, which is used if not set.

Category	Behavior	Use Case
ambient	- Mixes with other audio - Muted by silent switch - No background playback	Game sound effects
multiRoute	- Routes different audio data streams to different output devices simultaneously - Stops other audio - Not muted by silent switch - Background playback enabled	DJ app (?)
playAndRecord	- Stops other audio - Not muted by silent switch - Background playback enabled	Recording/Playback
playback	- Stops other audio - Not muted by silent switch - Background playback enabled	Music playback
record	- Stops other audio	Recording
soloAmbient (default)	- Stops other audio - Muted by silent switch - No background playback	General

*For background playback, you need an appropriate category and a plist setting to enable it.
For plist settings, see Configuring your app for media playback.

Mode (AVAudioSession.Mode)

The mode adjusts the app's sound settings and optimizes audio features for specific purposes.

Mode, Purpose, and Use Cases

Mode	Purpose / Use Case
default[1]	Default
gameChat[2]	For GameKit
measurement	For measurement
moviePlayback	For movie playback
shortFormVideo[3]	For short-form videos
spokenAudio	For podcasts, audiobooks
videoChat	For video chat
videoRecording	For video recording
voiceChat	For voice chat
voicePrompt	For voice prompts (TextToSpeech, navigation)

Category Options (AVAudioSession.CategoryOptions)

Category options allow additional customization of the category's behavior.
Setting an option enables something that is disabled by default. It cannot disable something; if needed, choose a different category.
Which options are disabled depends on the category.
For example, in the ambient category, mixWithOthers is enabled by default, but in the playback category it is disabled by default and can be enabled by specifying the option. Also, some options automatically enable others.
For example, setting duckOthers automatically also enables mixWithOthers.

Category Option, Description, and Behavior When Set

Category Option	Description / Behavior
default	Default
allowAirPlay	Enable AirPlay
allowBluetoothHFP	Enable Bluetooth HFP (Hands-Free Profile)
allowBluetoothA2DP	Enable Bluetooth A2DP
bluetoothHighQualityRecording	Enable full-bandwidth audio when Bluetooth route is supported (e.g., certain AirPods models)
defaultToSpeaker	Route audio to the bottom speaker by default instead of the receiver (top speaker)
duckOthers	Reduce the volume of other apps (ducking) while playing audio from your app
interruptSpokenAudioAndMixWithOthers	Spoken audio is interrupted; other audio mixes
mixWithOthers	Mix with other audio
overrideMutedMicrophoneInterruption	Prevent the audio session from being interrupted when the system mutes the built-in microphone

Route Sharing Policy (AVAudioSession.RouteSharingPolicy)

Route sharing policy routes audio output to something other than the default system output when available.
Only the playback category can set this policy, with longFormAudio and longFormVideo policies available.
Setting it for other categories will result in an error.

Route Sharing Policy and Behavior When Set

Route Sharing Policy	Behavior When Set
default[4]	Default policy
independent	System-managed policy for video wirelessly routed via route picker UI (cannot be specified)
longFormAudio	Policy to share routes with other music/podcast apps
longFormVideo	Policy to share routes with other video apps

Relationship Between Settable Categories and Modes

A table of settable category-mode combinations.
categoryOptions and routeSharingPolicy are default.
gameChat is omitted because it cannot be set manually.

category / mode	measurement	moviePlayback	shortFormVideo	spokenAudio
ambient				✔︎
multiRoute	✔︎			✔︎
playAndRecord	✔︎			✔︎
playback	✔︎	✔︎	✔︎	✔︎
record	✔︎
soloAmbient				✔︎

category / mode	videoChat	videoRecording	voiceChat	voicePrompt
ambient
multiRoute
playAndRecord	✔︎	✔︎	✔︎
playback				✔︎
record		✔︎
soloAmbient

*The online documentation for videoChat states "Use this mode for video chat apps that use the playAndRecord or record categories," but actually videoChat cannot be set with the record category (the source code comments correctly describe this).
*Note that specifying an invalid category-mode pair in the simulator does not produce an error.

Relationship Between Settable Categories and Category Options

A table of settable category-option combinations (mode and routeSharingPolicy are default).

category / option	allowAirPlay	allowBluetoothHFP	allowBluetoothA2DP
ambient
multiRoute
playAndRecord	✔︎		✔︎
playback
record
soloAmbient

category / option	bluetoothHighQualityRecording	defaultToSpeaker	duckOthers
ambient		️	✔︎
multiRoute	✔︎	✔︎	✔︎
playAndRecord	✔︎	✔︎	✔︎
playback		︎	✔︎
record	✔︎		✔︎
soloAmbient			✔︎

category / option	interruptSpokenAudioAndMixWithOthers	mixWithOthers	overrideMutedMicrophoneInterruption
ambient	✔︎	️ ✔︎
multiRoute	✔︎	✔︎	✔︎
playAndRecord	✔︎	✔︎	✔︎
playback	✔︎	✔︎	✔︎
record	✔︎	✔︎	✔︎
soloAmbient	✔︎	✔︎	✔︎

Configuring Category, Mode, Category Options, and Route Sharing Policy

Configuring the category, mode, category options, and route sharing policy is the most important part of using audio sessions.
To set them, use methods like AVAudioSession.setCategory(_, mode:, policy:, options:).

To set the playAndRecord category:

do {
  try AVAudioSession.sharedInstance().setCategory(.playAndRecord)
} catch {
  // handle error: unable to set category
}

Especially when recording, error handling is essential because input may be disabled if the category is not set correctly. For audio output only, setting the category incorrectly won't prevent output, so error handling is desirable but ignoring it won't cause a problem like no sound. Error codes range from obvious setup errors like AVAudioSession.ErrorCode.badParam (e.g., invalid category-mode pair) to vague errors like .unspecified. It is difficult to handle each case in detail (and whether the error content itself is reliable is questionable).

How long the configured values remain effective is not documented. To ensure they are set, it is safer to defensively check and set them just before starting recording/playback (especially for recording) rather than setting them only once at app launch.

// Defensively check and set before recording/playback
// At least check the category
let session = AVAudioSession.sharedInstance()
if session.category != expectedCategory || session.mode != expectedMode || session.options != expectedOptions || session.routeSharingPolicy != expectedRouteSharingPolicy {
    try set audio session category, mode, options, route sharing
}
// If the category etc. are set, start recording/playback
// (i.e., try recorder.record() or try player.play())

2. Activating/Deactivating the Audio Session

When the audio session is activated, the behavior configured with the category etc. takes effect, and audio can be used (active state). If mixing is not enabled, even if audio is not used immediately, background audio will be stopped. The documentation recommends activating the session just before using audio. While audio is being used, the session remains active; in the inactive state, audio cannot be used.

Explicit activation/deactivation of the audio session is almost unnecessary.
When using Audio Unit or Audio Queue Services, you need to explicitly activate the session (natural since it's a C API).
However, when using AVFoundation (AVFAudio) classes, explicit activation is not required.
The Audio Session Programming Guide states the following about activation:

Although the AVFoundation playback and recording classes automatically activate your audio session, manually activating it gives you an opportunity to test whether activation succeeded.

Translated:

AVFoundation playback and recording classes automatically activate the audio session, but manually activating it gives you an opportunity to test whether activation succeeded.

Since it says AVFoundation playback and recording classes automatically activate the audio session, there is no need to explicitly activate the audio session when using classes within AVFoundation (AVFAudio). (What use case would require testing whether activation succeeded? The purpose is unclear.)

AVAudioEngine.prepare/start source code comments also state "implicitly activates the audio session" (though not in the online documentation).

this method may cause the audio session to be implicitly activated. Activating the audio session (implicitly or explicitly) may cause other audio sessions to be interrupted or ducked depending on the session's configuration. It is recommended to configure and activate the app's audio session before preparing the engine.
See Audio Session Programming Guide for details.

When using AVFoundation (AVFAudio) classes like AVAudioPlayer:

// Configure category etc.
let player = try AVAudioPlayer(...) // Initialize AVAudioPlayer
// Audio session activation is done by AVAudioPlayer
let success = player.play()
if !success {
// This error should also occur if session activation fails (but error details cannot be obtained from AVAudioPlayer)
}

To explicitly activate/deactivate the audio session, do the following:

try AVAudioSession.sharedInstance().setActive(true or false)

You may deactivate the session when it is no longer needed, but generally deactivation is not required because the audio session is taken over by the user. A beneficial use case for deactivation is when you want to restore ducked audio.

3. Responding to System Audio Events

When responding to audio events, the two main events are:

• AVAudioSession.Interruption (Interruption)
• AVAudioSession.mediaServicesWereResetNotification (Media services reset)

*These notifications are only sent when the audio session is active (conversely, receiving them indicates the session is active).
*Higher-level classes like AVPlayer may only need to monitor AVPlayer.timeControlStatus or AVPlayer.rate. Check documentation to see if implementation is necessary.

AVAudioSession.Interruption

This notification is sent when another app activates/deactivates its audio session.
To implement, register for the notification AVAudioSession.interruptionNotification as per documentation, and update UI / save and restore state at interruption start/end.

// Register for notification in an appropriate class
NotificationCenter.default.addObserver(
    forName: AVAudioSession.interruptionNotification,
    object: nil,
    queue: .main,
    using: { [weak self] notification in
        // Assuming self is MainActor
        MainActor.assumeIsolated {
            self?.handleInterruption(notification)
        }
    })
// Handler
func handleInterruption(_ notification: Notification) {
    guard let userInfo = notification.userInfo,
          let typeValue = userInfo[AVAudioSessionInterruptionTypeKey] as? UInt,
          let type = AVAudioSession.InterruptionType(rawValue: typeValue) else {
        return
    }

    switch type {
    // Interruption began
    case .began:
        // Update UI (pause state) and save state
        // * At this point, the audio session has been deactivated by the system, so be careful if you need to manually activate it (another app activated its session, so this app's session is deactivated)
    // Interruption ended
    // * Note that end may not always be notified
    case .ended:
        // When resumable, the .shouldResume flag is set
        guard let optionsValue = userInfo[AVAudioSessionInterruptionOptionKey] as? UInt else { return }
        let options = AVAudioSession.InterruptionOptions(rawValue: optionsValue)
        if options.contains(.shouldResume) {
            // Update UI and restore state
        }
    @unknown default:
        break
    }
}

When the output changes (e.g., headphones unplugged), prior to iOS 16 this was handled via AVAudioSession.RouteChange, but from iOS 17 onward, it is notified via Interruption for Now Playing sessions^[5]. Setting AVAudioSession.sharedInstance().setPrefersInterruptionOnRouteDisconnect(true) will also cause Interruption to be notified for non-Now Playing sessions.

AVAudioSession.mediaServicesWereResetNotification

This notification is sent when the media services (system-level audio/video services) are reset.
The documentation says it happens rarely, but it does happen, so implementation is essential.

To monitor media service resets, register for AVAudioSession.mediaServicesWereResetNotification.

// Register for notification in an appropriate class
NotificationCenter.default.addObserver(
    forName: AVAudioSession.mediaServicesWereResetNotification,
    object: nil,
    queue: .main,
    using: { [weak self] notification in
        // Assuming self is MainActor
        MainActor.assumeIsolated {
            self?.handleMediaServicesWereReset(notification)
        }
    })
// Handler
func handleMediaServicesWereReset(_ notification: Notification) {
    // Reset (reinitialize) audio/video objects
    // Reconfigure audio session category, etc.
}

AVAudioSession.RouteChange (iOS 16 and earlier, etc.)

Route change is notified when audio input/output routes change, such as when headphones are plugged/unplugged. It is also notified when the category changes from playback to playbackAndRecord (adding input) or when input is removed.
To monitor, register for AVAudioSession.routeChangeNotification.

From iOS 17 onward, if you do not set AudioSession.setPrefersInterruptionOnRouteDisconnect(true), you may still need to implement this notification to stop playback when headphones are unplugged, but it is more convenient to implement via Interruption. From iOS 17 onward, there are fewer use cases for this notification (e.g., displaying the current input/output route).

An implementation to stop playback when headphones are unplugged:

// Register for notification in an appropriate class
NotificationCenter.default.addObserver(
    forName: AVAudioSession.routeChangeNotification,
    object: nil,
    queue: .main,
    using: { [weak self] notification in
        // Assuming self is MainActor
        MainActor.assumeIsolated {
            self?.handleRouteChange(notification)
        }
    })
// Handler
func handleRouteChange(_ notification: Notification) {
    guard let userInfo = notification.userInfo,
          let reasonValue = userInfo[AVAudioSessionRouteChangeReasonKey] as? UInt,
          let reason = AVAudioSession.RouteChangeReason(rawValue: reasonValue) else {
        return
    }

    switch reason {
        case .oldDeviceUnavailable:
            // If playing, stop playback, update UI (pause state), save state
        default:
            break
    }
}

4. Requesting Recording Permission

When recording, you need to implement permission requests. Specifically, do the following two things:

1. Add a Microphone Usage Description (NSMicrophoneUsageDescription) to the plist
2. Request microphone access permission if not already requested

Implementing Recording Permission Request for iOS 17+

let permission = AVAudioApplication.shared.recordPermission
if permission == .undetermined {
    Task {
        let granted = await AVAudioApplication.requestRecordPermission()
        if granted {
            // microphone is available, false otherwise
        }
    }
}

Implementing Recording Permission Request for iOS 16 and Earlier

let permission = AVAudioSession.sharedInstance().recordPermission
if permission == .undetermined {
    AVAudioSession.sharedInstance().requestRecordPermission { granted in
        if granted {
            // microphone is available, false otherwise
        }
    }
}

Other Notes

• The simulator may not produce errors or behave differently, so test on a real device
• Thread safety for category settings etc. is not documented, so executing on the main thread is safer
• If there is no sound, first check that the device is not muted and the silent switch is not on

Summary

AVAudioSession is a core component that governs audio behavior on Apple platforms (excluding macOS) including iOS. The patterns for configuration and implementation are not numerous, but many parts are implicitly handled when using AVFoundation (AVFAudio) classes.
Refer to documentation and source code comments as needed to confirm behavior.

References

• Audio Session Programming Guide
• AVAudioSession
• AVAudioSession.ErrorCode
• Configuring your app for media playback
• Technical Q&A QA1749 AVAudioSession - General recommendations for handling AVAudioSessionMediaServicesWereResetNotification
• Technical Q&A QA1631 AVAudioSession - Requesting Audio Session Preferences

脚注

1. No clear definition. It's natural to consider behavior not specified by other modes as default.

2. Cannot be set manually; GameKit sets it.

3. Available from iOS 26.

4. No clear definition. Behavior other than the specifiable policies is the default (per-app independent routing, last-in-wins rule).

5. The exact meaning of "Now Playing session" is unclear; it is a term that suddenly appears in setPrefersInterruptionOnRouteDisconnect. Possibly refers to something that supports Now Playing info and background playback?