Enum fidl_fuchsia_media::AudioCapturerRequest

pub enum AudioCapturerRequest {
    AddPayloadBuffer {
        id: u32,
        payload_buffer: Vmo,
        control_handle: AudioCapturerControlHandle,
    },
    RemovePayloadBuffer {
        id: u32,
        control_handle: AudioCapturerControlHandle,
    },
    ReleasePacket {
        packet: StreamPacket,
        control_handle: AudioCapturerControlHandle,
    },
    DiscardAllPackets {
        responder: AudioCapturerDiscardAllPacketsResponder,
    },
    DiscardAllPacketsNoReply {
        control_handle: AudioCapturerControlHandle,
    },
    SetPcmStreamType {
        stream_type: AudioStreamType,
        control_handle: AudioCapturerControlHandle,
    },
    CaptureAt {
        payload_buffer_id: u32,
        payload_offset: u32,
        frames: u32,
        responder: AudioCapturerCaptureAtResponder,
    },
    StartAsyncCapture {
        frames_per_packet: u32,
        control_handle: AudioCapturerControlHandle,
    },
    StopAsyncCapture {
        responder: AudioCapturerStopAsyncCaptureResponder,
    },
    StopAsyncCaptureNoReply {
        control_handle: AudioCapturerControlHandle,
    },
    BindGainControl {
        gain_control_request: ServerEnd<GainControlMarker>,
        control_handle: AudioCapturerControlHandle,
    },
    GetReferenceClock {
        responder: AudioCapturerGetReferenceClockResponder,
    },
    SetReferenceClock {
        reference_clock: Option<Clock>,
        control_handle: AudioCapturerControlHandle,
    },
    SetUsage {
        usage: AudioCaptureUsage,
        control_handle: AudioCapturerControlHandle,
    },
    GetStreamType {
        responder: AudioCapturerGetStreamTypeResponder,
    },
}

AudioCapturer

An AudioCapturer is an interface returned from an fuchsia.media.Audio’s CreateAudioCapturer method, which may be used by clients to capture audio from either the current default audio input device, or the current default audio output device depending on the flags passed during creation.

Format support

See (Get|Set)StreamType below. By default, the captured stream type will be initially determined by the currently configured stream type of the source that the AudioCapturer was bound to at creation time. Users may either fetch this type using GetStreamType, or they may choose to have the media resampled or converted to a type of their choosing by calling SetStreamType. Note: the stream type may only be set while the system is not running, meaning that there are no pending capture regions (specified using CaptureAt) and that the system is not currently running in ‘async’ capture mode.

Buffers and memory management

Audio data is captured into a shared memory buffer (a VMO) supplied by the user to the AudioCapturer during the AddPayloadBuffer call. Please note the following requirements related to the management of the payload buffer.

• The payload buffer must be supplied before any capture operation may start. Any attempt to start capture (via either CaptureAt or StartAsyncCapture) before a payload buffer has been established is an error.
• The payload buffer may not be changed while there are any capture operations pending.
• The stream type may not be changed after the payload buffer has been set.
• The payload buffer must be an integral number of audio frame sizes (in bytes)
• When running in ‘async’ mode (see below), the payload buffer must be at least as large as twice the frames_per_packet size specified during StartAsyncCapture.
• The handle to the payload buffer supplied by the user must be readable, writable, mappable and transferable.
• Users should always treat the payload buffer as read-only.

Synchronous vs. Asynchronous capture mode

The AudioCapturer interface can be used in one of two mutually exclusive modes: Synchronous and Asynchronous. A description of each mode and their tradeoffs is given below.

Synchronous mode

By default, AudioCapturer instances are running in ‘sync’ mode. They will only capture data when a user supplies at least one region to capture into using the CaptureAt method. Regions supplied in this way will be filled in the order that they are received and returned to the client as StreamPackets via the return value of the CaptureAt method. If an AudioCapturer instance has data to capture, but no place to put it (because there are no more pending regions to fill), the next payload generated will indicate that their has been an overflow by setting the Discontinuity flag on the next produced StreamPacket. Synchronous mode may not be used in conjunction with Asynchronous mode. It is an error to attempt to call StartAsyncCapture while the system still regions supplied by CaptureAt waiting to be filled.

If a user has supplied regions to be filled by the AudioCapturer instance in the past, but wishes to reclaim those regions, they may do so using the DiscardAllPackets method. Calling the DiscardAllPackets method will cause all pending regions to be returned, but with NO_TIMESTAMP as their StreamPacket’s PTS. See “Timing and Overflows”, below, for a discussion of timestamps and discontinuity flags. After a DiscardAllPackets operation, an OnEndOfStream event will be produced. While an AudioCapturer will never overwrite any region of the payload buffer after a completed region is returned, it may overwrite the unfilled portions of a partially filled buffer which has been returned as a result of a DiscardAllPackets operation.

Asynchronous mode

While running in ‘async’ mode, clients do not need to explicitly supply shared buffer regions to be filled by the AudioCapturer instance. Instead, a client enters into ‘async’ mode by calling StartAsyncCapture and supplying a callback interface and the number of frames to capture per-callback. Once running in async mode, the AudioCapturer instance will identify which payload buffer regions to capture into, capture the specified number of frames, then deliver those frames as StreamPackets using the OnPacketCapture FIDL event. Users may stop capturing and return the AudioCapturer instance to ‘sync’ mode using the StopAsyncCapture method.

It is considered an error to attempt any of the following operations.

• To attempt to enter ‘async’ capture mode when no payload buffer has been established.
• To specify a number of frames to capture per payload which does not permit at least two contiguous capture payloads to exist in the established shared payload buffer simultaneously.
• To send a region to capture into using the CaptureAt method while the AudioCapturer instance is running in ‘async’ mode.
• To attempt to call DiscardAllPackets while the AudioCapturer instance is running in ‘async’ mode.
• To attempt to re-start ‘async’ mode capturing without having first stopped.
• To attempt any operation except for SetGain while in the process of stopping.

Synchronizing with a StopAsyncCapture operation

Stopping asynchronous capture mode and returning to synchronous capture mode is an operation which takes time. Aside from SetGain, users may not call any other methods on the AudioCapturer interface after calling StopAsyncCapture (including calling StopAsyncCapture again) until after the stop operation has completed. Because of this, it is important for users to be able to synchronize with the stop operation. Two mechanisms are provided for doing so.

The first is to use StopAsyncCapture (not the NoReply variant). When the user’s callback has been called, they can be certain that stop operation is complete and that the AudioCapturer instance has returned to synchronous operation mode.

The second way to determine that a stop operation has completed is to use the flags on the packets which get delivered via the user-supplied AudioCapturerCallback interface after calling StopAsyncCapture. When asked to stop, any partially filled packet will be returned to the user, and the final packet returned will always have the end-of-stream flag (kFlagsEos) set on it to indicate that this is the final frame in the sequence. If there is no partially filled packet to return, the AudioCapturer will synthesize an empty packet with no timestamp, and offset/length set to zero, in order to deliver a packet with the end-of-stream flag set on it. Once users have seen the end-of-stream flag after calling stop, the AudioCapturer has finished the stop operation and returned to synchronous operating mode.

Timing and Overflows

All media packets produced by an AudioCapturer instance will have their PTS field filled out with the capture time of the audio expressed as a timestamp given by the reference clock timeline. Note: this timestamp is actually a capture timestamp, not a presentation timestamp (it is more of a CTS than a PTS) and is meant to represent the underlying system’s best estimate of the capture time of the first frame of audio, including all outboard and hardware introduced buffering delay. As a result, all timestamps produced by an AudioCapturer should be expected to be in the past relative to ‘now’ on the stream’s reference clock timeline.

The one exception to the “everything has an explicit timestamp” rule is when discarding submitted regions while operating in synchronous mode. Discarded packets have no data in them, but FIDL demands that all pending method-return-value callbacks be executed. Because of this, the regions will be returned to the user, but their timestamps will be set to NO_TIMESTAMP, and their payload sizes will be set to zero. Any partially filled payload will have a valid timestamp, but a payload size smaller than originally requested. The final discarded payload (if there were any to discard) will be followed by an OnEndOfStream event.

Two StreamPackets delivered by an AudioCapturer instance are ‘continuous’ if the first frame of audio contained in the second packet was captured exactly one nominal frame time after the final frame of audio in the first packet. If this relationship does not hold, the second StreamPacket will have the STREAM_PACKET_FLAG_DISCONTINUITY bit set in its flags field.

Even though explicit timestamps are provided on every StreamPacket produced, users who have very precise timing requirements are encouraged to always reason about time by counting frames delivered since the last discontinuity, rather than simply using the raw capture timestamps. This is because the explicit timestamps written on continuous packets may have a small amount of rounding error based on whether or not the units of the capture timeline reference clock are divisible by the chosen audio frame rate.

Users should always expect the first StreamPacket produced by an AudioCapturer to have the discontinuous flag set on it (as there is no previous packet to be continuous with). Similarly, the first StreamPacket after a DiscardAllPackets or a Stop/Start cycle will always be discontinuous. After that, there are only two reasons that a StreamPacket will ever be discontinuous:

1. The user is operating in synchronous mode and does not supply regions to be filled quickly enough. If the next continuous frame of data has not been captured by the time it needs to be purged from the source buffers, an overflow has occurred and the AudioCapturer will flag the next captured region as discontinuous.
2. The user is operating in asynchronous mode and some internal error prevents the AudioCapturer instance from capturing the next frame of audio in a continuous fashion. This might be high system load or a hardware error, but in general it is something which should never normally happen. In practice, however, if it does, the next produced packet will be flagged as being discontinuous.

Synchronous vs. Asynchronous Trade-offs

The choice of operating in synchronous vs. asynchronous mode is up to the user, and depending on the user’s requirements, there are some advantages and disadvantages to each choice.

Synchronous mode requires only a single Zircon channel under the hood and can achieve some small savings because of this. In addition, the user has complete control over the buffer management. Users specify exactly where audio will be captured to and in what order. Because of this, if users do not need to always be capturing, it is simple to stop and restart the capture later (just by ceasing to supply packets, then resuming later on). Payloads do not need to be uniform in size either, clients may specify payloads of whatever granularity is appropriate.

The primary downside of operating in synchronous mode is that two messages will need to be sent for every packet to be captured. One to inform the AudioCapturer of the instance to capture into, and one to inform the user that the packet has been captured. This may end up increasing overhead and potentially complicating client designs.

Asynchronous mode has the advantage requiring only 1/2 of the messages, however, when operating in ‘async’ mode, AudioCapturer instances have no way of knowing if a user is processing the StreamPackets being sent in a timely fashion, and no way of automatically detecting an overflow condition. Users of ‘async’ mode should be careful to use a buffer large enough to ensure that they will be able to process their data before an AudioCapturer will be forced to overwrite it.

Variants

AddPayloadBuffer

Adds a payload buffer to the current buffer set associated with the connection. A StreamPacket struct reference a payload buffer in the current set by ID using the StreamPacket.payload_buffer_id field.

A buffer with ID id must not be in the current set when this method is invoked, otherwise the service will close the connection.

Fields

id: u32

payload_buffer: Vmo

control_handle: AudioCapturerControlHandle

RemovePayloadBuffer

Removes a payload buffer from the current buffer set associated with the connection.

A buffer with ID id must exist in the current set when this method is invoked, otherwise the service will will close the connection.

Fields

id: u32

control_handle: AudioCapturerControlHandle

ReleasePacket

Releases payload memory associated with a packet previously delivered via OnPacketProduced.

Fields

packet: StreamPacket

control_handle: AudioCapturerControlHandle

DiscardAllPackets

Fields

responder: AudioCapturerDiscardAllPacketsResponder

DiscardAllPacketsNoReply

Fields

control_handle: AudioCapturerControlHandle

SetPcmStreamType

Sets the stream type of the stream to be delivered. Causes the source material to be reformatted/resampled if needed in order to produce the requested stream type. Must be called before the payload buffer is established.

Fields

stream_type: AudioStreamType

control_handle: AudioCapturerControlHandle

CaptureAt

Explicitly specifies a region of the shared payload buffer for the audio input to capture into.

Fields

payload_buffer_id: u32

payload_offset: u32

frames: u32

responder: AudioCapturerCaptureAtResponder

StartAsyncCapture

Places the AudioCapturer into ‘async’ capture mode and begin to produce packets of exactly ‘frames_per_packet’ number of frames each. The OnPacketProduced event (of StreamSink) will be used to inform the client of produced packets.

Fields

frames_per_packet: u32

control_handle: AudioCapturerControlHandle

StopAsyncCapture

Stops capturing in ‘async’ capture mode and (optionally) deliver a callback that may be used by the client if explicit synchronization is needed.

Fields

responder: AudioCapturerStopAsyncCaptureResponder

StopAsyncCaptureNoReply

Fields

control_handle: AudioCapturerControlHandle

BindGainControl

Binds to the gain control for this AudioCapturer.

Fields

gain_control_request: ServerEnd<GainControlMarker>

control_handle: AudioCapturerControlHandle

GetReferenceClock

Retrieves the stream’s reference clock. The returned handle will have READ, DUPLICATE and TRANSFER rights, and will refer to a zx::clock that is MONOTONIC and CONTINUOUS.

Fields

responder: AudioCapturerGetReferenceClockResponder

SetReferenceClock

Sets the reference clock that controls this capturer’s playback rate. If the input parameter is a valid zx::clock, it must have READ, DUPLICATE, TRANSFER rights and refer to a clock that is both MONOTONIC and CONTINUOUS. If instead an invalid clock is passed (such as the uninitialized zx::clock()), this indicates that the stream will use a ‘flexible’ clock generated by AudioCore that tracks the audio device.

SetReferenceClock cannot be called after the capturer payload buffer has been added. It also cannot be called a second time (even before capture). If the client wants a reference clock that is initially CLOCK_MONOTONIC but may diverge at some later time, they should create a clone of the monotonic clock, set this as the stream’s reference clock, then rate-adjust it subsequently as needed.

Fields

reference_clock: Option<Clock>

control_handle: AudioCapturerControlHandle

SetUsage

Sets the usage of the capture stream. This may be changed on the fly, but packets in flight may be affected by the new usage. By default the Capturer is created with the FOREGROUND usage.

Fields

usage: AudioCaptureUsage

control_handle: AudioCapturerControlHandle

GetStreamType

Gets the currently configured stream type. Note: for an AudioCapturer which was just created and has not yet had its stream type explicitly set, this will retrieve the stream type – at the time the AudioCapturer was created – of the source (input or looped-back output) to which the AudioCapturer is bound. Even if this matches the client’s desired format, SetPcmStreamType must still be called.

Fields

responder: AudioCapturerGetStreamTypeResponder

Implementations

impl AudioCapturerRequest

pub fn into_add_payload_buffer( self ) -> Option<(u32, Vmo, AudioCapturerControlHandle)>

pub fn into_remove_payload_buffer( self ) -> Option<(u32, AudioCapturerControlHandle)>

pub fn into_release_packet( self ) -> Option<(StreamPacket, AudioCapturerControlHandle)>

pub fn into_discard_all_packets( self ) -> Option<AudioCapturerDiscardAllPacketsResponder>

pub fn into_discard_all_packets_no_reply( self ) -> Option<AudioCapturerControlHandle>

pub fn into_set_pcm_stream_type( self ) -> Option<(AudioStreamType, AudioCapturerControlHandle)>

pub fn into_capture_at( self ) -> Option<(u32, u32, u32, AudioCapturerCaptureAtResponder)>

pub fn into_start_async_capture( self ) -> Option<(u32, AudioCapturerControlHandle)>

pub fn into_stop_async_capture( self ) -> Option<AudioCapturerStopAsyncCaptureResponder>

pub fn into_stop_async_capture_no_reply( self ) -> Option<AudioCapturerControlHandle>

pub fn into_bind_gain_control( self ) -> Option<(ServerEnd<GainControlMarker>, AudioCapturerControlHandle)>

pub fn into_get_reference_clock( self ) -> Option<AudioCapturerGetReferenceClockResponder>

pub fn into_set_reference_clock( self ) -> Option<(Option<Clock>, AudioCapturerControlHandle)>

pub fn into_set_usage( self ) -> Option<(AudioCaptureUsage, AudioCapturerControlHandle)>

pub fn into_get_stream_type(self) -> Option<AudioCapturerGetStreamTypeResponder>

pub fn method_name(&self) -> &'static str

Name of the method defined in FIDL

Trait Implementations

impl Debug for AudioCapturerRequest

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.