class StreamBufferConstraints

Returns whether no field is set.

This is a version number the server sets on the constraints to allow the

server to determine when the client has caught up with the latest

constraints sent by the server. The server won't emit output data until

the client has configured output settings and buffers with a

buffer_constraints_version_ordinal >= the latest

buffer_constraints_version_ordinal that had

buffer_constraints_action_required true. See

buffer_constraints_action_required comments for more.

A buffer_constraints_version_ordinal of 0 is not permitted, to simplify

initial state handling. Other than 0, both odd and even version ordinals

are allowed (in contrast to the stream_lifetime_ordinal, neither the

client nor server ever has a reason to consider the latest version to be

stale, so there would be no benefit to disallowing even values).

This is a version number the server sets on the constraints to allow the

server to determine when the client has caught up with the latest

constraints sent by the server. The server won't emit output data until

the client has configured output settings and buffers with a

buffer_constraints_version_ordinal >= the latest

buffer_constraints_version_ordinal that had

buffer_constraints_action_required true. See

buffer_constraints_action_required comments for more.

A buffer_constraints_version_ordinal of 0 is not permitted, to simplify

initial state handling. Other than 0, both odd and even version ordinals

are allowed (in contrast to the stream_lifetime_ordinal, neither the

client nor server ever has a reason to consider the latest version to be

stale, so there would be no benefit to disallowing even values).

If a codec has

`[fuchsia.mediacodec/DetailedCodecDescription.supports_dynamic_buffers]`

set to true, this field must be set by the server and will be the

minimum number of input or output buffers that the server needs to be

able to concurrently use (or continue using) to guarantee forward

progress. For video decoders, this number assumes a

bitstream-standard-compliant stream. Any time the server doesn't have

this many buffers available for the server's concurrent use (DPB size +

1), processing may pause, but will not fail - in this case the client

can add more buffers with AddBuffers to get the processing to start

again. When the DPB needs to reference more images, recycling an output

buffer won't necessarily cause processing to start again, as the image

may still be in the DPB, so the buffer can't be re-used yet despite

being recycled by the client.

For input, the codec may require this many buffers before the codec will

process any input. Currently, for input, this value is static per

StreamProcessor instance. The client is allowed to send QueueInput...()

messages before this many input buffers have been added, but should not

expect any output to necessarily be emitted until at least this many

input buffers have been added. This also applies for

QueueInputFormatDetails and QueueInputEndOfStream before any

QueueInputPacket, for a newly starting stream, so the first two can be

sent while there are still zero added input buffers so far. Codecs that

only need one input buffer to make forward progress (non-continuous

non-smooth forward progress still counts) should set this to one. Most

clients will want to add a low number of additional buffers to keep the

pipeline running smoothly, to avoid stalling processing while an

OnFreeInputPacket is on its way from the codec to the client. Decoders

that copy input data into a separate stream buffer (sometimes treated as

circular) typically will only need one input buffer for forward progress

to be possible, but most clients will still want to add a low number of

additional input buffers for smoother thread scheduling, even if such a

decoder core might be able to run at 100% utilization with a single

input buffer.

For video decoder output, if a client were to only `AddBuffers` this

many buffers overall, every time `OnOutputPacket` happened, the decoder

would be potentially stalled until `RecycleOutputPacket` is received at

the server. For this reason, most clients will prefer to allocate a few

more buffers, accounting for other parts of the pipeline (locally in the

client or not) that may also need to "camp" on buffers for reasons

unrelated to video decode, and possibly also a low number of additional

buffers to keep the pipeline running smoothly, to avoid stalling decode

while a free buffer on its way to being recycled or similar.

For video decoder output, when set, this value will always be less than

or equal to

`[fuchsia.mediacodec/DetailedCodecDescription.dynamic_buffers_video_decoder_output_safe]`.

This value can be updated by a new OnOutputConstraints from the server

with action_required false. However, despite "action_required" being

false, the client must still ensure that the number of buffers currently

available for concurrent use by the codec is at least this many.

If a client using dynamic buffers with a video decoder doesn't want to

pay attention to this field, the client should still pay attention to

`[fuchsia.mediacodec/DetailedCodecDescription.dynamic_buffers_video_decoder_output_safe]`.

Or the client can instead use

SetInputBufferPartialSettings/SetOutputBufferPartialSettings which has

approximately the same (potentially adverse) effect in terms of overall

buffer count (assuming a correctly operating pipeline). While these less

complicated client implementation options might initially seem like

they're wasting some memory, paying attention to

buffer_count_for_server_current (in contrast to only

dynamic_buffers_video_decoder_output_safe) only saves memory if the

stream has the needed optional fields with values less than the max DPB

size, which is unfortunately not guaranteed. In particular, h.264

streams without max_num_reorder_frames or max_dec_frame_buffering set in

vui_parameters (or anywhere else) seem more common than not (to this

author, among streams looked at so far), though this could depend

strongly on which particular set of streams a client wants to decode.

Client implementers may want to check some of the relevant streams for

relevant bitstream header fields before caring about this field and the

dynamic changes to the value conveyed by this field, for video decoders.

Also, client implementers are free to use other strategies to save

memory, such as being stingy with buffer counts but adding buffers

dynamically and quickly if it seems like the codec might be starving for

buffers (such as if downstream queued output buffers start to run low).

This sort of strategy is up to the client, and it's on the client to get

the timing right if it goes with that sort of strategy (not super easy

to get perfectly correct / tuned). The platform makes no specific

guarantee re. how quickly an additional buffer can be allocated, so a

client attempting to run with fewer than

dynamic_buffers_video_decoder_output_safe may encounter stream rendering

jank or similar, including mid-stream. The platform makes no promises

regarding the ability to avoid jank in this situation, such as if

there's a mid-stream header that increases this value for output (not

common).

Layers above are encouraged to permit storing smaller images in larger

buffers, so that mid-stream dimensions changes don't each need to

re-allocate buffers. Instead, it's almost always better to just keep

using existing larger buffers in this case, to avoid possibility of

rendering jank due to allocating buffers not always being as fast as

desired combined with output buffer count not necessarily having

sufficient extra buffers to absorb the buffer reallocation in a timing

sense. However, if a layer above really must re-allocate buffers when

the image size changes and the needed buffer size to hold the images

goes down, such a client must set force_new_buffers_for_new_dimensions

passed to CreateDecoder. Otherwise a video decoder is permitted to start

putting smaller images in the existing larger buffers, with

OnOutputFormat message(s) indicating this.

If a codec has

`[fuchsia.mediacodec/DetailedCodecDescription.supports_dynamic_buffers]`

set to true, this field must be set by the server and will be the

minimum number of input or output buffers that the server needs to be

able to concurrently use (or continue using) to guarantee forward

progress. For video decoders, this number assumes a

bitstream-standard-compliant stream. Any time the server doesn't have

this many buffers available for the server's concurrent use (DPB size +

1), processing may pause, but will not fail - in this case the client

can add more buffers with AddBuffers to get the processing to start

again. When the DPB needs to reference more images, recycling an output

buffer won't necessarily cause processing to start again, as the image

may still be in the DPB, so the buffer can't be re-used yet despite

being recycled by the client.

For input, the codec may require this many buffers before the codec will

process any input. Currently, for input, this value is static per

StreamProcessor instance. The client is allowed to send QueueInput...()

messages before this many input buffers have been added, but should not

expect any output to necessarily be emitted until at least this many

input buffers have been added. This also applies for

QueueInputFormatDetails and QueueInputEndOfStream before any

QueueInputPacket, for a newly starting stream, so the first two can be

sent while there are still zero added input buffers so far. Codecs that

only need one input buffer to make forward progress (non-continuous

non-smooth forward progress still counts) should set this to one. Most

clients will want to add a low number of additional buffers to keep the

pipeline running smoothly, to avoid stalling processing while an

OnFreeInputPacket is on its way from the codec to the client. Decoders

that copy input data into a separate stream buffer (sometimes treated as

circular) typically will only need one input buffer for forward progress

to be possible, but most clients will still want to add a low number of

additional input buffers for smoother thread scheduling, even if such a

decoder core might be able to run at 100% utilization with a single

input buffer.

For video decoder output, if a client were to only `AddBuffers` this

many buffers overall, every time `OnOutputPacket` happened, the decoder

would be potentially stalled until `RecycleOutputPacket` is received at

the server. For this reason, most clients will prefer to allocate a few

more buffers, accounting for other parts of the pipeline (locally in the

client or not) that may also need to "camp" on buffers for reasons

unrelated to video decode, and possibly also a low number of additional

buffers to keep the pipeline running smoothly, to avoid stalling decode

while a free buffer on its way to being recycled or similar.

For video decoder output, when set, this value will always be less than

or equal to

`[fuchsia.mediacodec/DetailedCodecDescription.dynamic_buffers_video_decoder_output_safe]`.

This value can be updated by a new OnOutputConstraints from the server

with action_required false. However, despite "action_required" being

false, the client must still ensure that the number of buffers currently

available for concurrent use by the codec is at least this many.

If a client using dynamic buffers with a video decoder doesn't want to

pay attention to this field, the client should still pay attention to

`[fuchsia.mediacodec/DetailedCodecDescription.dynamic_buffers_video_decoder_output_safe]`.

Or the client can instead use

SetInputBufferPartialSettings/SetOutputBufferPartialSettings which has

approximately the same (potentially adverse) effect in terms of overall

buffer count (assuming a correctly operating pipeline). While these less

complicated client implementation options might initially seem like

they're wasting some memory, paying attention to

buffer_count_for_server_current (in contrast to only

dynamic_buffers_video_decoder_output_safe) only saves memory if the

stream has the needed optional fields with values less than the max DPB

size, which is unfortunately not guaranteed. In particular, h.264

streams without max_num_reorder_frames or max_dec_frame_buffering set in

vui_parameters (or anywhere else) seem more common than not (to this

author, among streams looked at so far), though this could depend

strongly on which particular set of streams a client wants to decode.

Client implementers may want to check some of the relevant streams for

relevant bitstream header fields before caring about this field and the

dynamic changes to the value conveyed by this field, for video decoders.

Also, client implementers are free to use other strategies to save

memory, such as being stingy with buffer counts but adding buffers

dynamically and quickly if it seems like the codec might be starving for

buffers (such as if downstream queued output buffers start to run low).

This sort of strategy is up to the client, and it's on the client to get

the timing right if it goes with that sort of strategy (not super easy

to get perfectly correct / tuned). The platform makes no specific

guarantee re. how quickly an additional buffer can be allocated, so a

client attempting to run with fewer than

dynamic_buffers_video_decoder_output_safe may encounter stream rendering

jank or similar, including mid-stream. The platform makes no promises

regarding the ability to avoid jank in this situation, such as if

there's a mid-stream header that increases this value for output (not

common).

Layers above are encouraged to permit storing smaller images in larger

buffers, so that mid-stream dimensions changes don't each need to

re-allocate buffers. Instead, it's almost always better to just keep

using existing larger buffers in this case, to avoid possibility of

rendering jank due to allocating buffers not always being as fast as

desired combined with output buffer count not necessarily having

sufficient extra buffers to absorb the buffer reallocation in a timing

sense. However, if a layer above really must re-allocate buffers when

the image size changes and the needed buffer size to hold the images

goes down, such a client must set force_new_buffers_for_new_dimensions

passed to CreateDecoder. Otherwise a video decoder is permitted to start

putting smaller images in the existing larger buffers, with

OnOutputFormat message(s) indicating this.

Clients are free to ignore this field.

This field is set iff

`[fuchsia.mediacodec/DetailedCodecDescription.supports_dynamic_buffers]`

is true and the indicating port is conveying uncompressed video.

This field is provided for informational purposes, for clients that may

want to know this information. However, the StreamProcessor and sysmem

protocols work fine without the client ever reading this field.

Among the pixel formats supported by the StreamProcessor server which

can carry the full bit depth of the images, and which are broadly

supported for carrying images of the needed bit depth, and assuming

linear pixel_format_modifier, this is the pixel format that is most

performant for the StreamProcessor considered in isolation. If multiple

qualified formats are essentially the same performance, this should be

the most common format (or chosen arbitrarily among roughly

equally-common formats). Most StreamProcessor servers will hard code a

particular format to put in this field for each supported bit depth.

Currently, there is no anticipated need for a list here, but please

don't hesitate to reach out if a non-test scenario is encountered which

can only be addressed by having a list here (or by attempting several

test allocations via ParticipateInBufferAllocation, or by adding a

sysmem feature).

In general, sysmem is not guaranteed to select this pixel format, since

that depends on the full set of pixel formats supported by the

StreamProcessor instance and other sysmem participants. If only the

client and StreamProcessor are sysmem participants (such as in some

tests), if the client requires this pixel format, buffer allocation can

still succeed. Typical non-test clients will not need or want to

constrain the sysmem allocation to only allow this pixel format, as that

would be more likely to fail allocation in comparison to letting sysmem

pick a format mutually supported by all participants.

Clients are free to ignore this field.

This field is set iff

`[fuchsia.mediacodec/DetailedCodecDescription.supports_dynamic_buffers]`

is true and the indicating port is outputting uncompressed video.

This field is provided for informational purposes, for clients that may

want to know this information. However, the StreamProcessor and sysmem

protocols work fine without the client ever reading this field.

This is the required coded size, without any cropping down to display

size.

If the client gets directly involved as a sysmem participant, the client

should not set constraints which are incompatible with this size, or

allocation will fail. This field should not be taken to imply that a

given buffer only has a single allowed image size. That is incorrect in

general. Rather, this field indicates a size which will be covered by

the required_min_size and required_max_size set by the StreamProcessor

sysmem participant using token passed into

SetInputBufferPartialSettings, SetOutputBufferPartialSettings, or

ParticipateInBufferAllocation.

Despite this size indicating a specific width and height, the allocated

buffers may permit a range of image sizes (for video decoder output or

video encoder input).

Clients are free to ignore this field.

This field is set iff

`[fuchsia.mediacodec/DetailedCodecDescription.supports_dynamic_buffers]`

is true and the indicating port is outputting uncompressed video.

This field is provided for informational purposes, for clients that may

want to know this information. However, the StreamProcessor and sysmem

protocols work fine without the client ever reading this field.

This is the required coded size, without any cropping down to display

size.

If the client gets directly involved as a sysmem participant, the client

should not set constraints which are incompatible with this size, or

allocation will fail. This field should not be taken to imply that a

given buffer only has a single allowed image size. That is incorrect in

general. Rather, this field indicates a size which will be covered by

the required_min_size and required_max_size set by the StreamProcessor

sysmem participant using token passed into

SetInputBufferPartialSettings, SetOutputBufferPartialSettings, or

ParticipateInBufferAllocation.

Despite this size indicating a specific width and height, the allocated

buffers may permit a range of image sizes (for video decoder output or

video encoder input).

Clients are free to ignore this field.

This field is set iff

`[fuchsia.mediacodec/DetailedCodecDescription.supports_dynamic_buffers]`

is true and the indicating port is conveying uncompressed video.

This field is provided for informational purposes, for clients that may

want to know this information. However, the StreamProcessor and sysmem

protocols work fine without the client ever reading this field.

Among the pixel formats supported by the StreamProcessor server which

can carry the full bit depth of the images, and which are broadly

supported for carrying images of the needed bit depth, and assuming

linear pixel_format_modifier, this is the pixel format that is most

performant for the StreamProcessor considered in isolation. If multiple

qualified formats are essentially the same performance, this should be

the most common format (or chosen arbitrarily among roughly

equally-common formats). Most StreamProcessor servers will hard code a

particular format to put in this field for each supported bit depth.

Currently, there is no anticipated need for a list here, but please

don't hesitate to reach out if a non-test scenario is encountered which

can only be addressed by having a list here (or by attempting several

test allocations via ParticipateInBufferAllocation, or by adding a

sysmem feature).

In general, sysmem is not guaranteed to select this pixel format, since

that depends on the full set of pixel formats supported by the

StreamProcessor instance and other sysmem participants. If only the

client and StreamProcessor are sysmem participants (such as in some

tests), if the client requires this pixel format, buffer allocation can

still succeed. Typical non-test clients will not need or want to

constrain the sysmem allocation to only allow this pixel format, as that

would be more likely to fail allocation in comparison to letting sysmem

pick a format mutually supported by all participants.