class StreamProcessor

Permit the server to use OnStreamFailed() instead of the server just

closing the whole StreamProcessor channel on stream failure.

If the server hasn't seen this message by the time a stream fails, the

server will close the StreamProcessor channel instead of sending

OnStreamFailed().

When the client is not using dynamic buffers, a single

SetInputBufferPartialSettings() provides the StreamProcessor with the

client-specified input settings and a BufferCollectionToken which the

StreamProcessor will use to convey constraints to sysmem. Both the

client and the StreamProcessor will be informed of the allocated buffers

directly by sysmem via their BufferCollection channel (not via the

StreamProcessor channel).

The client must not QueueInput...() until after sysmem informs the

client that buffer allocation has completed and was successful.

The server should be prepared to see QueueInput...() before the server

has necessarily heard from sysmem that the buffers are allocated - the

server must tolerate either ordering, as the QueueInput...() and

notification of sysmem allocation completion arrive on different

channels, so the client having heard that allocation is complete doesn't

mean the server knows that allocation is complete yet. However, the

server can expect that allocation is in fact complete and can expect to

get the allocation information from sysmem immediately upon requesting

the information from sysmem.

Mixing of AddBuffer and SetInputBufferPartialSettings is not allowed

while there are any buffers still active on the port. To successfully

switch modes for a port, most clients will want to just start over with

a new StreamProcessor instance. This can also be done reliably by first

ensuring that every added buffer under the port up to "now" (including

those with older buffer_lifetime_ordinal) has completed a RemoveBuffer

request. Only then is it known ok to switch modes for that port under

the same StreamProcessor. Most clients will just use one way or the

other to add buffers and never need to switch to the other way,

especially within the same StreamProcessor.

This is the replacement for SetOutputBufferSettings().

When the client is using sysmem to allocate buffers, this message is

used instead of SetOutputBufferSettings()+AddOutputBuffer(). Instead, a

single SetOutputBufferPartialSettings() provides the StreamProcessor

with the client-specified output settings and a BufferCollectionToken

which the StreamProcessor will use to convey constraints to sysmem. Both

the client and the StreamProcessor will be informed of the allocated

buffers directly by sysmem via their BufferCollection channel (not via

the StreamProcessor channel).

Configuring output buffers is _required_ after OnOutputConstraints() is

received by the client with buffer_constraints_action_required true and

stream_lifetime_ordinal equal to the client's current

stream_lifetime_ordinal (even if there is an active stream), and is

_permitted_ any time there is no current stream.

Closing the current stream occurs on the StreamControl ordering domain,

so after a CloseCurrentStream() or FlushEndOfStreamAndCloseStream(), a

subsequent Sync() completion must be received by the client before the

client knows that there's no longer a current stream.

Mixing of AddBuffer and SetOutputBufferPartialSettings is not allowed

while there are any buffers still active on the port. To successfully

switch modes for a port, most clients will want to just start over with

a new StreamProcessor instance. This can also be done reliably by first

ensuring that every added buffer under the port up to "now" (including

those with older buffer_lifetime_ordinal) has completed a RemoveBuffer

request. Only then is it known ok to switch modes for that port under

the same StreamProcessor. Most clients will just use one way or the

other to add buffers and never need to switch to the other way,

especially not within the same StreamProcessor instance.

See also CompleteOutputBufferPartialSettings().

After SetOutputBufferPartialSettings(), the server won't send

OnOutputConstraints(), OnOutputFormat(), OnOutputPacket(), or

OnOutputEndOfStream() until after the client sends

CompleteOutputBufferPartialSettings().

This message isn't permitted after AddBuffer. When using dynamic buffers

the server can send OnOutputConstraints, OnOutputFormat, OnOutputPacket,

or OnOutputEndOfStream at any time after the first QueueInputPacket. In

the case of OnOutputPacket there must also be at least one output buffer

that's been added but not yet fully removed (RemoveBuffer not yet

complete).

Some clients may be able to send CompleteOutputBufferPartialSettings()

immediately after SetOutputBufferPartialSettings() - in that case the

client needs to be prepared to receive output without knowing the buffer

count or packet count yet - such clients may internally delay processing

the received output until the client has heard from sysmem (which is

when the client will learn the buffer count and packet count).

Other clients may first wait for sysmem to allocate, prepare to receive

output, and then send CompleteOutputBufferPartialSettings().

This message is optional; a client isn't required to send this ever.

Terminology note: In the name of this message, "flush" means flush

through, sometimes called "drain". This messge does not discard. To

discard, just QueueInputPacket with a new stream_lifetime_ordinal

without first waiting for OnOutputEndOfStream of the old stream and

without sending FlushEndOfStreamAndCloseStream.

There is currently no way to cancel the flush effect of this message

short of the client closing the StreamProcessor channel.

This message is only valid after QueueInputEndOfStream() for this

stream. The stream_lifetime_ordinal input parameter must match the

stream_lifetime_ordinal of the QueueInputEndOfStream(), else the server

will close the channel.

A client can use this message to flush through (drain, not discard) the

last input data of a stream so that the stream processor server

generates corresponding output data for all the input data before the

server moves on to the next stream, without forcing the client to wait

for OnOutputEndOfStream() before queueing data of another stream.

The difference between QueueInputEndOfStream() and

FlushEndOfStreamAndCloseStream(): QueueInputEndOfStream() is a promise

from the client that there will not be any more input data for the

stream (and this info is needed by some stream processors for the stream

processor to ever emit the very last output data). The

QueueInputEndOfStream() having been sent doesn't prevent the client from

later completely discarding the rest of the current stream by closing

the current stream (with or without a stream switch). In contrast,

FlushEndOfStreamAndCloseStream() is a request from the client that all

the previously-queued input data be processed including the logical

"EndOfStream" showing up as OnOutputEndOfStream() (in success case)

before moving on to any newer stream - this essentially changes the

close-stream handling from discard to flush-through for this stream

only.

A client using this message can start providing input data for a new

stream without that causing discard of old stream data. That's the

purpose of this message - to allow a client to flush through (not

discard) the old stream's last data (instead of the default when closing

or switching streams which is discard).

Because the old stream is not done processing yet and the old stream's

data is not being discarded, the client must be prepared to continue to

process OnOutputConstraints() messages until the stream_lifetime_ordinal

is done. The client will know the stream_lifetime_ordinal is done when

OnOutputEndOfStream(), OnStreamFailed(), or the StreamProcessor channel

closes.

This "closes" the current stream, leaving no current stream. In

addition, this message can optionally release (and unregister) input

buffers or output buffers.

If there has never been any active stream, the stream_lifetime_ordinal

must be zero or the server will close the channel. If there has been an

active stream, the stream_lifetime_ordinal must be the most recent

active stream whether that stream is still active or not. Else the

server will close the channel.

Multiple of this message without any new active stream in between is not

to be considered an error, which allows a client to use this message to

close the current stream to stop wasting processing power on a stream

the user no longer cares about, then later decide that buffers should be

released and send this message again with release_input_buffers and/or

release_output_buffers true to get the buffers released, if the client

is interested in trying to avoid overlap in resource usage between old

buffers and new buffers (not all clients are).

See also Sync().

On completion, all previous StreamProcessor calls have done what they're

going to do server-side, _except_ for processing of data queued using

QueueInputPacket().

The main purpose of this call is to enable the client to wait until

CloseCurrentStream() with release_input_buffers and/or

release_output_buffers set to true to take effect, before the client

allocates new buffers and re-sets-up input and/or output buffers. This

de-overlapping of resource usage can be worthwhile for media buffers

which can consume resource types whose overall pools aren't necessarily

vast in comparison to resources consumed. Especially if a client is

reconfiguring buffers multiple times.

Note that Sync() prior to allocating new media buffers is not alone

sufficient to achieve non-overlap of media buffer resource usage system

wide, but it can be a useful part of achieving that.

The Sync() transits the Output ordering domain and the StreamControl

ordering domain, but not the InputData ordering domain.

This request can be used to avoid hitting kMaxInFlightStreams which is

presently 10. A client that stays

<

= 8 in-flight streams will

comfortably stay under the limit of 10. While the protocol permits

repeated SetInputBufferSettings() and the like, a client that spams the

channel can expect that the channel will just close if the server or the

channel itself gets too far behind.

After the client is done with an output packet, the client needs to tell

the stream processor that the output packet can be re-used for more

output, using this message.

It's not permitted to recycle an output packet that's already free with

the stream processor server.

If a client is using EnableOldOutputBuffers, the client must recycle all

packets when done with them, even those of old buffer_lifetime_ordinal.

This is also permitted behavior for all clients.

If a client is not using EnableOldOutputBuffers, the client may

optionally omit this message for packets with an old

buffer_lifetime_ordinal. In other words, packets from before an explicit

or implicit output buffer de-configuration don't need to be recycled if

the client isn't using and won't be using EnableOldOutputBuffers.

If the input format details are still the same as specified during

StreamProcessor creation, this message is unnecessary and does not need

to be sent.

If the stream doesn't exist yet, this message creates the stream.

The server won't send OnOutputConstraints() until after the client has

sent at least one QueueInput* message.

All servers must permit QueueInputFormatDetails() at the start of a

stream without failing, as long as the new format is supported by the

StreamProcessor instance. Technically this allows for a server to only

support the exact input format set during StreamProcessor creation, and

that is by design. A client that tries to switch formats and gets a

StreamProcessor channel failure should try again one more time with a

fresh StreamProcessor instance created with CodecFactory using the new

input format during creation, before giving up.

These format details override the format details specified during stream

processor creation for this stream only. The next stream will default

back to the format details set during stream processor creation.

This message is permitted at the start of the first stream (just like at

the start of any stream). The format specified need not match what was

specified during stream processor creation, but if it doesn't match, the

StreamProcessor channel might close as described above.

This message queues input data to the stream processor for processing.

If the stream doesn't exist yet, this message creates the new stream.

The server won't send OnOutputConstraints() until after the client has

sent at least one QueueInput* message.

When using dynamic buffers the server can send OnOutputConstraints,

OnOutputFormat, OnOutputPacket, or OnOutputEndOfStream at any time after

the first QueueInputPacket. In the case of OnOutputPacket there must

also be at least one output buffer that's been added but not yet fully

removed (RemoveBuffer not yet complete).

The client must continue to deliver input data via this message even if

the stream processor has not yet generated the first

OnOutputConstraints, and even if the StreamProcessor is generating

OnFreeInputPacket for previously-queued input packets. The input data

must continue as long as there are free packets to be assured that the

server will ever generate the first OnOutputConstraints.

The server will close the channel if this packet refers to an old

buffer_lifetime_ordinal. Clients that need to deliver input images of

different dimensions to a video encoder can either (a) allocate buffers

large enough to contain the range of needed image sizes and use those to

deliver all the input images, (b) keep their own sets of buffers used

for different image dimensions and re-add old buffers with AddBuffer,

moving to a new buffer_lifetime_ordinal each time image dimensions

change, or (c) allocate new buffers each time image dimensions change.

The (a) option is preferred, for clients that have the flexibiilty to

store images of varying dimensions in a single set of buffers. Clients

that require each buffer to have a single image size can use option (b)

or (c).

Inform the server that all QueueInputPacket() messages for this stream

have been sent.

If the stream isn't closed first (by the client, or by OnStreamFailed(),

or StreamProcessor channel closing), there will later be a corresponding

OnOutputEndOfStream().

The corresponding OnOutputEndOfStream() message will be generated only

if the server finishes processing the stream before the server sees the

client close the stream (such as by starting a new stream). A way to

force the server to finish the stream before closing is to use

FlushEndOfStreamAndCloseStream() after QueueInputEndOfStream() before

any new stream. Another way to force the server to finish the stream

before closing is to wait for the OnOutputEndOfStream() before taking

any action that closes the stream.

In addition to serving as an "EndOfStream" marker to make it obvious

client-side when all input data has been processed, if a client never

sends QueueInputEndOfStream, no amount of waiting will necessarily

result in all input data getting processed through to the output. Some

stream processors have some internally-delayed data which only gets

drained (pushed through) by additional input data _or_ by this

EndOfStream marker. In that sense, this message can be viewed as a drain

at InputData domain level, but the drain only takes effect if the stream

processor even gets that far before the stream is just closed at

StreamControl domain level. This message is not alone sufficient to act

as an overall drain at StreamControl level. For that, send this message

first and then send FlushEndOfStreamAndCloseStream (at which point it

becomes possible to queue input data for a new stream without causing

discard of this older stream's data). Alternately, the client can wait

for the OnOutputEndOfStream before closing the current stream.

After a client sends QueueInputEndOfStream for a stream, if the client

then sends for the same stream any of QueueInputPacket,

QueueInputFormatDetails, QueueInputEndOfStream, the server will close

the StreamProcessor channel.

This message results in channel closure unless supports_dynamic_buffers

is set to true.

This participates in allocation of buffers to be used with AddBuffer

later. The client can get VMO handles for these buffers by also

participating in the sysmem allocation, using the client's own related

sysmem token (associated with the same logical buffer collection). It's

up to the client to separately set any constraints needed by the client

using the client's own related sysmem token, if any.

Some clients may prefer to use SetInputBufferPartialSettings and/or

SetOutputBufferPartialSettings. Servers must support those messages.

In handling this message, if `allow_single_buffer` is set to true, the

server must not constrain the number of buffers allocated. The server

must set min_buffer_count to 1, and must leave max_buffer_count un-set

or set it to 0xFFFFFFFF, and must leave all min_buffer_count_* fields

un-set. The sender can set min_buffer_count and max_buffer_count to the

same value if the intent is to allocate exactly that many buffers. If

`allow_single_buffer` is un-set or set to false, the server will

indicate needed buffer counts to sysmem.

The server's BufferCollection channel (created from the passed-in

sysmem2_token) may see ZX_CHANNEL_PEER_CLOSED at any time, but in

particular, the server shouldn't expect the BufferCollection channel to

remain connected to sysmem beyond the server sending SetConstraints. For

this reason, the server may not be able to call

WaitForAllBuffersAllocated or similar, so the server should just send

SetConstraints, Close, then close the server's BufferCollection

client_end. This means the server in general shouldn't attempt to get

VMO handles for these buffers while processing this message.

The server should not assume that these buffers will necessarily ever be

added with AddBuffer to this StreamProcessor instance or any other

StreamProcessor instance (owned by the server or not). These buffers may

instead be dropped, or as a less-common example, possibly added to a

different codec served by a different server implementation which also

participated in the same sysmem buffer collection allocation.

For input buffers, AddBuffer of the allocated buffer(s) to a different

StreamProcessor instance of the same codec (same per CodecFactory) is

likely to work, but using the same StreamProcessor instance is

recommended when feasible.

In contrast, for output buffers, AddBuffer of the allocated buffer(s) to

a different StreamProcessor instance of the same codec (same per

CodecFactory) can't (within reason) be made work in general, especially

for video decoders. Therefore, for output buffers, the same

StreamProcessor instance must be used for this message and AddBuffer.

While a client may currently be able to get away with using different

StreamProcessor instances for this message and AddBuffer for output

buffers for some codecs, this may break at any time without it being

considered a server-side bug.

The allocated buffers can later be added using AddBuffer (piecemeal),

and can be removed (piecemeal) using RemoveBuffer.

Multiple different ParticipateInBufferAllocation messages can have their

buffers later added to the same StreamProcessor instance using the same

buffer_lifetime_ordinal. This can be useful if the client wants to

allocate buffers incrementally, or dynamically adjust the number of

buffers, potentially while actively processing. See also the

`buffer_lifetime_ordinal` field of this message.

Server implementations may use sysmem to help verify buffer

compatibility later when buffers are added with AddBuffer.

Add buffers previously created with the help of

ParticipateInBufferAllocation.

For input buffers, the client can send QueueInputFormatDetails or

QueueInputEndOfStream before any AddBuffer messages. At least one input

buffer must be added before a valid QueueInputPacket can be sent. Most

clients will want to continue quickly adding buffers up to at least

buffer_count_for_server_current to avoid the codec potentially stalling,

and typically a low number of buffers beyond that to keep the pipeline

running smoothly. Input buffers beyond the first input buffer can be

added after the first QueueInputPacket.

Even if buffer_constraints_version_ordinal is current, the server must

not close the channel if the buffer isn't consistent with the current

buffer_constraints_version_ordinal (per sysmem GetVmoInfo given

consistent StreamProcessor constraints). Instead, the server must send a

new OnOutputConstraints. This simplifies some edge cases for some

clients, particularly when a client can't reliably detect whether a

newly-obtained buffer was actually allocated after

buffer_constraints_version_ordinal changed, or may have been cached from

before.

The client can add additional buffers to the same port and

buffer_lifetime_ordinal at any time using this message. If the

buffer_lifetime_ordinal is no longer the most recent, the message will

be ignored, the handle to the buffer dropped, and any later RemoveBuffer

message re. the same buffer will complete immediately.

If dynamic_buffers_input_max or dynamic_buffers_output_max is exceeded

by the sum of buffers added by all AddBuffer calls with the same

buffer_lifetime_ordinal, the server may close the channel. Servers are

required to close the channel in this case if performance degradation or

un-tested behavior would result from adding too many buffers.

Switching to a new buffer_lifetime_ordinal starts the process of

removing buffers associated with an old buffer_lifetime_ordinal.

However, until the remove is complete, those buffers can still be used

by the codec as normal. See also RemoveBuffer, which can be used to

detect when removal is complete, regardless of whether the RemoveBuffer

started the removal (including when not using dynamic buffers).

The buffer stays added until removal later completes. The buffer remains

added across potentially multiple buffer re-uses. Removal can be

initiated (and/or confirmed/fenced) by the client using RemoveBuffer.

The codec server can unilaterally initiate buffer removal; if the server

does this, it must send a new buffer_constraints_version_ordinal with

action_required true. Some other client-initiated messages can also

begin buffer removal, such as CloseCurrentStream with

release_input_buffers and/or release_output_buffers. The buffers are

also automatically removed and released if the client closes the

StreamProcessor client_end or the server closes the StreamProcessor

server_end.

Mixing of AddBuffer and SetInputBufferPartialSettings /

SetOutputBufferPartialSettings is not allowed while there are any

buffers still active on the port. To successfully switch modes for a

port, most clients will want to just start over with a new

StreamProcessor instance. This can also be done reliably by first

ensuring that every added buffer under the port up to "now" (including

those with older buffer_lifetime_ordinal) has completed a RemoveBuffer

request. Only then is it known ok to switch modes for that port under

the same StreamProcessor. Most clients will just use one way or the

other to add buffers and never need to switch to the other way,

especially within the same StreamProcessor.

If a client might plausibly "spam" creation of many new

buffer_lifetime_ordinal values without the buffers seeing any actual

usage in between, the client should consider starting a Sync every few

buffer_lifetime_ordinal(s) to fence cleanup of old

buffer_lifetime_ordinal values, and avoid getting ahead of Sync

completions by more than 16 buffer_lifetime_ordinal values. Else the

channel may close from a backlog of new buffer_lifetime_ordinal(s)

getting too far ahead of closing out old ones. The threshold of 16 is

well below the enforcement threshold. Clients don't need to Sync if they

won't be spamming new buffer_lifetime_ordinal values, or if added

buffers will see at least some actual usage visible to the client before

being replaced again.

All buffers of the same port and buffer_lifetime_ordinal must share the

same `[fuchsia.sysmem2/SingleBufferSetttings]`. The client can ensure

this in various ways. One way is to use ParticipateInBufferAllocation

then AddBuffer for at least the first buffer, before

ParticipateInBufferAllocation for any subsequent buffers. Another way is

to observe a mismatch in `SingleBufferSettings` before sending AddBuffer

and bump to the next odd buffer_lifetime_ordinal value for the

AddBuffer.

When using dynamic buffers, this call removes a buffer as soon as the

buffer can be removed without adversely impacting any ongoing processing

or an in-flight output packet referring to the buffer.

If a client wants to remove a buffer with an in-flight output packet

referring to the buffer, the client must RecycleOutputPacket for that

in-flight output packet before the RemoveBuffer will complete.

When using SetInputBufferPartialSettings /

SetOutputBufferPartialSettings, this call doesn't initiate removal of

the buffer. This call will complete when the buffer is done removing due

to other reasons, such as a new buffer_lifetime_ordinal starting.

Until RemoveBuffer completes, the codec is still allowed to send

OnOutputPacket messages referencing the buffer, and the codec may still

have a VMO handle open to the buffer. After RemoveBuffer completes, the

codec guarantees that no subsequent output packet will reference the

buffer, and that the server holds no VMO handles to the buffer.

The client may need to recycle an output packet before the RemoveBuffer

call can complete. Clients should take care to avoid blocking packet

recycling while the RemoveBuffer request is in progress, since this

would create a potential deadlock.

The server closing any VMO handles to the buffer prior to completing

this call is important for the client's ability to prevent memory usage

spikes.

If the client has "paused" processing by not providing any more input,

the client will potentially need to send CloseCurrentStream before

RemoveBuffer will complete. When using dynamic buffers, setting

release_input_buffers or release_output_buffers to true is not

necessary; the RemoveBuffer is explicitly telling the codec to release a

specific buffer. When not using dynamic buffers, the client will need to

set release_input_buffers or release_output_buffers to true, since

RemoveBuffer alone doesn't initiate removal when not using dynamic

buffers. A current "paused" stream needs to be stopped because frames

can be held as reference frames, and codecs are never reqiured to copy

their output data.

For any video decoder output buffers being removed while there's an

active stream (being fed input or not), it's typically best for the

client to assume that removal of a video decoder output buffer may take

a very long duration. This is because bitstreams, especially

non-standard-compliant bitstreams, but in some cases potentially even

standard-compliant bitstreams, can keep a video decoder output buffer in

the set of active reference frames (aka DPB) indefinitely. The server is

not required to notice that a stream is not conforming to a bitstream

standard in this regard.

In the case of h.264 decode, a standard-complient bitstream will limit

the reorder delay to no longer than the max DPB occupancy. However, a

server is not required to detect or reject non-compliant streams that

potentially keep a frame in the DPB for longer.

For HEVC, the situation is similar to h.264 (IIUC).

In the case of VP9, there is nothing in the bitstream spec that limits

the reorder delay (IIUC), meaning a frame can potentially stay in VP9's

set of 8 reference frames (aka DPB) indefinitely. The server is not

required to detect or mitigate this.

Until this call completes, the server may still be using the buffer. For

output buffers, the server can still send OnOutputPacket message(s) that

references this buffer, but only up until the RemoveBuffer completion

message is sent by the server. The client must continue to

RecycleOutputPacket for packets that reference the buffer, until

RemoveBuffer completes.

When using dynamic buffers, upon receiving this message, the server will

stop selecting the buffer for any new usage (as in, for any usage that

moves the buffer from "free" to "not free" within the server). This

applies even if the server has no other buffers available for use (aka

no other "free" buffers). Any existing usage of the buffer is not ended

early by this call alone.

When not using dynamic buffers, upon receiving this message, the server

will just remember to complete this call shortly after the buffer has

completed removal triggered by some other cause (removal is not

triggered/caused by this call).

For video decoders, when using dynamic buffers, typically the client

should take care to avoid removing too many output buffers for continued

decode to be possible. If this occurs, the decoder will wait until the

client adds another output buffer with AddBuffer. This can be a deadlock

if the client never sends that AddBuffer. Due to DPB mechanism(s) and

frame reordering, sending a single AddBuffer doesn't necessarily

guarantee another OnOutputPacket, since additional output buffers can be

needed before the server can send OnOutputPacket.

The client must not call RemoveBuffer on the same buffer more than once,

whether overlapping in time or not. The server should enforce this when

not enforcing would require tracking additional concurrent requests. The

server is not required to enforce this when enforcing would use more

server memory.

The server is allowed to complete this request quickly with success for

buffer_lifetime_ordinal and buffer_index combinations that were never

real buffers, but must close the channel if the buffer_lifetime_ordinal

hasn't been started by the client yet (no removing potential future

buffers).

When using dynamic buffers, a buffer_lifetime_ordinal and buffer_index

combination may be re-used after completion of RemoveBuffer, but only if

the buffer_lifetime_ordinal is the current buffer_lifetime_ordinal. In

other words, no adding buffers under an old buffer_lifetime_ordinal.

Assuming a valid historical buffer is specified, successful completion

of this call means the buffer has been fully released by the server and

won't be referenced in any subsequent OnOutputPacket.

This informs the StreamProcessor that the client is prepared to handle

output packets that specify a buffer with buffer_lifetime_ordinal older

than the most recent buffer_lifetime_ordinal.

If the client doesn't send this message, the StreamProcessor will omit

any such output, even if

DetailedCodecDescription.supports_dynamic_buffers is true. For relevant

decoders such as VP9 decoders, not sending this message can result in

output that isn't bistream spec compliant, and the output can be

visually different than intended by the bitstream.

Such streams are only possible with some bitstream formats (such as

VP9), and are rare, but can happen and can be valid per the bitstream

spec. For example, this can be specified by a VP9 bitstream using

show_existing_frame to output an old-dimensions buffer after having

already output a new-dimensions buffer.

Most clients that send this message will also want to use RemoveBuffer

to know when it becomes safe to stop tracking an old buffer.

Most of the time this makes no difference as most bitstreams don't

actually emit old buffers, even if the bitstream spec would allow it.

Old output buffers are especially rare for RTC streams which typically

don't have any frame reordering in the first place.

In most video streaming scenarios that use dimension switching as part

of their bitrate control strategy (among those that I've observed), at

the StreamProcessor layer the new dimensions are part of a new stream

instead of being spliced together as a continuation of the old stream.

That said, using a continuation of the old stream is also a completely

valid way to implement dimension switching. When a stream switch occurs

as part of dimension switching, the decoder state is not retained and

there won't be any old buffer(s) emitted after new buffer(s), since the

new stream doesn't know anything about old buffers filled by the old

stream.

Clients which haven't tested their ability to handle old output buffers

should not send this message. Clients decoding bitstreams like VP9 for

decoder compliance testing purposes should send this message (and use a

VP9 decoder with DetailedCodecDescrption.supports_dynamic_buffers true).

Clients which are required to support old output frames and/or fully

comply with a relevant bitstream spec should/must send this message, and

should test using a test stream that outputs packets referencing an old

output buffer.

Sending this message more than once closes the channel. If sent, this

message must be sent prior to the client establishing the first output

buffer_lifetime_ordinal. This requirement avoids ambiguity re. free/busy

status of packets of old buffer_lifetime_ordinal(s), as the server can

auto-recycle packets with old buffer_lifetime_ordinal on behalf of the

client when this message was not sent by the client.

This message is only permitted when

`[fuchsia.mediacodec/CodecFactory.DetailedCodecDescription.supports_dynamic_buffers]`

is true.

This informs the StreamProcessor that the client is prepared to handle

output packets that specify the same buffer as another packet that's

also concurrently in flight to the client (not yet recycled).

Most bitstream formats don't do this. In formats that can do this such

as VP9, most actual bitstreams don't do this.

As an example, in VP9, a stream can cause the same output buffer to be

used by another emitted output packet/frame by using show_existing_frame

on the same VP9 reference/held frame slot more than once without

decoding a new frame into that slot in between.

Clients that need to achieve full bitstream spec compliance for such a

bitstream format must send this message.

If this message is not in effect, the server will omit any such output,

which can result in output that is not compliant to the relevant

bitstream spec, and which can be visually different than the stream

intended.

Sending this message more than once closes the channel. If sent, this

message must be sent prior to the client establishing the first output

buffer_lifetime_ordinal.

This message is only permitted when

`[fuchsia.mediacodec/CodecFactory.DetailedCodecDescription.supports_dynamic_buffers]`

is true.

For video decoders, this forces the output buffers to be reallocated if

the image size needs to change. This is wasteful as it forces extra

buffer reallocations given typical video bitrate control strategies

(applicable to both streaming and RTC) involving shifting the image

dimensions up and down repeatedly as a logical video/stream plays,

sometimes even if network conditions remain fairly stable. This extra

buffer reallocation cost is incurred vs. baseline whether the dimension

switching is achieved within a single StreamProcessor stream or by using

a new StreamProcessor stream for new dimensions.

A client should not send this message unless the client really must

force the output buffers to be reallocated every time the output image

size changes.

Sending this message more than once closes the channel. If sent, this

message must be sent prior to any SetInputBufferPartialSettings,

SetOutputBufferPartialSettings, ParticipateInBufferAllocation, or

AddBuffer.

This message is only permitted if this StreamProcessor is a video

decoder.

This message is only permitted when

`[fuchsia.mediacodec/CodecFactory.DetailedCodecDescription.supports_dynamic_buffers]`

is true.