class WireAsyncEventHandler

The stream has failed, but the StreamProcessor instance is still usable

for a new stream.

This message is only ever sent by the server if the client previously

sent EnableOnStreamFailed(). If the client didn't send

EnableOnStreamFailed() then the server closes the StreamProcessor

channel instead.

StreamProcessor server implementations are encouraged to handle stream

errors (and ideally to also report them via error_ bools of

OnOutputPacket() and OnOutputEndOfStream()) without failing the whole

stream, but if a stream processor server is unable to do that, but still

can cleanly contain the failure to the stream, the stream processor

server can (assuming EnableOnStreamFailed() was called) use

OnStreamFailed() to indicate the stream failure to the client without

closing the StreamProcessor channel.

An ideal StreamProcessor server handles problems with input data without

sending this message, but sending this message is preferred vs. closing

the server end of the StreamProcessor channel if the StreamProcessor

server can 100% reliably contain the stream failure to the stream,

without any adverse impact to any later stream.

No further messages will arrive from the server regarding the failed

stream. This includes any OnOutputEndOfStream() that the client would

have otherwise expected.

The server sends this shortly after StreamProcessor creation to indicate

input buffer constraints. The "min" and "max" input constraints don't

change for the life of the StreamProcessor.

The "max" values for buffer size and count are large enough to support

the most demanding format the server supports on input. The

"recommended" values should be workable for use with the input

FormatDetails conveyed during StreamProcessor creation. The

"recommended" values are not necessarily suitable if the client uses

QueueInputFormatDetails() to change the input format. In that case it's

up to the client to determine suitable values, either by creating a new

StreamProcessor instance instead, or knowing suitable values outside the

scope of this protocol.

See comments on StreamBufferConstraints.

This message is guaranteed to be sent unsolicited to the StreamProcessor

client during or shortly after StreamProcessor creation. Clients should

not depend on this being the very first message to arrive at the client.

The "min" and "max" input constraints are guaranteed not to change for a

given StreamProcessor instance. The "recommended" values may

effectively change when the server processes QueueInputFormatDetails().

There is not any way in the protocol short of creating a new

StreamProcessor instance for the client to get those new "recommended"

values.

This event informs the client of new output constraints.

This message is ordered with respect to other output (such as output

packets, output format, output end-of-stream).

Before the first OnOutputPacket() of a stream, the server guarantees that

at least one OnOutputConstraints() and exactly one OnOutputFormat() will

be sent. The server may not set buffer_constraints_action_required true

in OnOutputConstraints() if the buffer config is already suitable for the

stream (buffer_constraints_action_required false means the buffer config

is already fine). The client must tolerate multiple

OnOutputConstraints() (and 1 OnOutputFormat() message) before the first

output packet. As long as the client hasn't moved to a new stream, the

server won't send another OnOutputConstraints() until after the client

has configured output buffers.

This message can be sent mid-stream by a server. If

buffer_constraints_action_required false, the message is safe to

ignore, but a client may choose to stash the new constraints for

later use the next time the client wants to unilaterally re-configure

buffers (when allowed). If later the server needs the output config to

change, the server may send a new OnOutputConstraints() with

buffer_constraints_action_required true.

On buffer_constraints_action_required true, a client that does not wish

to fully handle mid-stream output buffer config changes should either

give up completely on the processing, or at least re-config the output

as specified before starting a new stream (and possibly re-delivering

input data, if the client wants). This avoids useless retry with a new

stream starting from just before the output buffer config change which

would hit the same mid-stream output config change again.

Similarly, some servers may only partly support mid-stream format

changes, or only support a mid-stream format change if the buffers are

already large enough to handle both before and after the format change.

Such servers should still indicate buffer_constraints_action_required

true, but then send OnStreamFailed() after the client has re-configured

output buffers (seamlessly dealing with the mid-stream output config

change is even better of course, but is not always feasible depending on

format). When the client retries with a new stream starting from a

nearby location in the client's logical overall media timeline, the

output buffers will already be suitable for the larger size output, so

the new stream will not need any mid-stream output buffer re-config,

only a mid-stream OnOutputFormat(). This strategy avoids the problem

that would otherwise occur if a client were to retry with a new stream

starting just before the mid-stream output buffer config change (the

retry wouldn't be effective since the same need for an output buffer

config change would be hit again). Servers are discouraged from sending

OnStreamFailed() solely due to a mid-stream need for different output

buffer config without first sending OnOutputConstraints() with

buffer_constraints_action_required true and waiting for the client to

re-configure output buffers (to avoid the useless client retry with a

new stream from a logical location before the config change).

When buffer_constraints_action_required true, the server will not send

any OnOutputPacket() for this stream until after the client has

configured/re-configured output buffers.

A client that gives up on processing a stream on any mid-stream

OnOutputConstraints() or mid-stream OnOutputFormat() should completely

ignore any OnOutputConstraints() with buffer_constraints_action_required

false. Otherwise the client may needlessly fail processing, or server

implementations might not be able to use

buffer_constraints_action_required false for fear of simpler clients

just disconnecting.

All clients, even those which don't want to support any mid-stream

output buffer re-config or mid-stream OnOutputFormat() are required to

deal with 1..multiple OnOutputConstraints() messages before the first

output packet, and 1 OnOutputFormat() messages before the first output

packet.

This message is ordered with respect to output packets, and with respect

to OnOutputFormat().

This message is sent by the server before the first output packet of any

stream, and potentially mid-stream between output packets of the stream,

ordered with respect to output packets, and ordered with respect to

OnOutputConstraints().

The server guarantees that the first packet of every stream will be

preceeded by an OnOutputFormat().

The server guarantees that there will be an OnOutputFormat() between an

OnOutputConstraints() with buffer_constraints_action_required true and an

OnOutputPacket(). In other words, the client is essentially allowed to

forget what the output format is on any OnOutputConstraints() with

buffer_constraints_action_required true, because the server promises a

subsequent OnOutputFormat() before any OnOutputPacket().

If the server sets buffer_constraints_action_required true in

OnOutputConstraints(), the server won't send OnOutputFormat() (and

therefore also won't send OnOutputPacket()) until the client has

re-configured output buffers.

The server is allowed to send an OnOutputFormat() mid-stream between two

output packets.

A server won't send two adjacent OnOutputFormat() messages without any

output packet in between. However an OnOutputFormat() message doesn't

guarantee a subsequent packet, because for example the server could send

OnOutputEndOfStream() or OnStreamFailed() instead.

A client that does not wish to seamlessly handle mid-stream output format

changes should either ensure that no stream processed by the client

ever has any mid-stream format change, or the client should ensure that

any retry of processing starts the new attempt at a point logically at or

after the point where the old format has ended and the new format starts,

else the client could just hit the same mid-stream format change again.

An example of this message being sent mid-stream is mid-stream change

of dimensions of video frames output from a video decoder.

Not all servers will support seamless handling of format change. Those

that do support seamless handling of format change may require that the

format change not also require output buffer re-config, in order for the

handling to be seamless. See the comment block for OnOutputConstraints()

for more discussion of how servers and clients should behave - in

particular when they don't seamlessly handle output constraint change

and/or output format change.

If this message isn't being sent by the server when expected at the

start of a stream, the most common reason is that a OnOutputConstraints()

with buffer_constraints_action_required true hasn't been processed by the

client (by configuring output buffers using

SetOutputBufferPartialSettings() etc).

This is how the stream processor emits an output packet to the stream

processor client.

Order is significant.

The client should eventually call RecycleOutputPacket() (possibly after

switching streams multiple times), unless the buffer_lifetime_ordinal

has moved on. A stream change doesn't change which packets are busy

with the client vs. free with the server.

The relevant buffer is always the one specified in the packet's buffer_index field.

For low-level buffer types that support it, a StreamProcessor is free to

emit an output packet before the low-level buffer actually has any

usable data in the buffer, with the mechanism for signalling the

presence of data separate from the OnOutputPacket() message. For such

low-level buffer types, downstream consumers of data from the emitted

packet must participate in the low-level buffer signalling mechanism to

know when it's safe to consume the data. This is most likely to be

relevant when using a video decoder and gralloc-style buffers.

The error_ bool(s) allow (but do not require) a StreamProcessor server

to report errors that happen during an AU or between AUs.

The scope of error_detected_before starts at the end of the last

delivered output packet on this stream, or the start of stream if there

were no previous output packets on this stream. The scope ends at the

start of the output_packet.

The error_detected_before bool is separate so that discontinuities can be

indicated separately from whether the current packet is damaged.

The scope of error_detected_during is from the start to the end of this

output_packet.

After QueueInputEndOfStream() is sent by the StreamProcessor client,

within a reasonable duration the corresponding OnOutputEndOfStream()

will be sent by the StreamProcessor server. Similar to

QueueInputEndOfStream(), OnOutputEndOfStream() is sent a maximum of once

per stream.

No more stream data for this stream will be sent after this message. All

input data for this stream was processed.

While a StreamProcessor client is not required to

QueueInputEndOfStream() (unless the client wants to use

FlushEndOfStreamAndCloseStream()), if a StreamProcessor server receives

QueueInputEndOfStream(), and the client hasn't closed the stream, the

StreamProcessor server must generate a corresponding

OnOutputEndOfStream() if nothing went wrong, or must send

OnStreamFailed(), or must close the server end of the StreamProcessor

channel. An ideal StreamProcessor server would handle and report stream

errors via the error_ flags and complete stream processing without

sending OnStreamFailed(), but in any case, the above-listed options are

the only ways that an OnOutputEndOfStream() won't happen after

QueueInputEndOfStream().

There will be no more OnOutputPacket() or OnOutputConstraints() messages

for this stream_lifetime_ordinal after this message - if a server doesn't

follow this rule, a client should close the StreamProcessor channel.

The error_detected_before bool has the same semantics as the

error_detected_before bool in OnOutputPacket().

The server sends this message when the stream processor is done

consuming the data in this packet (but not necessarily done processing

the data) and the packet can be re-filled by the client.

This is not sent for all packets when a new buffer_lifetime_ordinal

starts as in that case all the packets are initially free with the

client.

After receiving the available input buffer via this event, the stream

processor client can call later call QueueInputBuffer with appropriate

offset and length set, with the same packet_index, to re-use the

packet_index.

OnFreeInputPacket() does _not_ imply that the data in the input packet

has been processed successfully, only that the input data is no longer

needed by the StreamProcessor. If a client needs to know which input

data has generated corresponding output, using timestamp_ish values for

that is recommended.

Any reliance on the relative order of OnFreeInputPacket() and

OnStreamFailed() is discouraged and deprecated. Instead, use

timstamp_ish values to establish which input packets generated

corresponding output packets. Note that even using timestamp_ish values

doesn't necessarily imply that the processing of input data with a given

timestamp_ish value is fully complete, as in some StreamProcessor(s) the

data derived from an input packet can be kept for reference purposes for

a long time (in general indefinitely) after the input data has generated

its primary output data (the output data to which the timestamp_ish

value is attached). The StreamProcessor interface currently does not

provide any way to determine when all data derived from an input packet

has been discarded by the StreamProcessor, and if such a mechanism is

ever added to the StreamProcessor protocol, it would be an optional

StreamProcessor capability, since it would be infeasible to implement

for some StreamProcessor implementations that rely on external means to

process data, where the external means won't necessarily provide info

regarding when an input packet's derived data is fully discarded. An

input packet's derived data will never generate or contribute to any

output data for a different stream.

The order of OnFreeInputPacket() is not guaranteed to be the same as the

order of QueueInputPacket(). Any reliance on the order being the same

is strongly discouraged and deprecated. Clients are expected to work

properly even if the order of OnFreeInputPacket() messages is

intentionally scrambled with respect to each other (but not scrambled

across OnStreamFailed(), for now).