channel count changes in filter nodes with tail time

[karlt]

Consider a graph where a filter node with a tail-time initially has a mono
input, then a stereo input is added, then the stereo input is removed.

In order to avoid glitches in the audio signal, implementations should take
care to up-mix the tail when the stereo input is added, and continue to
produce stereo output for a the tail period after the stereo input is removed.

It may be helpful to mention this in the spec.

[karlt]

#25 discussed DelayNode, but
did not discuss BiquadFilterNode, IIRFilterNode, and WaveShaperNode.

DynamicsCompressorNode may be similar to DelayNode.

Since https://github.com/WebAudio/web-audio-api/pull/1176/files this also
affects ConvolverNode.

[karlt]

For a mono signal m, transient stereo signal s, filter node with linear
transform f(), and + to denote mixing, we would expect

f(m + s) = f(m) + f(s)

if mixing is always performed with the same channelInterpretation.

In order to achieve this, the node would need to up-mix the mono signal and
its tail whenever mixing with effects of the stereo signal.

[karlt]

Graphs with changing channelInterpretation don't need to be a priority IMO,
but they do complicate things somewhat. Usually up-mixing occurs at input to
a node. If a stereo signal arrives at a node with an existing tail from mono
signal, it would be simpler to up-mix the mono tail based on the
channelInterpretation at the time the stereo signal arrived, than to recall
the history of channelInterpretation for the mono input and up-mix the
mono tail using a mix of channelInterpretations.

[rtoy]

+1 to everything you said. I think it's kind of implied in tail-time, but we should be more explicit about it.

Chrome currently does this incorrectly, even in the case where we're not processing the tail. If you have a stereo input and a mono input, then take away the stereo input, Chrome's biquad filter suddenly produces mono output, effectively killing off the tail that should be there for the stereo signal.

[karlt]

Thanks, rtoy. Gecko has the same bug: https://bugzilla.mozilla.org/show_bug.cgi?id=924718

[padenot]

TPAC 2018 resolution: the channel count on the output takes into account the tail of the node, as described in this issue.

[padenot]

I'm trying to determine whether it's better to spec this globally or for each node. It might be enough to state something like:

If a node has a tail time, any change in channel count, channel count mode and channel interpretation MUST be reflected in the output of the node after the tail time has elapsed.

[hoch]

Perhaps you meant until instead of after? We want to keep the configuration until the node consumes its tail time. Right?

[karlt]

after is close to appropriate for DelayNode, where input has no effect on output until the delay time has passed, but there is also the complication that channel counts can only change on rendering quantum boundaries. See #25 (comment)

"Output channel counts remain until the tail time passes" would be close to appropriate for decreases in channel count. For increases in channel count, however, changes in input channel count will need to be applied immediately to output channel count for some nodes such as ConvolverNode.

The principal for all filter nodes is that, where a rendering quantum of output is derived from input from multiple rendering quanta, the channel layout in the output is a superset of the input channel layouts for each input quantum.
Up-mixing is used to convert layouts. Down-mixing never occurs because this would lose information and so produce glitches when it starts and stops.

[padenot]

I'm thinking maybe something like you last paragraph would be accurate enough.