Update the text to reflect the resolution on GC observability #1810

padenot · 2019-01-15T13:23:30Z

This is largely based off of @karlt's prose, with minor adjustments, and fixes a bunch of things. The commit that is the most important is the second one.

This fixes #1471.

Preview | Diff

hoch

First pass. Overall, it looks good but I have few comments.

audionode.include

hoch · 2019-01-17T20:49:07Z

index.bs

@@ -2581,6 +2581,37 @@ value explicitly. <em>block-size</em> is defined to be 128
 sample-frames which corresponds to roughly 3ms at a sample-rate of
 44.1 kHz.

+<h4 id="AudioNode-tail">AudioNode tail-time</h4>
+
+An {{AudioNode}} can have a <dfn>tail-time</dfn>. This means that even when the


Perhaps this can be:

All AudioNodes have a tail-time. When an AudioNode has a tail-time greater than zero, it can produce non-silent output even when it has silent input.

I'm rephrasing this. I'm not a fan of saying that all nodes have X, but X has no consequences for most of the nodes.

hoch · 2019-01-17T21:00:12Z

index.bs

+
+<h4 id="AudioNode-actively-processing">AudioNode lifetime</h4>
+
+{{AudioNode}} can be <dfn id="actively-processing">actively processing</dfn>


Perhaps this can be:

An AudioNode accessed during a render quantum is referred to as actively processing. When an AudioNode is not actively processing, it MUST generate a single channel of silence as an output. This state of "actively processing" varies on the type of AudioNodes:

I find this to be too vague. When we're checking whether an AudioScheduledSource is supposed to write something in its output, we're somehow accessing it, aren't we?

Then how about "being rendered" instead of "accessed"?

As a definition, this sentence basically ends without defining what actively processing. Maybe say that it's actively processing if the any of the following conditions hold? This probably also means that the links to "actively processing" shouldn't be links.

index.bs

hoch

Before we sign off on this, I would love to have this reviewed by @karlt and @rtoy as well.

hoch · 2019-01-18T18:49:52Z

index.bs

+
+<h4 id="AudioNode-actively-processing">AudioNode lifetime</h4>
+
+{{AudioNode}} can be <dfn id="actively-processing">actively processing</dfn>


Then how about "being rendered" instead of "accessed"?

index.bs

rtoy · 2019-01-23T23:54:15Z

index.bs

+
+<h4 id="AudioNode-actively-processing">AudioNode lifetime</h4>
+
+{{AudioNode}} can be <dfn id="actively-processing">actively processing</dfn>


As a definition, this sentence basically ends without defining what actively processing. Maybe say that it's actively processing if the any of the following conditions hold? This probably also means that the links to "actively processing" shouldn't be links.

karlt

Thank you, @padenot for writing this up. This captures what I had in mind.

index.bs

karlt · 2019-02-02T02:38:46Z

index.bs

+- All other {{AudioNode}}s start [=actively processing=] when any
+	{{AudioNode}} connected to one of its inputs is [=actively processing=], and
+	stops [=actively processing=] when the input that was received from other
+	[=actively processing=] {{AudioNode}} no longer affects the output.


Referring to the tail-time is fine with me if that's clearer. See my comment on tail-time for an attempt to more clearly describe this there. We need to be a little careful to avoid the two definitions becoming too cyclic.

"the input that was received from other" is no longer required if using tail-time. Perhaps:

"When there are no longer any actively processing AudioNodes connected to its inputs, an AudioNode continues actively processing for the length of its tail-time. The AudioNode stops actively processing when its tail-time expires before it again has any actively processing AudioNodes connected to its inputs."

index.bs

padenot · 2019-02-19T14:17:40Z

I've addressed all comments, often with something based of off @karlt's suggestions.

padenot · 2019-02-19T18:21:54Z

This is now green on travis, but I pushed this before pushing the workaround in 354a193.

rtoy

Still looks really good. Just some minor comments and some corner cases I think we need to consider.

rtoy · 2019-02-20T15:34:23Z

index.bs

+	that is <code>true</code>.
+- A {{DelayNode}} in a cycle is [=actively processing=] only when the absolute value
+	of any output sample for the current [=render quantum=] is greater than or equal
+	to \( 2^{-126} \).


This is a really, really tight constraint. Let's say you have a biquad filter in the loop. It's possible, due to round-off that the biquad never produces 0 output for 0 input.

Not sure if this is achievable in practice.

I wonder whether you may have missed my reply to your previous comment about this.
#1810 (comment)

The reason for choosing FLT_MIN here rather than FLT_TRUE_MIN is that the output does not need to reach zero.

Because full precision is available at FLT_MIN, the round-off occurring with values around FLT_MIN is similar to round-off at larger values. If the decay (biquad or other) is such that input of any magnitude will attenuate, then values around FLT_MIN will attenuate. After values around FLT_MIN attenuate, the cutoff is passed and the node outputs a single channel of silence.

Regardless of the value of the cutoff, perhaps there may be a chance of instability due to the sharp gradient at the cutoff if a biquad does not attenuate that sufficiently. I don't know whether or not the output from such a biquad would decay to zero without the cutoff. Even if the cutoff is introducing some new instability, it would be preferable that any remaining non-zero output from this corner-case would be determined by the precision of float32 rather than an arbitrary cutoff.

This is acceptable to me. I still find the constraint very tight. Does Firefox stop a BiquadFilter tail when it reaches FLT_MIN? Chrome just estimates the tail time by computing when the impulse response is about 60 dB down (I think). Even then, there are filters whose tail time is in the thousands of seconds.

I wouldn't say RT60 is an arbitrary cutoff. It is considered the industry standard for reverberation decay. I think it has been accepted as a reasonable threshold for "perceptually inaudible" sound.

Can we make it up to UA? I think the numeric difference between RT60 and FLT_MIN is quite significant, but the audible difference would not really big. In that case, UA might be able to try different optimization techniques to improve the rendering performance.

index.bs

rtoy · 2019-02-20T15:41:52Z

index.bs

+- All other {{AudioNode}}s start [=actively processing=] when any
+	{{AudioNode}} connected to one of its inputs is [=actively processing=], and
+	stops [=actively processing=] when the input that was received from other
+	[=actively processing=] {{AudioNode}} no longer affects the output.


"no longer affects the output" is rather vague. Say you have a constant source with a constant value of 1 that is actively processing. Say a 4x WaveShaper follows the source and the wave curve is the identity, more or less.. After a while, the constant source no longer affects the output because it too is a constant. Does this mean the WaveShaper is no longer actively processing? If so, the sentence below says it must output a single channel of silence. That would be wrong since it should continue to output a constant value.

Perhaps we should say it stops actively processing if the inputs are not actively processing and the tail-time has passed?

The input affects the output, because if the input were to stop or change in value, then the output would change.

What if I have a gain node with a gain of 0? The input doesn't affect the output.

I'm not sure, but maybe what I'm looking for is that a nodes stops actively processing when all inputs are no longer actively processing?

index.bs

rtoy

Just a bunch of nits

index.bs

rtoy · 2019-02-25T16:00:01Z

index.bs

+- All other {{AudioNode}}s start [=actively processing=] when any
+	{{AudioNode}} connected to one of its inputs is [=actively processing=], and
+	stops [=actively processing=] when the input that was received from other
+	[=actively processing=] {{AudioNode}} no longer affects the output.


What if I have a gain node with a gain of 0? The input doesn't affect the output.

I'm not sure, but maybe what I'm looking for is that a nodes stops actively processing when all inputs are no longer actively processing?

hoch

Some suggestions for clarification.

index.bs

rtoy

A few minor nits. Otherwise I'm happy with these changes once the nits are fixed.

rtoy · 2019-04-15T20:48:27Z

index.bs

@@ -2785,6 +2785,52 @@ enum ChannelInterpretation {
 </table>
 </div>

+<h4 id="AudioNode-tail">AudioNode tail-time</h4>


Nit: Maybe capitalize as "AudioNode Tail-Time".

rtoy · 2019-04-15T20:50:57Z

index.bs

+may continue to produce non-silent output for the calculated tail-time even
+after the input transitions from non-silent to silent.
+
+<h4 id="AudioNode-actively-processing">AudioNode lifetime</h4>


Nit: Capitalize as "AudioNode Lifetime".

rtoy · 2019-04-15T20:51:16Z

index.bs

@@ -4034,6 +3999,11 @@ the source MUST then output silence (0).
 {{AudioScheduledSourceNode}} cannot be instantiated directly, but
 is instead extended by the concrete interfaces for the source nodes.

+An {{AudioScheduledSourceNode}} is said to be <dfn>playing</dfn> when
+its associated {{BaseAudioContext}}'s {{BaseAudioContext|currentTime}} is
+greater or equal to the time it's set to start, and lesser than the time it's


"it's set to start" -> "AudioScheduledSourceNode is set to start"
"lesser than the time" -> "less than the time"

hoch

LGTM with a nit.

hoch · 2019-04-18T01:45:35Z

index.bs

+	that is <code>true</code>.
+- A {{DelayNode}} in a cycle is [=actively processing=] only when the absolute value
+	of any output sample for the current [=render quantum=] is greater than or equal
+	to \( 2^{-126} \).


I wouldn't say RT60 is an arbitrary cutoff. It is considered the industry standard for reverberation decay. I think it has been accepted as a reasonable threshold for "perceptually inaudible" sound.

Can we make it up to UA? I think the numeric difference between RT60 and FLT_MIN is quite significant, but the audible difference would not really big. In that case, UA might be able to try different optimization techniques to improve the rendering performance.

…ssing" concept

…time

padenot · 2019-04-29T12:22:15Z

The issue here is that there can be arbitrary processing after the delay, including very high gain, unlike a normal reverb. Any difference can be amplified to a point where it's really different.

padenot added the In PR Review label Jan 15, 2019

padenot added this to the Web Audio V1 milestone Jan 15, 2019

padenot self-assigned this Jan 15, 2019

padenot requested a review from hoch January 15, 2019 13:23

hoch reviewed Jan 17, 2019

View reviewed changes

hoch reviewed Jan 18, 2019

View reviewed changes

hoch requested a review from rtoy January 18, 2019 18:52

rtoy requested changes Jan 23, 2019

View reviewed changes

hoch mentioned this pull request Jan 31, 2019

[audioworklet] Permit process() calls to be skipped on inactive AudioWorkletProcessors with silent input #1453

Closed

karlt approved these changes Feb 2, 2019

View reviewed changes

padenot force-pushed the 1471-gc branch from 19626ae to a1c128a Compare February 19, 2019 14:17

rtoy reviewed Feb 20, 2019

View reviewed changes

padenot force-pushed the 1471-gc branch from a1c128a to e782990 Compare February 25, 2019 13:22

rtoy reviewed Feb 25, 2019

View reviewed changes

hoch reviewed Feb 26, 2019

View reviewed changes

padenot force-pushed the 1471-gc branch from e782990 to 9f4654a Compare April 11, 2019 14:25

padenot requested a review from rtoy April 15, 2019 09:24

rtoy approved these changes Apr 15, 2019

View reviewed changes

hoch approved these changes Apr 18, 2019

View reviewed changes

padenot added 9 commits April 29, 2019 14:15

Add a new section about AudioNode tail-time

36a3c9f

Add the new AudioNode lifetime section, based off @karlt's prose

b7e4628

Fix tail-time reference references throughout

9ae5992

Remove previous AudioNode lifetime section

07c71e8

Replace the "active processing condition" flag by the "actively proce…

a1de282

…ssing" concept

Update references to the old AudioContext lifetime and AudioNode life…

ff4dd51

…time

Address rtoy comments, often by applying karl's suggestions.

572268c

Address rtoy's comments

d1dce34

Address more comments

1ae15ba

Address more comments

ef592a5

padenot force-pushed the 1471-gc branch from 9f4654a to ef592a5 Compare April 29, 2019 12:23

padenot merged commit f47c7f5 into WebAudio:master May 9, 2019


		<h4 id="AudioNode-actively-processing">AudioNode lifetime</h4>

		{{AudioNode}} can be <dfn id="actively-processing">actively processing</dfn>

Update the text to reflect the resolution on GC observability #1810

Update the text to reflect the resolution on GC observability #1810

Uh oh!

Conversation

padenot commented Jan 15, 2019 • edited by pr-preview bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hoch left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hoch left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karlt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

padenot commented Feb 19, 2019

Uh oh!

padenot commented Feb 19, 2019

Uh oh!

rtoy left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rtoy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hoch left a comment

padenot commented Jan 15, 2019 •

edited by pr-preview bot

Loading