Population of "node name to parameter descriptor map" needs to be defined

[bzbarsky]

https://webaudio.github.io/web-audio-api/#node-name-to-parameter-descriptor-map says:

This internal storage is populated as a consequence of calling the registerProcessor() method in the rendering thread.

but the registerProcessor steps don't describe how this population happens. There's a bunch of code in Gecko that does this, converting things to AudioParamDescriptor values, doing sanity-checks on their members, etc, but none of that seems to be described by the spec.

@karlt

[karlt]

Thank you for filing.

The hint that these values are converted to AudioParamDescriptor comes from
https://webaudio.github.io/web-audio-api/#parameterdescriptors:
"Users can define a custom audio processor by extending AudioWorkletProcessor. The subclass [...] may have a valid static property named parameterDescriptors which is an iterable of AudioParamDescriptor."

"static property" sounds like an ECMAScript concept, but "AudioWorkletProcessor" and "AudioParamDescriptor" are IDL concepts.
This is tangled with #1945 and #1947

registerProcessor(name, processorCtor) is an IDL method, but much of https://webaudio.github.io/web-audio-api/#dom-audioworkletglobalscope-registerprocessor involves ECMAScript operations. I wonder whether it would be practical to use IDL concepts for registerProcessor.

The code for sanity checks on the members derives from https://webaudio.github.io/web-audio-api/#dictionary-audioparamdescriptor-members
There is only a single exception value and so the order among those checks is not observable, but the spec is not clear re when this set of checks is performed. It doesn't seem practical to have inter-member constraints applied at the time of initializing the individual members. Conversion from ECMAScript Object values initializes members in lexicographical order.

[bzbarsky]

The hint that these values are converted to AudioParamDescriptor comes from

Sure, but given that the conversion can have side-effects, throw exceptions, etc, its exact ordering with other observable specification steps needs to be clearly defined.

https://webaudio.github.io/web-audio-api/#dictionary-audioparamdescriptor-members

This should probably be an algorithm that is explicitly invoked from the places where such validation is expected to take place. Again, since it has observable behavior (throwing) its ordering with other observable behavior needs to be specified.

There is only a single exception value and so the order among those checks is not observable

Sort of. The non-specced messages on the exceptions are observable. But you're right that in terms of the concepts that are standardized these checks can happen in any order.

but the spec is not clear re when this set of checks is performed

Yep.

It doesn't seem practical to have inter-member constraints applied at the time of initializing the individual members

It's not, correct. This needs to be a separate explicit step after the dictionary conversion.

In the case of the "sequence of dictionaries" situation, it's critical to define whether this check is applied after each dictionary conversion or whether the entire sequence<AudioParamDescriptor> is built up and then validated. My suspicion is that the latter is much simpler to spec; you can invoke https://heycam.github.io/webidl/#es-to-sequence to do all the ES-related bits (as long as you're OK with arbitrary iterables, not just arrays) and then validate the resulting sequence.

[hoch]

Thanks for your feedback. It would've been a lot better if we had this discussion two years ago.

I am concerned about mixing concepts from ECMAScript and IDL. You need IDL to describe the interface, but also you need ES concepts for the algorithm. Is there any clear boundary? FWIW, Our spec work here is largely based on CSS Paint API.

The rough algorithm on the population step would be:

1. Let descriptorsValue be the result of getParameterDescriptors().
2. Let descriptorSequence be the result of conversion from descriptorsValue to sequence<AudioParamDescriptor>.
3. For each element in descriptorSequence:
   1. Check if element conforms the type of AudioParamDescriptors. Otherwise, throw an exception.

[bzbarsky]

It would've been a lot better if we had this discussion two years ago.

I wasn't aware of this spec section until I was asked to review it (or rather some code implementing it) a few days ago...

Was there a TAG review for this spec section? I'm somewhat sad that this wasn't caught there, if so.

Is there any clear boundary?

It's a hard problem, in general. In theory, you can do everything with ES concepts plus infra types (or even ES List and whatnot), as long as you're careful. IDL exists to factor out common patterns of ES concept usage into more easily reusable pieces. So to that extent there is no clear boundary: every so often we add a new thing to IDL that reifies a commonly desired set of ES concept steps.

That said, if webassembly is going to be doing direct IDL-type access, there is some value in recasting things that don't need to rely on ES concepts in terms of IDL concepts which might have a non-ES webassembly reflection. Maybe.

IDL might end up growing a "class" concept, because several places do want that: pass in the constructor, and then grab stuff from its prototype, etc. It's not there yet. It could be simulated by passing in an object in the original IDL which you then use to manually initialize both a Web IDL callback (for storing the thing you plan to call) and a Web IDL dictionary with a "prototype" property whose value is another dictionary with the things you want off the prototype, with relevant IDL types. So "process" would be a Function, say. And "parameterDescriptors" could be a sequence<AudioParamDescriptor> member of the outer dictionary, etc. It's a fair amount of boilerplate and not obviously clearer than doing the ES bits that are there now.

largely based on CSS Paint API

I'd love a pointer to the section this is based on, in case it has similar issues.

Let descriptorsValue be the result of getParameterDescriptors().

So this is the Get(O=processorCtor, P="parameterDescriptors") step, right?

If so, then the proposed algorithm sounds fine, aassuming "Check if element conforms the type" means doing the checks described for the various members under https://webaudio.github.io/web-audio-api/#dictionary-audioparamdescriptor-members. This would need to be made clear.

[hoch]

First of all, I really appreciate you taking time to review the section. Also thanks for the explanation on the boundary between IDL and ES.

Was there a TAG review for this spec section? I'm somewhat sad that this wasn't caught there, if so.

TAG review (round 2): w3ctag/design-reviews#212

I'd love a pointer to the section this is based on, in case it has similar issues.

registerProcess() is largely from registerPaint() and it has diverged later due to the complexity of Audio Worklet system. However, I followed the general pattern of it when I wrote the AW's algorithm. I am sure that registerPaint() algorithm also changed a lot, so I don't know how much of difference we have now.

So this is the Get(O=processorCtor, P="parameterDescriptors") step, right?

Yes.

If so, then the proposed algorithm sounds fine, aassuming "Check if element conforms the type" means doing the checks described for the various members under https://webaudio.github.io/web-audio-api/#dictionary-audioparamdescriptor-members. This would need to be made clear.

Good to hear. But could you point an example of such case? Is there any pre-defined algorithm that I can use for the conformity check? (Object -> AudioParamDescriptor) Or should I just check it manually by going through the keys in the object?

[hoch]

Just to set up the order: this is blocked on #1947.

[bzbarsky]

TAG review (round 2): w3ctag/design-reviews#212

Thank you.

is largely from registerPaint()

Again, thank you. I'll take a look at that too.

But could you point an example of such case? Is there any pre-defined algorithm that I can use for the conformity check?

There's no pre-defined algorithm. If I were writing this spec, what I would probably do is:

1. Convert the ES Value to an AudioParamDescriptor.
2. Define an algorithm that operates on AudioParamDesctiptor instances and implements the checks defined in https://webaudio.github.io/web-audio-api/#dictionary-audioparamdescriptor-members. No opinion on whether the algorithm should throw exceptions or return a boolean and have callers throw exceptions.
3. Invoke that algorithm as needed on the relevant AudioParamDescriptor instances.

[hoch]

(revised based on #1946 (comment))

This is an expansion of the step 8 in registerProcessor():

1. Let descriptorList be the result of Get(O=processorCtor, P="parameterDescriptors").
2. Let descriptorSequence be the result of conversion from descriptorList to a value of type sequence<AudioParamDescriptor>.
3. For each descriptor of descriptorSequence:
   1. Let defaultValue be the value of AudioParamDescriptor.defaultValue member.
   2. Let minValue be the value of AudioParamDescriptor.minValue member.
   3. Let maxValue be the value of AudioParamDescriptor.maxValue member.
   4. If defaultValue is less than minValue or greater than maxValue, throw a TypeError.

[bzbarsky]

"conversion" should link to https://heycam.github.io/webidl/#es-to-sequence, no?

Use the validation algorithm when converting a value from descriptorList to an instance of AudioParamDescriptor.

That makes it impossible to use existing "convert-to-sequence" code, because this has observably different behavior, right? Why is that desirable? It's basically monkeypatching https://heycam.github.io/webidl/#es-to-sequence

It seems to me that it would be better to do the conversion to sequence per the normal rules, then validate all the items in the sequence.

If the validation algorithm throws an exception, handles the exception, throw a TypeError and abort the rest.

I'm not quite sure what this is trying to say. The validation algorithm already throws a TypeError. What does it mean to "handle" it and why is a different TypeError then thrown?

[hoch]

I made some edits based on your suggestions: #1946 (comment).

[bzbarsky]

That looks pretty reasonable to me, thank you.

[hoch]

Thanks for your input @bzbarsky!

[hoch]

Closed by #2012.