Is there a way to disable audio port output processing for specific ports?

[spongeboburu]

Hi! I've been writing a few simple, host implementations and thought it'd be fun to do something more complex such as a graph/node style connection between plugins. If I understand the docs the process audio outputs and inputs map audio buffers 1:1 with the number of ports the plugin has.

Ideally there would be a member in process, perhaps bool *skip_audio_output; // Matches the audio_outputs array. Two workarounds
I've been playing with is to set both audio buffer data32 and data64 members to NULL, but the documentation doesn't explicitly say this is allowed. The workaround is to set channel count to 0, but I'm not sure if that is valid either. Using plugin template as a reference it doesn't check the channel count of the audio buffer, it just assumes 2. Then again, the plugin also does not check if data32 is NULL either, so this may just be that the plugin template is ill-behaved in making these assumptions?

Also it seems the documentation in process.h has a typo for audio inputs/outputs, it refers to get_count and get_info, but the clap_plugin_audio_ports members are named simply count and get.

Thanks for taking time to read through all of this!

[baconpaul]

Why would you skip an input or an output the host asks for? I’m not sure I understand the goal

If a plugin wants it can ask the host to rescan its ports if topology changes using the audio ports host extension. And if a host wishes to advertise multiple topologies to a host we have the audio ports config extension also, but I am not sure if any host or plugin implements that today.

But I also know that surge and clap saw demo will both crash or fail if you don’t give them the buffers you ask for

thank you for spotting that typo thoigh! You know now the names those apis had in clap 0.06 :) we should fix that up for sure though (or if you want to feel free to send a pr against the next branch fixing it )

[spongeboburu]

Basically, what's preventing the plugin from rendering frames for all ports even if they are not needed for the given topology? For example, a plugin has 2 output ports, but only 1 is connected to the next plugin. The same goes for the number of channels, the plugin should honour rendering frames for the selected ports and intended number of channels only, otherwise the plugins will process frames that will simply be thrown away. And that is the issue, the documentation and the reference plugin does not show any way of asking the plugin to render only selected ports and a specified number of channels.

I would like to help mending the documentation, I'll get on it tomorrow morning.

Edit: Regarding the channels. If a plugin output port has stereo channels, but the receiving plugin input port has mono channel the output plugin should only render frames for 1 channel. The spec could even specify how that is to be done, for instance enforce multi channel expansion or reduction by mixing. But, the easiest aproach IMHO is that only the number of channels specified in the buffer audio channels_count is actually rendered by the plugin. If you have a plugin that does left-right channels to an audio buffer that is mono you'll only get the left.

[baconpaul]

So in that case a clap plugin would advertise mono to stereo mono to mono and stereo to stereo as port configs and the host could select one. But there’s no concept of “hooked up” in plugin apis there’s just a concept of “bus configurations available”. And if a host makes an instance of a plugin in a configuration it has to provide the busses. And if a plugin doesn’t provide a configuration the host wants then the host shouldn’t instantiate it. Just like if you have a mono to mono only au logic won’t let you put it in a stereo track.

or maybe I’m missing your point? But the handshake around bus config happens at create time and there’s apis for the plugin to say it’s options and for the host to query then

[robbert-vdh]

FWIW VST3 does have this API through bus activation (the plugin can even request a bus to be activated). But I don't know if hosts and plugins actually make use of it.

[baconpaul]

The VST3 activation api is really not much (semantically) different than the audio-config and host side audio-ports extension though. The host can say 'what can you do' and can pick one; and the plugin can say 'i want to be this' then the host has to do that (or not but it can't half do that)

[robbert-vdh]

It is different in the sense that the plugin and/or host can know which busses are active/connected to something and which are not. That means you end up with n booleans for n busses/ports, as opposed to 2^n different port configurations. Whether that is worth the additional complexity it brings with it is another matter entirely of course.

[spongeboburu]

Yes, the point is that the host queries and the plugin (in extreme) says I'll do 8 output ports with each 16 channels and I don't really need that as a host, I'm only connecting you to a 1 port input with 2 channels. Nevermind the connection bit, that's of course handled by the higher level API. I think the host should dictate what ports and how many channels need rendering, but with the current api and documentation it suggests the plugin is free to render as many ports and channels as it wants and if the receiver isn't game it still has to provide buffers for all those ports and channels those frames are just wasted. I'll try to join the discord at some point because I am unable to describe this more clearly in text. Thanks for your time baconpaul, much appreciated!

Edit: and it's absolutely possible that I have just misunderstood how these things work entirely. :P
Also as a side note, if anyone is writing hosts (because currently we are lacking a host template), feel free to chime in and tell me if I'm completely off. Is anyone else trying to make a graph/node hierarchy with clap plugins? If you have solved this without allowing wasted frames to be rendered I am interrested! Even better, if you are interrested in trying to make one I'll publish what I have here on github, maybe we can combine our work somehow.

[baconpaul]

Loads of people in rust land are doing that yes. And the example host, DPF/Carla and QTractor are foss hosts as is, to a lesser extend, clap-info.

The plugin tells the host what its busses are, basically, is the API here (and in VST2, VST3, AU and LV2). If a host supports multiple bus configs it will advertise it. But if it doesn't you have to give the plugin what the plugin can handle, which is what it advertises.

We should have a good demo of this! But we don't.

[baconpaul]

"this" being the audio-config API and a multi-bus-config plugin with a preference.

[spongeboburu]

Here's the question again, if a plugin has 4 output ports and 6 channels on each port, but my host only wants one port and 2 channels, how can I convince the plugin to not render/process those other ports and channels that are not needed/requested? The documentation and examples don't support this scenario, the plugin simply assumes that you have the buffers to cover all the ports and channels, whether you want them or not.

I mean, it's easy to filter, by why render frames for the ports and channels that aren't used for the actual output?

[baconpaul]

The answer is: if the plugin is capable of running at 2x2 of 4x6 then it will advertise both configurations and dynamically change its processing loop.

If it isn't, it won't.

but for a host to desire that a plugin could render to a different topology than it advertises may be a common desire, but won't match the code in the plugin.

Does that help?

[baconpaul]

Or another way: The plugin tells the host what it can do, not vice versa.

[spongeboburu]

With the current API and reference implementation the host must handle whatever the plugin says it's doing. If the plugin has 4 ports with 6 channels each and the host is a single port mono output, allocating buffers and rendering all the frames for 4 ports with 6 channels each gives nothing to the host which only wants 1 port with a single channel. All those other frames are thrown away.

Paul, talk to you tomorrow it's getting late here. Hopefully we can chat in a less constricted environment than github. There's a discord or something right? In any case I wish you a good evening (or whatever your TZ is hehe). Laters!

[baconpaul]

Right. So the thing I invite you to consider is: what if that plugin can't make 1 port outputs?

If it can, it will advertise that.

If it can't - the host has to do something. "Not allow the plugin to participate in that graph" is what some hosts do. "Allow you to wire up a subset of inputs and outputs and copy those onto the bus" is another.

But I think your issue is really: Some plugin you want to use is 4x6 and you want it to be 1x1. The solution to that is go ask the plugin author to support 1x1!

And yeah discord works well! If you hit me up on surge discord I can invite you over to a good place to chat. Tag me in the 'clap-chatter' room https://discord.gg/fTEnwGyB and then we can DM and I'll help you find the right servers.

[spongeboburu]

Oh, I thought the clap community had some official channels aside from github already. I'm not good with the community communication sites, I have slack because of work, but generally stick to github and gitlab and even those I dont handle well XD.

I get what you're saying, but it would be easy for the plugin writers to adhere to the rule that you only generate what the hosts asks for and I think that would be the cleanest aproach. If you have written a 6 channel plugin it is quite easy to only make it render the first channel. Same with ports, it's easy to skip ports. It's such an easy branching/condition statement that it should probably be mandatory for plugins to be able to render less channels than their maximum capability and only on selected ports. IMHO! I'm pretty new to audio so I'm not even sure if this is is an issue. It seems like an edge case at least, but it comes up when you write the buffer allocation code etc. Why do I need a larger buffer than the actual host output? Is what I asked myself (and also the thrown away frames good for nothing in this scenario). The host may mix the superflous channels, but rendering unused ports is still an issue.

[baconpaul]

We do; I can invite you to them if you dm me on discord. I just gave you the easiest way to find me on discord :)

So I agree with you. The surge VST3 can work in either 0x2, 1x2, 2x2, 0x6, 1x6, or 2x6 mode. And hosts can find that. The CLAP only exposed 2x6 for now. But if you load the Surge VST3 in reaper it looks at the possible configurations and asks you which one you want.

Similarly, when you create a plugin in Logic you get to choose between all the available bus configurations. Here's a quick screenshot of instantiating pianoteq and surge audio units in LPX 10.

Now the surge team is a bunch of lazy slackers, who haven't implemented audio-config in their CLAP, so they only have one config available. But it they actually get off their butts and do some work, then a host could ask for all the configs and present a similar option. (I'm a dev on surge so I'm making fun of myself!)

But all the APIs work with the handshake that

Host asks plugin what its ports are
Plugin answers
Host picks one if there are several and instantiates

The thing CLAP adds is Plugin can also change the port map by notifying the host it wants to

But none of the audio apis have the host send (essentially) a random bus config to a plugin and tell the plugin 'good luck'. The plugin, not the host, is the source of bus configuration in the APIs and the job of the host writer is to give the plugin what it says it needs.

[baconpaul]

basically your proposal is "turn the bus negotiation on its head from a plugin says what it can do and a host adapts; to the host says what it must do and the plugin adapts". There's a lot of reasons to not do that (two of the biggest being 'thats not how any other API works' and 'plugin count >>> host count') but if you were starting the world from a blank slate its not crazy.

[spongeboburu]

Yep, I get it. Thanks for your effort!