AI Music Segmentation as Filter

[YeonV]

Thx for this amazing project!!

Hi Bros!

I'm a total noob when it comes to audio-technology, so please forgive me if this idea is stupid.

1. Is it possible to apply a filter on a stream-input or device-output?
2. if the filter's processing time is known, can I just delay all other devices by that time?
3. Can this filter be a "pulseaudio-module" or a "vst"? (omg most likely the noobest question of all time)

Why?

Open Source AI Music Segmentation:
Run example in Colab
Source: Deezer - Spleeter

Next-level: Realtime
VST
pulseaudio module

-> then maybe we could choose this filter on a device-level [Vocals / drums / bass / piano / other]

Final Goal

Map LED-Strips to "instruments" while doing audio-reactive-effects
(Im the frontend developer of LedFx)

[vjdw]

Sounds interesting! I don't know if this would work but one way might be to run an instance of snapclient dedicated to spleeter, that snapclient can use the --player command to output the audio to stdout (or alternatively maybe a named pipe would work?) and spleeter might be able to use that as its input?

Each snapclient can have latency configured. I haven't tried this but you might be able to set a negative latency on the spleeter snapclient (instead of having to set a postive latency on every other snapclient).