[WASAPI Hints]Soundcard APOs cause channel issues on various devices when Enhancements are enabled

[Squall-Leonhart]

this is one of the legacy non-smpte devices that uses Rear Left and Right

When Enhancements disabled, the audio is output without problem,

A previous update fixed the rear channels so that Enhancements could remain on but the issue is back again as far back as Firefox 82.

Test site is: https://www2.iis.fraunhofer.de/AAC/multichannel.html

Previously APO enhancements could cause no audio or potential crashing: see https://bugzilla.mozilla.org/show_bug.cgi?id=971693

[Squall-Leonhart]

And yes, Edge/Chrome have the rear channels working just fine without disabling the enhancements.

[Squall-Leonhart]

I suspect mozilla/cubeb-coreaudio-rs#35 is involved.

[ChunMinChang]

this is one of the legacy non-smpte devices that uses Rear Left and Right

When Enhancements disabled, the audio is output without problem,

A previous update fixed the rear channels so that Enhancements could remain on but the issue is back again as far back as Firefox 82.

Test site is: https://www2.iis.fraunhofer.de/AAC/multichannel.html

Previously APO enhancements could cause no audio or potential crashing: see https://bugzilla.mozilla.org/show_bug.cgi?id=971693

Have you updated the driver? Is this possible a driver issue?

I suspect mozilla/cubeb-coreaudio-rs#35 is involved.

I don't think I understand the reasoning here. Could you describe why you suspect this update is the cause?

If this is a WASAPI issue, then I am pretty sure mozilla/cubeb-coreaudio-rs#35 is unrelated since it's used on macOS only. On Windows, the current mixer code is still cubeb_mixer.

Would you mind using mozregression to find out what commit is related? Or you can to test in cubeb individually by using git-bisect and run run_channel_rate_test to find out which commit causes the problem.

[Squall-Leonhart]

Have you updated the driver? Is this possible a driver issue?

There are no driver updates, the issue is caused by wasapi suggestions not working as expected on soundblaster drivers implementing creative labs APO/Enhancements for CMSS3D and Crystalizer.

I don't think I understand the reasoning here. Could you describe why you suspect this update is the cause?

I didn't realise it was osx only, i saw you mention the windows issue i posted and assumed it could be.

Would you mind using mozregression to find out what commit is related? Or you can to test in cubeb individually by using git-bisect and run run_channel_rate_test to find out which commit causes the problem.

I don't have the time to do mozilla's job for them anymore, if Paul Adenot is still with mozilla he can test it on the Soundblaster Z he originally reproduced the non-smpte issue with in https://bugzilla.mozilla.org/show_bug.cgi?id=1368938, also bumped that issue since the issue is returned, i only noticed cubeb has regressed and this issue returned because the firefox derivative i use pulled in updated cubeb and i thought to check it today, otherwise i wouldn't even be aware of it since i don't use browser for multichannel audio otherwise.

[ChunMinChang]

I saw the comment mentioning crash. Do you have any crash report in your about:crashes ? Or Are you able to launch Firefox with MOZ_LOG=cubeb:4 to get the logs? That may help us to investigate what’s happening.

[Squall-Leonhart]

there is no crash, the RR/RL plays no sound

was the Rust refactor ever brought to windows?

[ChunMinChang]

there is no crash, the RR/RL plays no sound

was the Rust refactor ever brought to windows?

No. The Rust refactoring is macOS-only.

Let's get the logs first to see what's going on. Would you mind navigating to your about:networking#logging page and set cubeb:5 to Current Log Modules? The log files could be found in what Current Log File is set.

[ChunMinChang]

this is one of the legacy non-smpte devices that uses Rear Left and Right

When Enhancements disabled, the audio is output without problem,

A previous update fixed the rear channels so that Enhancements could remain on but the issue is back again as far back as Firefox 82.

Is the Enhancements a Windows setting? or it's a feature developed for Sound Blaster X-Fi only?

Is this reproducible for other sound card? Or it can be reproduced by one sound card only? I am trying to figure out how difficult to get one soundcard that can reproduce this issue.

[Squall-Leonhart]

Is the Enhancements a Windows setting? or it's a feature developed for Sound Blaster X-Fi only?

its a checkbox on a tab of the sound device properties that is enabled when audio vendors ship their own APO's plugins, in this case it enables the CMSS3D or SBX TruSurround feature to work on stereo audio in music or videos to upmix them - this APO doesn't do anything to media already encoded with multichannel capabilities.

The cubeb logging doesn't appear to be working, file is empty.

I just noticed if i seek back 5-10s and play from there the rear speakers work with the enhancements on.

[padenot]

Here are instructions to get logs: https://docs.google.com/document/d/e/2PACX-1vRdFw9ojDRuikWaZeGO4A-uNkRAUtoTCtNqMkpk8h1iyfQDoLsDdIu0Ra8PxQF5dHYym8CMrdZEEis_/pub, the Firefox sandbox prevents the process from writing logs (as it should), so just disabling it for a session to get logs is necessary.

I'm going to try things out with the URL above and the Creative sound cards I still have, hopefully I can reproduce with at least one (which should be 5.1).

[Squall-Leonhart]

Hey Paul, the instructions in that are actually invalid on Windows, its looks like its intended to set the environment parameters for that instance of command prompt but you'e not actually passing SET, so the commands are returned as not recognised.

[padenot]

huh weird I thought I double-checked this, and it worked. I'll triple check, thank you very much for the heads-up.

[Squall-Leonhart]

Yeah, definitely not working for Powershell or CMD

SET MOZ_DUMP_AUDIO=1 & SET MOZ_DISABLE_CONTENT_SANDBOX=1 & SET MOZ_LOG=cubeb:5 & SET MOZ_LOG_FILES=firefox-logs

works

[Squall-Leonhart]

With doing that, the only file Firefox is generating on the desktop is audiostream-1

Attached here.
AudioStream-1.zip

MOZ_DISABLE_CONTENT_SANDBOX 1
MOZ_DUMP_AUDIO 1
MOZ_LOG cubeb:5
MOZ_LOG_FILES firefox-logs

are all present on about:troubleshooting
Perhaps i need to run nightly to get the firefox-logs?

[Squall-Leonhart]

i managed to get a moz.logs file saved to the desktop by starting logging on about:networking#logging, and restarting the browser, apparently the moz.log with cubeb specific logging only gets created after doing this restart, not when you press the button

alas, it doesn't present useful information to my eyes.

AudioStream-1.zip

I see you updated the document, but you only SET the first variable, you need to pass set after the &'s as well

[ChunMinChang]

The log looks normal. I don't see any error messages. I wish we have more logs around output mixer so we could have better understanding.

I suspect the cause is the same as BMO 1468381. The cubeb_mixer forces to use SMPTE layout when mixing the audio. The issue on macOS is fixed by cubeb-coreaudio-rs #23 but the new mixer is never imported to Windows (BMO 1368938).

Maybe we should get the output channel layout first to see if what's going on.

[ChunMinChang]

@Squall-Leonhart

Allow me to make the problem clearer. I feel the problem statement is vague.

The problem is that there are no sounds played from the rear-left and the rear-right channels of your 5.1 speakers when playing the 5.1 audio in the provided link. All other channels including front-left, front-right, centre, and low-frequency-effect work fine. Is that right?

its a checkbox on a tab of the sound device properties that is enabled when audio vendors ship their own APO's plugins, in this case it enables the CMSS3D or SBX TruSurround feature to work on stereo audio in music or videos to upmix them - this APO doesn't do anything to media already encoded with multichannel capabilities.

If I understand correctly, this APO thing is developed by Creative for its Sound Blaster series. It's not a common Windows audio setting or feature. The APO feature mentioned is used to upmix or downmix the audio. For example, if the audio file is stereo and you have audio 5.1 speakers, enabling this feature will do the audio upmixing. Is that right?

The audio in the above link is a 5.1 channel file. The problem you encountered here is that there is no sound coming from the rear-left and the rear-right channels of your 5.1 speakers when you play the audio in the above link with this APO thing enabled, right?

Since this APO thing is used to upmix or downmix the audio, it shouldn't take any effect here. The 5.1 audio should be played to the 5.1 speakers directly to the mapped channels. However, only front-left, front-right, centre, and low-frequency-effect channels work fine. No sound comes from the rear-left and rear-right channels.

BTW, is the APO tab mentioned something like what the video shows here? Do you know if the problem can be reproduced with the Super X-FI mentioned in the video? It seems it's a successor of CMSS3D.

Feel free to let me know if I misunderstand something.

[Squall-Leonhart]

all 3 front speakers and the LFE work.

the Super X-Fi might demonstrate the issue, Paul's Soundblasters have in the past.

The Enhancements(APO) checkbox is on the sound properties