Enhance SAC with Mixture-of-Expert and BEE Operator for Improved Stability and Performance

[XuGW-Kevin]

This PR introduces two plug-and-play enhancements to the Soft Actor-Critic (SAC) algorithm, inspired by recent advancements in reinforcement learning research.

The integration is based on insights from several papers:
https://arxiv.org/abs/2306.02865
https://arxiv.org/abs/2402.08609
https://arxiv.org/abs/2410.14972

To maintain robustness and practicality, I excluded experimental techniques (e.g., dormant ratio, perturbing network weights, and dynamic hyperparameter tuning) that have not yet stood the test of time. The retained methods are relatively well-validated and provide substantial improvements in both stability and performance.

Key Features:
(1) Mixture-of-Expert Network
(2) Blended Exploration and Exploitation (BEE) Operator

This enhancement aims to provide a more performant SAC-based RL baseline algorithm.
Feedback and suggestions are welcome!

[StoneT2000]

nice work! I'll review this next week probably (busy with icml/rss)

[XuGW-Kevin]

Thank you so much! Good luck with ICML/RSS!