[Feature] MCTS Scoring functions

[vmoens]

Stack from ghstack (oldest at bottom):

• -> [Feature] MCTS Scoring functions #2358
• [Feature] TensorSpec.enumerate() #2354
• [Feature] MCTSForest #2307

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2358

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 21 New Failures, 6 Unrelated Failures

As of commit 20d685f with merge base 4736fac ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  E ImportError: cannot import name 'ListStorage' from partially initialized module 'torchrl.data' (most likely due to a circular import) (/home/runner/work/rl/rl/torchrl/data/__init__.py)
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  E ImportError: cannot import name 'ListStorage' from partially initialized module 'torchrl.data' (most likely due to a circular import) (/__w/rl/rl/torchrl/data/__init__.py)
• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 7fbb847840a962dd3d330eaf27c8ba5bd01f63024641534fb87d8d165e3d7f21 /exec failed with exit code 1
• Generate documentation / build-docs (3.10, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t f582746f07a83c8ac64309b2390996753358bac44fc7ccedcac7971e2ee83ddd /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 5c7d85425f8dde7f7ba8b033b8b77845856ff5cb9a088dd4f6563278146fd089 /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t f1598ac392dce9c99b8152e30d813e1f4dd9b7479028f17649bbbd17fcb8b2c9 /exec failed with exit code 1
• Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 044d8bf4bda95b8b6b1d8028b4263fb1c0da5d6bc9fbe146c3dfd0c86d759f78 /exec failed with exit code 1
• Lint / python-source-and-configs / linux-job (gh)
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t daecc13f886de287873a4343923bbedf625ea6f2f7b0d374ccf8547e927c5834 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  RuntimeError: Command docker exec -t 1fccc7371c2b6506054adb3b90e12b9bc592eb2d3c313d760e4e9b956b05e98c /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  RuntimeError: Command docker exec -t ffd8e7878f3fc8e5f12889f483a8444eb12234753f996cecda22c824334a0ef1 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  RuntimeError: Command docker exec -t 30d8dde4292e56063488890de1f59575ae6df2088f526e63c1c7bb04a319702c /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  RuntimeError: Command docker exec -t 7eb4c80ee78746d2a51af76ef3312327e231bb655976786455e7f9ace1133d57 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu-oldget (3.12) / linux-job (gh)
  RuntimeError: Command docker exec -t 78fcb78d28354b091944a98774757bd014726d7fbfac8e6d465c153bb1910726 /exec failed with exit code 1
• Unit-tests on Linux / tests-gpu (3.11, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 6d8ab2d87a6e93c10ddc0b2457333e07cab13217b1f90d152e41b956ee7aa96f /exec failed with exit code 1
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 95e40e8ab4ca823a8f861645127c3a16362216e08d0c6713144c1e18e7398462 /exec failed with exit code 1
• Unit-tests on Linux / tests-optdeps (3.11, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t c12c2c95f0c6b9e7499870cdff6f43801e1ceee5cf49982120a2820c63749109 /exec failed with exit code 1
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 2089ad7c12ef97b59ad47b2457202ad99d20ec3313c42733a6c2347e99836932 /exec failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  ImportError: cannot import name 'ListStorage' from partially initialized module 'torchrl.data' (most likely due to a circular import) (C:\actions-runner\_work\rl\rl\pytorch\rl\torchrl\data\__init__.py)
• Wheels / test-wheel-windows (3.11) (gh)
  ImportError: cannot import name 'ListStorage' from partially initialized module 'torchrl.data' (most likely due to a circular import) (C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\torchrl\data\__init__.py)
• Wheels / test-wheel-windows (3.9) (gh)
  ImportError: cannot import name 'ListStorage' from partially initialized module 'torchrl.data' (most likely due to a circular import) (C:\hostedtoolcache\windows\Python\3.9.13\x64\lib\site-packages\torchrl\data\__init__.py)

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_4 (gh) (similar failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Wheels / test-wheel-windows (3.10) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.
• Wheels / test-wheel-windows (3.12) (gh) (matched win rule in flaky-rules.json)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cpu (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda11_8 (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow
• Build Windows Wheels / pytorch/rl / upload / wheel-py3_9-cuda12_1 (gh) (trunk failure)
  ##[error]Unable to find any artifacts for the associated workflow

This comment was automatically generated by Dr. CI and updates every 15 minutes.