-
Notifications
You must be signed in to change notification settings - Fork 306
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Performance] Prioritised TensorDict replay buffers use for loops over the batch dimension #1574
Comments
Any update on this? Can I help? |
I am not super sure about all the cases that were envisioned when the component was created this way or why we were doing this. |
We just wanted to be able to handle lists of tensordicts i guess, and it should work ok with a list storage and stuff that you can't necessarily stack well, no more insight than this I'm afraid. |
i don't think this is currently the case as the while furthermore, if i can try to do some patch work to make |
If the method is broken with list it won't be bc-breaking if we don't support lists anymore, and any fix that enables the passing of lists to extend will be a "new feature" (since now it isn't a feature). |
Ok I'll do this in #1598 |
In prioritised tensordict replay buffers, the
update_tensordict_priority
, method performs a for loop over the batch dimensionrl/torchrl/data/replay_buffers/replay_buffers.py
Lines 760 to 763 in 434fe58
This causes significant slowdowns as this is the vectorised dimension used in the training pipelines and can get to really high sizes.
This method is called every time the buffer is extended or the priorities are updated.
The text was updated successfully, but these errors were encountered: