Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor the metal backend to always reuse command encoders/buffers unless a shared memory access is requested #2037

Draft
wants to merge 44 commits into
base: main
Choose a base branch
from
Draft

Refactor the metal backend to always reuse command encoders/buffers unless a shared memory access is requested #2037

wants to merge 44 commits into from

Commits on Apr 10, 2024

  1. oh my god it works

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    b2ead34 View commit details
    Browse the repository at this point in the history

  2. fix other packages

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    adaf302 View commit details
    Browse the repository at this point in the history

  3. refactor metal backend to split storage and device files for readability

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    bb594bc View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' of github.com:huggingface/candle into CommandEnco… 

    …derReuse
    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    73088cc View commit details
    Browse the repository at this point in the history

  5. Revert "Merge branch 'main' of github.com:huggingface/candle into Com… 

    …mandEncoderReuse"

This reverts commit 73088cc, reversing
changes made to bb594bc.
    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    ba0aaa9 View commit details
    Browse the repository at this point in the history

  6. Revert "refactor metal backend to split storage and device files for … 

    …readability"

This reverts commit bb594bc.
    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    826b16d View commit details
    Browse the repository at this point in the history

  7. undo setting chnage

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    b1e09e6 View commit details
    Browse the repository at this point in the history

  8. undo messed up revert

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    90e6909 View commit details
    Browse the repository at this point in the history

  9. undo messed up revert

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    591e94c View commit details
    Browse the repository at this point in the history

  10. enable metal tracing option

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    0879f67 View commit details
    Browse the repository at this point in the history

  11. fix kernel tests

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    789510e View commit details
    Browse the repository at this point in the history

  12. rearrange the spot where we call wait until completed

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    93abe5c View commit details
    Browse the repository at this point in the history

  13. remove unused import

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    843326c View commit details
    Browse the repository at this point in the history

  14. reintroduce drop unused buffers

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    1a5792a View commit details
    Browse the repository at this point in the history

  15. simplify even further by removing creation of buffers during copy ope… 

    …rations
    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    cf5e2fc View commit details
    Browse the repository at this point in the history

  16. fix synchronization method

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    c51c71e View commit details
    Browse the repository at this point in the history

  17. cleanup memory barriers to only depend on inputs

    @tomsanbear
    tomsanbear committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    4067bdd View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. refactor command encoder and buffer to be lazily initialized

    @tomsanbear
    tomsanbear committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    41bb216 View commit details
    Browse the repository at this point in the history

  2. add syncchronize back in

    @tomsanbear
    tomsanbear committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    0a1d47e View commit details
    Browse the repository at this point in the history

  3. clean up documentation

    @tomsanbear
    tomsanbear committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    ef69643 View commit details
    Browse the repository at this point in the history

  4. simplify the to_cpu op

    @tomsanbear
    tomsanbear committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    79647c8 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' of https://github.com/huggingface/candle into Com… 

    …mandEncoderReuse
    @tomsanbear
    tomsanbear committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    912755d View commit details
    Browse the repository at this point in the history

Commits on Apr 12, 2024

  1. Merge branch 'main' of https://github.com/huggingface/candle into Com… 

    …mandEncoderReuse
    @tomsanbear
    tomsanbear committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    3577e24 View commit details
    Browse the repository at this point in the history

  2. cleanup constructor for the metal device

    @tomsanbear
    tomsanbear committed Apr 12, 2024
    Configuration menu
    Copy the full SHA
    f57a24f View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2024

  1. Merge branch 'main' of https://github.com/huggingface/candle into Com… 

    …mandEncoderReuse
    @tomsanbear
    tomsanbear committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    071199c View commit details
    Browse the repository at this point in the history

  2. update docs explaining usage of the end_compute_encoding function

    @tomsanbear
    tomsanbear committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    5ba51c1 View commit details
    Browse the repository at this point in the history

  3. remove unnecessary memory barrier

    @tomsanbear
    tomsanbear committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    74a265f View commit details
    Browse the repository at this point in the history

Commits on Apr 14, 2024

  1. review feedback changes on command encoder/buffer internals

    @tomsanbear
    tomsanbear committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    c026881 View commit details
    Browse the repository at this point in the history

  2. prevent oom errors

    @tomsanbear
    tomsanbear committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    7d28704 View commit details
    Browse the repository at this point in the history

  3. actually increment

    @tomsanbear
    tomsanbear committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    4089486 View commit details
    Browse the repository at this point in the history

  4. allocate data directly

    @tomsanbear
    tomsanbear committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    6fcfdde View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' of https://github.com/huggingface/candle into Com… 

    …mandEncoderReuse
    @tomsanbear
    tomsanbear committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    675ef87 View commit details
    Browse the repository at this point in the history

  6. change to independent command buffers for blit encoders

    @tomsanbear
    tomsanbear committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    f988a52 View commit details
    Browse the repository at this point in the history

  7. update ones impl to use direct buffer allocation

    @tomsanbear
    tomsanbear committed Apr 14, 2024
    Configuration menu
    Copy the full SHA
    572e269 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2024

  1. Merge branch 'main' of https://github.com/huggingface/candle into Com… 

    …mandEncoderReuse
    @tomsanbear
    tomsanbear committed Apr 28, 2024
    Configuration menu
    Copy the full SHA
    566bcb2 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2024

  1. Merge branch 'main' of https://github.com/huggingface/candle into Com… 

    …mandEncoderReuse
    @tomsanbear
    tomsanbear committed May 4, 2024
    Configuration menu
    Copy the full SHA
    4f73ca1 View commit details
    Browse the repository at this point in the history

  2. update impl to use command encoder

    @tomsanbear
    tomsanbear committed May 4, 2024
    Configuration menu
    Copy the full SHA
    b8b52ce View commit details
    Browse the repository at this point in the history

  3. use sat sub

    @tomsanbear
    tomsanbear committed May 4, 2024
    Configuration menu
    Copy the full SHA
    0d8f4f2 View commit details
    Browse the repository at this point in the history

  4. fix zero tests

    @tomsanbear
    tomsanbear committed May 4, 2024
    Configuration menu
    Copy the full SHA
    3add705 View commit details
    Browse the repository at this point in the history

  5. fix

    @tomsanbear
    tomsanbear committed May 4, 2024
    Configuration menu
    Copy the full SHA
    b161320 View commit details
    Browse the repository at this point in the history

  6. adjust buffer behaviour:

    @tomsanbear
    tomsanbear committed May 4, 2024
    Configuration menu
    Copy the full SHA
    042e001 View commit details
    Browse the repository at this point in the history

Commits on May 6, 2024

  1. Merge branch 'main' of https://github.com/huggingface/candle into Com… 

    …mandEncoderReuse
    @tomsanbear
    tomsanbear committed May 6, 2024
    Configuration menu
    Copy the full SHA
    54c7e9b View commit details
    Browse the repository at this point in the history

  2. clippy fix

    @tomsanbear
    tomsanbear committed May 6, 2024
    Configuration menu
    Copy the full SHA
    53ac546 View commit details
    Browse the repository at this point in the history

  3. format

    @tomsanbear
    tomsanbear committed May 6, 2024
    Configuration menu
    Copy the full SHA
    c4e2434 View commit details
    Browse the repository at this point in the history