Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Allow for attention caching during CoCa generation #502

Draft
wants to merge 11 commits into
base: main
Choose a base branch
from
Draft

[WIP] Allow for attention caching during CoCa generation #502

wants to merge 11 commits into from
+112 −27

Commits on Apr 16, 2023

  1. initial caching for generation

    @sramshetty
    sramshetty committed Apr 16, 2023
    Configuration menu
    Copy the full SHA
    ecb93d9 View commit details
    Browse the repository at this point in the history

  2. remove timing

    @sramshetty
    sramshetty committed Apr 16, 2023
    Configuration menu
    Copy the full SHA
    8ad9240 View commit details
    Browse the repository at this point in the history

  3. fix beamsearch caching

    @sramshetty
    sramshetty committed Apr 16, 2023
    Configuration menu
    Copy the full SHA
    71d01f5 View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2023

  1. WIP Setup base for text encoder caching

    @sramshetty
    sramshetty committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    09e3fec View commit details
    Browse the repository at this point in the history

  2. Merge branch 'mlfoundations:main' into inference_caching

    @sramshetty
    sramshetty authored Apr 20, 2023
    Configuration menu
    Copy the full SHA
    df85d0c View commit details
    Browse the repository at this point in the history

  3. Fix transformer caching default

    @sramshetty
    sramshetty committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    2c961e6 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2023

  1. avoid passing cache when not necessary

    @sramshetty
    sramshetty committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    69a936a View commit details
    Browse the repository at this point in the history

  2. simplify caching argument

    @sramshetty
    sramshetty committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    8dc14ed View commit details
    Browse the repository at this point in the history

  3. fix transformer cache typing for true branch

    @sramshetty
    sramshetty committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    5ed4cf2 View commit details
    Browse the repository at this point in the history

  4. reorder typing to address list invariance

    @sramshetty
    sramshetty committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    c0686de View commit details
    Browse the repository at this point in the history

  5. remove unnecessary placeholder lists

    @sramshetty
    sramshetty committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    e561c8f View commit details
    Browse the repository at this point in the history