Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add better model properties table to docs #640

Closed

Conversation

mivanit
Copy link

@mivanit mivanit commented Jun 17, 2024

[defunct -- replacing with #769]

Description

This PR ports features from my transformerlens-model-table repo to TransformerLens, implementing many of the features requested in #97. I still need some feedback on this, and presumably building docs will fail for one reason or another once I make the PR.

Features:

The static table has a few more fields added to it, but the primary focus is the interactive table. This provides:

  • information on parallel attn/mlps, positional embeddings, and other config elements
  • filtering and searching on any column (i.e. sort by parameter count and only show standard positional embeddings)
  • links back to the huggingface model page, where applicable (extracted from the "official model name")
  • tokenizer information, including vocab hash (need feedback on if there is a better way to do this)
  • full config in title text or new window
  • organized view of dimensions of all tensors in state dict and activation cache (via setting device to meta, doesn't require actually loading models)

Adds dependencies

under group docs:

  • tiktoken for dealing with certain tokenizers
  • muutils for pretty-printed data on tensor shapes

Type of change

  • This change requires a documentation update

Screenshots

Before:

Original model properties table

image

After (static):

You can see what the generated data looks like here

image

After (interactive):

See demo

tl-new

image

Checklist:

(currently draft PR, testing incomplete)

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

mivanit and others added 30 commits June 16, 2024 21:31
@mivanit
Copy link
Author

mivanit commented Oct 29, 2024

I can't figure out why the tests are failing, it seems to have nothing to do with my changes and merging in the dev branch doesn't help. Making a new PR #769

@mivanit mivanit closed this Oct 29, 2024
mivanit added a commit to mivanit/TransformerLens that referenced this pull request Oct 29, 2024
bringing in changes from TransformerLensOrg#640
since I cannot get the tests in that PR to pass :(
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants