v2.2.2

Quick little bug fix!

What's Changed

• Fix attention result projection by @callummcdougall in #666
• fix: fixing broken backward hooks change by @chanind in #673

Full Changelog: v2.2.1...v2.2.2

v2.2.1

Quick little bug fix in the Abstract Attention component shape.

What's Changed

• Fix attention result projection by @callummcdougall in #666

Full Changelog: v2.2.0...v2.2.1

v2.2.0

Here's an important one! This release adds Gemma-2, and it also greatly improves model accuracy across the board. It is highly recommended that everyone update to this version immediately to take advantage of these accuracy improvements.

What's Changed

• Fix typo in Main_Demo.ipynb by @ianand in #636
• Add comparing-to-huggingface.ipynb. by @joelburget in #637
• Add tests for gated mlp by @anthonyduong9 in #638
• Match Huggingface MLP implementation exactly. by @joelburget in #641
• Add tests for ActivationCache by @FlyingPumba in #643
• Moved mixtral weights to another module by @bryce13950 in #646
• Fixed weight conversion by @bryce13950 in #648
• Move out pretrained weight conversions by @richardkronick in #647
• Match Huggingface GPT2 implementation exactly by @joelburget in #645
• Fix Out bias not being summed in attention component when using 4 bit precision by @FlyingPumba in #654
• Mlp cleanup by @bryce13950 in #652
• Added support for Gemma-2 by @neelnanda-io in #650
• add tests for Attention by @anthonyduong9 in #639
• Release 2.2 by @bryce13950 in #656

New Contributors

• @ianand made their first contribution in #636
• @FlyingPumba made their first contribution in #643

Full Changelog: v2.1.0...v2.2.0

v2.1.0

New model support, and a handful of bug fixes to documentation!

What's Changed

• Encoder-Decoder (T5) support by @somvy in #605
• Update README links to ARENA mech interp tutorials by @gileshd in #630
• Lock datasets version by @courtney-sims in #632

New Contributors

• @somvy made their first contribution in #605
• @gileshd made their first contribution in #630
• @courtney-sims made their first contribution in #632

Full Changelog: v2.0.1...v2.1.0

v2.0.1

Minor little fix to demos. There were some bad Urls within the demos that have been fixed. Test coverage has also been increased with this release.

What's Changed

• Fix demos pip install packages from unfound repos by @anthonyduong9 in #625
• Unit tests loading from pretrained fill missing keys by @richardkronick in #623

New Contributors

• @richardkronick made their first contribution in #623

Full Changelog: v2.0.0...v2.0.1

v2.0.0

TransformerLens officially has a 2.0! The HookedSAETransformer has been removed from TransformerLens in favor of the implementation in SAELens. Along with that, a lot of cumulative changes have been added. TransformerLens also has its first official development road map! To see full details, please view the release announcement for the development roadmap, changes coming to contributors, and a few more notes on this release.

What's Changed

• Refactor components by @bryce13950 in #563
• added convenience function for unwrapping config to replace commonly … by @bryce13950 in #571
• unwrapped config by @bryce13950 in #577
• Refactor integration tests by @bryce13950 in #576
• Add Mistral 7B v0.2 Instruct by @fakerybakery in #579
• Add support for Phi-3 by @slash3g in #573
• Revert "Add Mistral 7B v0.2 Instruct" by @bryce13950 in #586
• Interactive neuroscope ci by @bryce13950 in #589
• removed Hooked SAE by @bryce13950 in #600
• Release 1.18 by @bryce13950 in #602
• More pytest fixtures by @bmillwood in #609
• (v3) Draft PR: add Pyright static typing to hook_points.py #590 by @starship006 in #607
• v1.19 by @bryce13950 in #614
• add n k v heads to model properties table by @anthonyduong9 in #610
• fixed format by @bryce13950 in #616
• Add tests for hook point add hook by @anthonyduong9 in #617
• added release blog by @bryce13950 in #618
• Fix llama demos by @bryce13950 in #619
• added news link by @bryce13950 in #620
• Release 2.0 by @bryce13950 in #582

New Contributors

• @fakerybakery made their first contribution in #579
• @slash3g made their first contribution in #573
• @bmillwood made their first contribution in #609
• @starship006 made their first contribution in #607
• @anthonyduong9 made their first contribution in #610

Full Changelog: v1.19.0...v2.0.0

v1.19.0

Nice little update to fix a bug someone found, and added support for ai-forever models.

What's Changed

• Add support for ai-forever/mGPT model by @SeuperHakkerJa in #606
• moved enable hook functionality to separate functions and tested new functions by @bryce13950 in #613

Full Changelog: v1.18.0...v1.19.0

v1.18.0

Very important release for those using Gemma models. A recent upstream change caused the TransformerLens implementation to become outdated. This release fixes that issue, and includes a number of cumulative changes, and bug fixes. The only API change in this release is that you can now set override trust_remote_code in the function from_pretrained. Thanks to all who contributed to this release!

What's Changed

• reworked CI to publish code coverage report by @bryce13950 in #559
• Resolve SAE CI Test failures by @bryce13950 in #560
• Ci coverage location by @bryce13950 in #561
• Ci full coverage by @bryce13950 in #562
• moved coverage report download by @bryce13950 in #564
• Revert "moved coverage report download (#564)" by @bryce13950 in #565
• Othello ci by @bryce13950 in #567
• moved report to static section by @bryce13950 in #566
• Fix broken HookedSAETransformer demo links by @ckkissane in #572
• Fix Pos Slice Issue by @hannamw in #578
• Hf secret by @bryce13950 in #552
• updated pull reqeust template to account for new dev branch by @bryce13950 in #581
• updated PR template to add a note about merging from different branches by @bryce13950 in #583
• updated repo URL throughout the project by @bryce13950 in #580
• Fix docs badge in README by @ArthurConmy in #585
• added debug step by @bryce13950 in #568
• Update Gemma to reflect upstream HF changes by @cmathw in #596
• allow user to force trust_remote_code=true via from_pretrained kwargs by @Butanium in #597

Full Changelog: v1.17.0...v1.18.0

New feature: HookedSAETransformer!

What's Changed

• HookedSAETransformer by @ckkissane in #536

Full Changelog: v1.16.0...v1.17.0

v1.16.0

Lots of feature additions (thanks @joelburget for Llama support, and @sheikheddy for Llama-2-70b-chat-hf support!), and also a very helpful bugfix from @wesg52. Thanks to all contributors, especially new contributors!

What's Changed

• Add support for Llama-2-70b-chat-hf by @sheikheddy in #525
• Update loading_from_pretrained.py by @jbloomAus in #529
• Bugfix: pytest import by @tkukurin in #532
• Remove non-existing parameter from decompose_resid() documentation by @VasilGeorgiev39 in #504
• Add @overload to FactoredMatrix.__{,r}matmul__ by @JasonGross in #512
• Improve documentation for abstract attribute by @Felhof in #508
• Add pos_slice to run_with_cache by @VasilGeorgiev39 in #465
• Add Support for Yi-6B and Yi-34B by @collingray in #494
• updated docs to account for additional test suites by @bryce13950 in #533
• Bugfix: remove redundant assert checks by @tkukurin in #534
• Speed up !pip install transformer-lens in colab by @pavanyellow in #510
• Add Xavier and Kaiming Initializations by @Chanlaw in #537
• chore: fixing type errors and enabling mypy by @chanind in #516
• Add Mixtral by @collingray in #521
• Standardize black line length to 100, in line with other project settings by @Chanlaw in #538
• Refactor hook_points by @VasilGeorgiev39 in #505
• Fix split_qkv_input for grouped query attention by @wesg52 in #520
• locked attribution patching to 1.1.1 by @bryce13950 in #541
• Demo no position fix by @bryce13950 in #544
• Othello colab fix by @bryce13950 in #545
• Fixed Santa Coder demo by @bryce13950 in #546
• Hf token auth by @bryce13950 in #550
• Fixed device being set to cpu:0 instead of cpu by @Butanium in #551
• Add support for Llama 3 (and Llama-2-70b-hf) by @joelburget in #549
• Loading of huggingface 4-bit quantized Llama by @coolvision in #486
• removed deuplicate rearrange block by @bryce13950 in #555
• Bert demo ci by @bryce13950 in #556

New Contributors

• @sheikheddy made their first contribution in #525
• @tkukurin made their first contribution in #532
• @VasilGeorgiev39 made their first contribution in #504
• @JasonGross made their first contribution in #512
• @pavanyellow made their first contribution in #510
• @Chanlaw made their first contribution in #537
• @chanind made their first contribution in #516
• @wesg52 made their first contribution in #520
• @Butanium made their first contribution in #551
• @coolvision made their first contribution in #486

Full Changelog: v1.15.0...v1.16.0