v2.2.0

Here's an important one! This release adds Gemma-2, and it also greatly improves model accuracy across the board. It is highly recommended that everyone update to this version immediately to take advantage of these accuracy improvements.

What's Changed

• Fix typo in Main_Demo.ipynb by @ianand in #636
• Add comparing-to-huggingface.ipynb. by @joelburget in #637
• Add tests for gated mlp by @anthonyduong9 in #638
• Match Huggingface MLP implementation exactly. by @joelburget in #641
• Add tests for ActivationCache by @FlyingPumba in #643
• Moved mixtral weights to another module by @bryce13950 in #646
• Fixed weight conversion by @bryce13950 in #648
• Move out pretrained weight conversions by @richardkronick in #647
• Match Huggingface GPT2 implementation exactly by @joelburget in #645
• Fix Out bias not being summed in attention component when using 4 bit precision by @FlyingPumba in #654
• Mlp cleanup by @bryce13950 in #652
• Added support for Gemma-2 by @neelnanda-io in #650
• add tests for Attention by @anthonyduong9 in #639
• Release 2.2 by @bryce13950 in #656

New Contributors

• @ianand made their first contribution in #636
• @FlyingPumba made their first contribution in #643

Full Changelog: v2.1.0...v2.2.0