Issue with 'GemmaFixedRotaryEmbedding' object has no attribute 'cos_cached':

[Bruce-Azar-Wayne]

Im using unsloth to import gemma-2-9b and fine tune it. This exact code used to work less than 12 hours ago, but it's not working now. I need this for a project by the end of today, so would appreciate the help. The code is on Kaggle.

AttributeError: 'GemmaFixedRotaryEmbedding' object has no attribute 'cos_cached'

[ShambaC]

Its an issue caused by the latest version of transformers. Downgrade transformers package to 4.47.1 after installing unsloth to fix the issue.

! pip -q uninstall transformers -y
! pip -q install transformers==4.47.1

[danielhanchen]

Working on a fix - sorry on the issue - but yes it's because transformers updated to the latest version

[thewebscraping]

Same error with latest version

[thewebscraping]

Here my fix, thanks @ShambaC

!pip install -q torch torchvision torchaudio xformers --index-url https://download.pytorch.org/whl/cu121
!pip install -q 'unsloth==2025.1.1'
!pip uninstall -q transformers -y
!pip install -q 'transformers==4.47.1'

[danielhanchen]

Working on a fix - yep downgrading is a good solution for now - apologies again!