Loading the lora adapter using PEFT vs Unsloth

[hessaAlawwad]

I have trained a lora adapter by following the toturial in: Llama 3.2 Vision finetuning - Radiography use case.
Why there is difference in the number of trainable parameters when I load the adapter? (I need to load the lora adapter using peft not unsloth)

1- I load the lora_adapter using unsloth:
trainable params: 67,174,400 || all params: 10,737,395,235 || trainable%: 0.6256
code:

from unsloth import FastVisionModel
import torch

model, tokenizer = FastVisionModel.from_pretrained(
    model_name = "Hessa/lora_llama3.2_10k",
    dtype = torch.bfloat16,
    load_in_4bit = False,
)

2- I load using PEFT
Base model parameters: 9824213008
LoRA adapter parameters: 9824213008
Combined model parameters: 9824213008

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained("unsloth/Llama-3.2-11B-Vision-Instruct")
model_with_lora = PeftModel.from_pretrained(base_model, "Hessa/lora_llama3.2_10k")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Llama-3.2-11B-Vision-Instruct")

[danielhanchen]

I'm assuming some modules in Hugging Face are all trainable, hence the discrepancy.