Code in Book has issues

[maheshrajrp]

Code in discussion

alpaca_template = """Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{}
### Response:
{}"""
EOS_TOKEN = tokenizer.eos_token
dataset = dataset.map(format_samples, batched=True, remove_columns=dataset.column_names)
dataset = dataset.train_test_split(test_size=0.05)
trainer = SFTTrainer(
    model=model,
   tokenizer=tokenizer,
    train_dataset=dataset["train"],
    eval_dataset=dataset["test"],
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    dataset_num_proc=2,
    packing=True,
    args=TrainingArguments(
        learning_rate=3e-4,
        lr_scheduler_type="linear",
        per_device_train_batch_size=2,
        gradient_accumulation_steps=8,
        num_train_epochs=3,
        fp16=not is_bfloat16_supported(),
        bf16=is_bfloat16_supported(),
        logging_steps=1,
        optim="adamw_8bit",
        weight_decay=0.01,
        warmup_steps=10,
        output_dir="output",
        report_to="comet_ml",
        seed=0,
    ),
)
trainer.train()

I copy pasted as is from book, found format_samples function to be missing, which I replaced with

def format_samples(examples):
    text = []
    for instruction, output in zip(examples["instruction"], examples["output"], strict=False):
        message = alpaca_template.format(instruction, output) + EOS_TOKEN
        text.append(message)

    return {"text": text}

However, now I'm facing with

Generating train split: 
 2148/0 [00:04<00:00, 667.85 examples/s]
Generating train split: 
 110/0 [00:00<00:00,  6.53 examples/s]
==((====))==  Unsloth - 2x faster free finetuning | Num GPUs = 1
   \\   /|    Num examples = 2,148 | Num Epochs = 3
O^O/ \_/ \    Batch size per device = 2 | Gradient Accumulation steps = 8
\        /    Total batch size = 16 | Total steps = 402
 "-____-"     Number of trainable parameters = 83,886,080
COMET INFO: An experiment with the same configuration options is already running and will be reused.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-25-6abc222dd3b9>](https://localhost:8080/#) in <cell line: 27>()
     25             ),
     26         )
---> 27 trainer.train()

28 frames
[/usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py](https://localhost:8080/#) in _CausalLM_fast_forward(self, input_ids, causal_mask, attention_mask, position_ids, past_key_values, inputs_embeds, labels, use_cache, output_attentions, output_hidden_states, return_dict, num_logits_to_keep, *args, **kwargs)
    974             )
    975         else:
--> 976             causal_mask = xformers.attn_bias.LowerTriangularMask()
    977 
    978             output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions

AttributeError: 'NoneType' object has no attribute 'attn_bias'

[iusztinpaul]

Hello,

Please use directly the code from the repository under the llm_engineering/model directory