dasanindya15/llama3-8b_qlora_Cladder_v1

Loading Model and Tokenizer:

base_model_id = "NousResearch/Meta-Llama-3-8B"
new_model_id = "dasanindya15/llama3-8b_qlora_Cladder_v1"

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
from peft import PeftModel
from transformers import BitsAndBytesConfig

# Load the entire model on the GPU 0
device_map = {"": 0}

# Reload model in FP16 and merge it with LoRA weights
# specify the quantize the model
quantization_config = BitsAndBytesConfig(
            load_in_4bit=True,
            bnb_4bit_use_double_quant=True,
            bnb_4bit_quant_type="nf4",
            bnb_4bit_compute_dtype=torch.bfloat16,
)
base_model = AutoModelForCausalLM.from_pretrained(base_model_id,
                                             quantization_config=quantization_config,
                                             device_map=device_map)
model = PeftModel.from_pretrained(base_model, new_model_id)

# Reload tokenizer to save it
tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"

---

license: mit datasets: - dasanindya15/Cladder_v1 pipeline_tag: text-classification