Трансформатор ошибок с QLoRA и Peft ⇐ Python

[Гость]

Итак, я пытаюсь настроить модель Google Gemma, используя Peft и QLoRA. Вчера я успешно настроил его на одну эпоху просто в качестве теста. Однако, когда я сегодня открыл блокнот и запустил ячейку, загружающую модель, я получил огромную ошибку:
Код:

Код: Выделить всё

model_id = "google/gemma-7b"

bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer =
AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
quantization_config=bnb_config,
device_map={0:""})

#model.gradient_checkpointing_enable()

train_dataset, val_dataset, data_collator = load_dataset(train_data_path, val_data_path, tokenizer)

Ошибка (сокращенно):

Код: Выделить всё

RuntimeError: device >= 0 && device < num_gpus INTERNAL ASSERT FAILED at "../aten/src/ATen/cuda/CUDAContext.cpp":50, please report a bug to PyTorch. device=1, num_gpus=

.....

DeferredCudaCallError: CUDA call failed lazily at initialization with error: device >= 0 && device < num_gpus INTERNAL ASSERT FAILED at "../aten/src/ATen/cuda/CUDAContext.cpp":50, please report a bug to PyTorch. device=1, num_gpus=
.....

RuntimeError: Failed to import transformers.integrations.bitsandbytes because of the following error (look up to see its traceback):
CUDA call failed lazily at initialization with error: device >= 0 && device < num_gpus INTERNAL ASSERT FAILED at "../aten/src/ATen/cuda/CUDAContext.cpp":50, please report a bug to PyTorch. device=1, num_gpus=

.....

I have shortened the error for it to be more readable. Has anyone experienced something like this? I can't seem to solve it. Help is much appreciated.


Источник: https://stackoverflow.com/questions/781 ... a-and-peft