LLAMA3 инструктирует 8B галлюцинации, хотя я использую правильный формат подсказки ⇐ Python

[Anonymous]

Я запускаю конечную точку Meta-llama/Meta-Llama-3-8B-Instruct на AWS и по какой-то причине не могу получить приемлемый результат при запросе модели. У него галлюцинации, даже когда я отправляю простую подсказку. Может ли кто-нибудь посоветовать, что я делаю не так?
Примерная подсказка:
system

You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content.
Please ensure that your responses are socially unbiased and positive in nature.
If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.user

Please tell me a jokeassistant

Ответ, который я получаю:
I'd be happy to share a joke with you! Here's one:

Why couldn't the bicycle stand up by itself?

Because it was two-tired!

I hope that brought a smile to your face! Do you want to hear another one?assistant

I'd be happy to share another one! Here's one:

What do you call a fake noodle?

An impasta!

I hope that one made you laugh! Do you want to hear another one?assistant

Here's another one:

Why don't scientists trust atoms?

Because they make up everything!

I hope that one was atom-ically funny! Do you want to hear another one?assistant

Here's another one:

Why don't eggs tell jokes?

Because they'd crack each other up!

I hope that one cracked you up! Do you want to hear another one?assistant

(продолжается некоторое время).
Развертывание:
Я развернул его с помощью:
hub = {
"HF_MODEL_ID": "meta-llama/Meta-Llama-3-8B-Instruct",
"HF_AUTO_CAST_TYPE": "bf16",
"HUGGING_FACE_HUB_TOKEN": "******",
}

llm_image = '763104351884.dkr.ecr.ap-southeast-2.amazonaws.com/huggingface-pytorch-tgi-inference:2.0.1-tgi1.0.3-gpu-py39-cu118-ubuntu20.04'
endpoint_name = 'data-science-llm-llama3-8b'

# create Hugging Face Model Class
llm_model = HuggingFaceModel(
image_uri=llm_image,
env=hub,
role=role,
name=endpoint_name
)

Кварги модели:
model_kwargs:
temperature: 0.001
do_sample: True
max_new_tokens: 500
typical_p: 0.2
seed: 1
use_cache: False
return_full_text: False


Подробнее здесь: https://stackoverflow.com/questions/783 ... mpt-format