Mark Pesce on Nostr: The new LLaMA 3.1 8B Instruct model just *failed* my basic agent test. Which is ...
The new LLaMA 3.1 8B Instruct model just *failed* my basic agent test. Which is unexpected, because the 3.0 model passes with flying colours. So maybe all is not exactly right in Meta-AI-land...
It failed in a way I've seen numerous other models fail - because they can't follow instructions.
Ironic, in an Instruct-tuned model.
Published at
2024-07-24 04:03:46Event JSON
{
"id": "2185f0e624b1cc3e554643069e7b4fafe2b7a10c6dcd24194d07d136b31ea77a",
"pubkey": "8b0ef49d11c147634fe81e5df544407f35ac0ca01d288082da576d0404fbbd8f",
"created_at": 1721793826,
"kind": 1,
"tags": [
[
"proxy",
"https://arvr.social/users/mpesce/statuses/112839480219922791",
"activitypub"
]
],
"content": "The new LLaMA 3.1 8B Instruct model just *failed* my basic agent test. Which is unexpected, because the 3.0 model passes with flying colours. So maybe all is not exactly right in Meta-AI-land...\n\nIt failed in a way I've seen numerous other models fail - because they can't follow instructions.\n\nIronic, in an Instruct-tuned model.",
"sig": "4be15aa3f02a3e8aee5e8aa37bda91391d1047b28a61e7ac7b8ef4d503182918a32e781f0893ab8865721da1907b45259bd41fcc43330e27bea425361a9de7de"
}
