TechPostsFromX on Nostr: Example when you ask eg “top 10 sights in Amsterdam” or something, some hired ...
Example when you ask eg “top 10 sights in Amsterdam” or something, some hired data labeler probably saw a similar question at some point, researched it for 20 minutes using Google and Trip Advisor or something, came up with some list of 10, which literally then becomes the correct answer, training the AI to give that answer for that question. If the exact place in question is not in the finetuning training set, the neural net imputes a list of statistically similar vibes based on its knowledge gained from the pretraining stage (language modeling of internet documents).
Source: x.com/karpathy/status/1862569569006887118
Published at
2024-12-01 01:06:11Event JSON
{
"id": "404274f473387f8b06610f4a8f25467e55bba94981075338e9f2d0cbb201885b",
"pubkey": "52d119f46298a8f7b08183b96d4e7ab54d6df0853303ad4a3c3941020f286129",
"created_at": 1733015171,
"kind": 1,
"tags": [],
"content": "Example when you ask eg “top 10 sights in Amsterdam” or something, some hired data labeler probably saw a similar question at some point, researched it for 20 minutes using Google and Trip Advisor or something, came up with some list of 10, which literally then becomes the correct answer, training the AI to give that answer for that question. If the exact place in question is not in the finetuning training set, the neural net imputes a list of statistically similar vibes based on its knowledge gained from the pretraining stage (language modeling of internet documents).\nhttps://image.nostr.build/bbc51a1b533a899f019faf74318f07892065da3b1ae13b4f92a73626d0dec505.png\n\nSource: x.com/karpathy/status/1862569569006887118",
"sig": "a08d0b86d8923333a5182db425339d19c6462dd09931d3374c4b5098a09c4fd72919a586812d8f7abd39f6a21042d74263e18b19e8d1c3c075418470dcb86805"
}
