Filip W on Nostr: “ChatGPT’s ability to produce functional code for “easy” coding problems ...
“ChatGPT’s ability to produce functional code for “easy” coding problems dropped from 89 percent to 52 percent after 2021. And its ability to generate functional code for “hard” problems dropped from 40 percent to 0.66 percent after this time as well.”
“A reasonable hypothesis for why ChatGPT can do better with algorithm problems before 2021 is that these problems are frequently seen in the training dataset,” Tang says.”
🔗
https://spectrum.ieee.org/chatgpt-for-codingPublished at
2024-07-09 10:25:47Event JSON
{
"id": "6bcf64ddca3007bde8881ee96d98dbf2c52cd8fe1aa2d2011b1f2e2bce67540c",
"pubkey": "0c0d593646e8f3403d6463582983ae28bae37b8fc58b0b51113f7f1c0b4c8c72",
"created_at": 1720520747,
"kind": 1,
"tags": [
[
"proxy",
"https://mathstodon.xyz/@filipw/112756047732157084",
"web"
],
[
"proxy",
"https://mathstodon.xyz/users/filipw/statuses/112756047732157084",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://mathstodon.xyz/users/filipw/statuses/112756047732157084",
"pink.momostr"
],
[
"expiration",
"1723141233"
]
],
"content": "“ChatGPT’s ability to produce functional code for “easy” coding problems dropped from 89 percent to 52 percent after 2021. And its ability to generate functional code for “hard” problems dropped from 40 percent to 0.66 percent after this time as well.”\n“A reasonable hypothesis for why ChatGPT can do better with algorithm problems before 2021 is that these problems are frequently seen in the training dataset,” Tang says.”\n🔗 https://spectrum.ieee.org/chatgpt-for-coding",
"sig": "bd30fb5d3f0ae8aa50c1cd17c38244761028b865006da0a41bc5649ef1dc4524b2043b30ea6178eb4c851f04fc09ab302aec05089f03527666de1689769d176c"
}
