ChatGPT's odds of getting code questions correct are worse than a coin flip
https://www.theregister.com/2023/08/07/chatgpt_stack_overflow_ai/
"'Our analysis shows that 52 percent of ChatGPT answers are incorrect and 77 percent are verbose,' the team's paper concluded. 'Nonetheless, ChatGPT answers are still preferred 39.34 percent of the time due to their comprehensiveness and well-articulated language style.' Among the set of preferred ChatGPT answers, 77 percent were wrong." -- #ThomasClaburn
#api360 #chatGPT #programming
:mastodon: Mike Amundsen /
npub1qf…6rzhj
2023-09-16 14:00:02
Author Public Key
npub1qf0nldvdxqzqlwexkyrtseph85prxnhrxxvkkph25tgrsk35pe5sv6rzhjPublished at
2023-09-16 14:00:02Event JSON
{
"id": "627bccb2a5c74f250abc90fcac0a62a8c8b46c6b4ca8bd5595403f59d6b0cb9a",
"pubkey": "025f3fb58d30040fbb26b106b864373d02334ee331996b06eaa2d0385a340e69",
"created_at": 1694865602,
"kind": 1,
"tags": [
[
"t",
"thomasclaburn"
],
[
"t",
"api360"
],
[
"t",
"chatgpt"
],
[
"t",
"programming"
],
[
"proxy",
"https://mastodon.social/users/mamund/statuses/111074712119981746",
"activitypub"
]
],
"content": "ChatGPT's odds of getting code questions correct are worse than a coin flip\n\nhttps://www.theregister.com/2023/08/07/chatgpt_stack_overflow_ai/\n\n\"'Our analysis shows that 52 percent of ChatGPT answers are incorrect and 77 percent are verbose,' the team's paper concluded. 'Nonetheless, ChatGPT answers are still preferred 39.34 percent of the time due to their comprehensiveness and well-articulated language style.' Among the set of preferred ChatGPT answers, 77 percent were wrong.\" -- #ThomasClaburn\n\n#api360 #chatGPT #programming",
"sig": "e371eb0fbc5cd20ad31afbd4b64705317f0d2416b7bf2b7a7e874465ba67a317ccacddb94688666ecc1f0a1b82d4250bcb16560c7d6ed60785a8fe7f569eb6f1"
}