Are 52% Of ChatGPT Coding Answers Incorrect?

The truth about the “latest” research is more interesting than how media twisted it

With the raise of LLM, large language models, more and more interesting solutions appear. One of such solutions is ChatGPT. A AI chatbot. It’s not used only for natural language. It’s also used by programmers to find answers to their questions.

It’s great to answer simple questions that were already answered properly somewhere on the internet.

This article was brought to you, because people read it on Medium.

The latest version also is able to browse provided links to gather additional information.

The possibilities are impressive, but programmers start to notice ChatGPT is not always right.

From time to time software engineers share ChatGPT answers that are ridiculous, unsafe or totally wrong.

Yesterday I’ve checked how the new model works. I’ve provided it with the link to release notes of Kotlin and asked ChatGPT to tell me what is the funniest update. There were like 150 changes made, so I figured out it will be an interesting experiment.

To my surprise ChatGPT fetched the changelog, and wrote that there’s a bugfix for emoji error that’s the funniest…