AI is getting worse...Wait, what did you say?

According to a new study from Stanford and Berkely which measured how Chat CPT-3.5 and 4 behaviors has changed over time, from March 2023 to June 2023 the systems showed significant decline in multiple categories. But wait a minute, I thought AI was about to take over the world any day now and enslave us all like in Terminator? Perhaps us humans will delay our demise a bit more.

The study looked at four key areas:
1) Solving math problems
2) Answering sensitive / dangerous questions
3) Generating code
4) Visual reasoning

In reviewing the study it doesn’t give a clear, concise explanation as to why this might be happening, other than to say “this highlights the need to continuously evaluate and assess the behavior”. In my search for a better answer, I decided to ask Chat CPT-4 what it though of the study and here is what I got:

Last update was almost a year ago, really? That’s a bit surprising.

In my experience with Chat GPT, which includes using it a decent amount, reading articles, and listening to podcasts; the best summary I’ve heard is “think of Chat GPT like an intern”. It’s great for helping to write marketing content, perform tasks like SEO keyword searches but it’s not putting people out of work anytime soon (except for maybe interns) and for now at least it seems it won’t be taking over the world.

We use it to help with coding, if the development team gets stuck, they ask Chat GPT instead of Google. It’s helpful in providing feedback for portions of code but unfortunately our development team still needs to perform 95% of the work.

Obviously AI is field that is advancing quickly and besides Chat GPT all the big tech players are jumping in and making AI a core focus of their future growth plans. We’ll keep on eye things and provide updates worth mentioning.

Link to article: https://arxiv.org/pdf/2307.09009.pdf