Is chatGPT good at logical thinking?
This is perhaps the most important question on people’s mind since chatGPT was released in November 2022, and has serious implications for life and work in the AI era. There have also been some studies done to analyse the logical thinking abilities of chatGPT and other such AI tools. The primary finding has been that current AI tools do have logical thinking skills, but not good enough to replace humans at work as of now. This may surely change in the future as AI tools become better and better!
I decided to answer this question my own through some data collection using the Pariksha portal. This question is important for me because as a teacher, I need to know if my students’ careers are safe or do we need to make dramatic changes in what we teach. Computers have been helping humans do a lot of things more efficiently, but if AI tools become good at logical thinking, we may see serious job losses across all sectors. And people who wish to stay in the race need to make sure they have better logical reasoning skills than AI!
I created an online test on Pariksha which anyone can take for free (details in the image below). This test was taken by ~40 people scattered all across India, and the average score was ~34%. I then fed these questions into chatGPT, and the score came out to be ~48%. So clearly, on average chatGPT seems better than the people who took my online test. My sample size of 40 is not very large, but good enough to do statistical comparisons. There are surely a lot of human beings who are much better than chatGPT at logical thinking but the fact that chatGPT was able to score ~48% is truly astonishing for me at least. Based on my 10+ years of teaching experience at some of the best educational institutions in India, I would place chatGPT at the average level among students at IITs and other such educational institutions. And this has serious implications for the future of jobs!
The questions in my online test are a mix of mathematical reasoning and logical thinking. Interestingly, chatGPT was able to perform much better at mathematical reasoning type of questions which involved doing some calculations (eg. algebra, probability, statistics, etc). And at purely logical thinking questions (no equations or numbers), its score was ~32% which is much lower than the overall score of ~48%, but still not too bad.
The main takeaway for the readers of this blog should be that chatGPT and other such AI tools are quite good at logical thinking and will only get better with time. So if you wish to stay in the game, start working on your reasoning skills today! You may be good at it as on today, but may not remain good enough in the future.
