When a five-year old says “two plus two is four” we take it as proof of the growing intelligence of a child. In reality it is not that the child understands the abstract concept of numbers and where they come from to represent identities in our physical world. It is just the child applying its own language model to predict the best next word. This is based on the data it has been fed and the supervised and unsupervised learning it has received from its trainers. The child is merely going through an auto-complete process with its “best” guess for the next word. based on what knowledge it has been fed and the training it has received.
Chomsky does not like chatGPT and dismisses it as being a glorified auto-completer using statistics and probability to estimate the “best” next word. But I think he’s got it wrong again. Whereas human brains may not exclusively use just a large language model, we certainly do use language when we choose the “best” option for the next word we use (speak, write or even think). We may also use logic, or what we call reason or even other languages to judge what the next word ought to be. This includes all forms of mathematics and specialised languages with esoteric symbols or hieroglyphs. Language is overwhelmingly the method of communicating output from a human brain. We use a variety of processes in our brains to ultimately choose the next word we use. Just like chatGPT, the input is the previous word and the output is the next word.
In judging whether a brain (or a neural network) is intelligent, what is critical is what is generated rather than how it is generated. The process by which a brain for a human, or a neural network for a chatbot, generates the next word based on the previous word(s), is irrelevant in judging whether the brain or the neural network is intelligent. The fundamental problem is that we cannot define intelligence. We cannot, as humans, define what we mean when we say we understand something. We cannot tell what process takes place in our brains when we claim we understand addition or subtraction or some other mathematical or logical process.
It seems to me then that if in the future, a chatbot eventually does do mathematics in practice and is always correct, then it is irrelevant if its neural network got there by calculating probabilities of occurrence of the next most likely word or did it in some other way. If it does mathematics then our assessment of its understanding mathematics becomes moot. If it does generate useful and correct code then its understanding of the objectives is irrelevant. Moreover, we cannot say it does not understand when we cannot determine what understanding means for us, let alone for it. We cannot either impose on an AI chatbot a definition of its understanding when we cannot define it for ourselves.
Perhaps understanding is nothing more than weightage numbers in a network of neurons whether in a human brain or in an AI’s neural network software.
The k2p blog 