Worth a read + insights I found compelling:

+ While chatbots have improved their humanlike performances, they still have trouble with negation. They know what it means if a bird can’t fly, but they collapse when confronted with more complicated logic involving words like “not,” which is trivial to a human.

+ It’s hard to coax a computer into reading and writing like a human. Machines excel at storing lots of data and blasting through complex calculations, so developers build LLMs as neural networks: statistical models that assess how objects (words, in this case) relate to one another.

+ Each linguistic relationship carries some weight, and that weight — fine-tuned during training — codifies the relationship’s strength.

+ For example, “rat” relates more to “rodent” than “pizza,” even if some rats have been known to enjoy a good slice.

+ In the same way that your smartphone’s keyboard learns that you follow “good” with “morning,” LLMs sequentially predict the next word in a block of text.

+ The bigger the data set used to train them, the better the predictions, and as the amount of data used to train the models has increased enormously, dozens of emergent behaviors have bubbled up.

+ Unlike humans, LLMs process language by turning it into math. This helps them excel at generating text — by predicting likely combinations of text — but it comes at a cost.

+ “The problem is that the task of prediction is not equivalent to the task of understanding,” said Allyson Ettinger, a computational linguist at the University of Chicago.

+ Negations like “not,” “never” and “none” are known as stop words, which are functional rather than descriptive.

+ So why can’t LLMs just learn what stop words mean? Ultimately, because “meaning” is something orthogonal to how these models work.

+ Models learn “meaning” from mathematical weights: “Rose” appears often with “flower,” “red” with “smell.” And it’s impossible to learn what “not” is this way.

+ When children learn language, they’re not attempting to predict words, they’re just mapping words to concepts. They’re “making judgments like ‘is this true’ or ‘is this not true’ about the world,” Ettinger said.

Full article here.