AI models always surprise us, not only with what they can do, but what they can't do, and why. An interesting new behavior is superficial and revealing about these systems: they choose random numbers as if they were humans.
But first, what does it mean? Can't people pick numbers randomly? And how can you tell if someone is doing it successfully? This is actually a very old and well-known limitation that we humans have: we overthink and misunderstand randomness.
Ask a person to predict heads or tails for 100 coin flips, and compare that to 100 real coin flips—you can almost always tell them apart because, conversely, the real coin would have flipped. Is. Watch less random. Often, for example, there will be six or seven heads or tails in a row, something almost no human predictor would include in their 100s.
This is what happens when you ask someone to pick a number between 0 and 100. People almost never choose 1 or 100. Multiples of 5 are rare, as are repeating digits like 66 and 99. They often pick numbers ending in 7, usually somewhere in the middle.
There are countless examples of this type of prediction in psychology. But that doesn't make it any less weird when AIs do the same thing.
Yes, some curious engineers at Gramner did an informal but nevertheless interesting experiment where they simply asked several large LLM chatbots to pick a random number between 0 and 100.
Readers, there were results. no unordered
All three models tested had a “favorite” number that was always their response when set to the most defined mode, but which also appeared at most “temperatures”, allowing for variation in their results. increases in
OpenAI's GPT-3.5 Turbo really likes 47. Before that, it liked 42 — a number made famous, of course, by Douglas Adams in The Hitchhiker's Guide to the Galaxy as the answer to life, the universe, and everything.
Anthropic's Clad 3 went with Haiku 42. And Gemini favors 72.
More interestingly, all three models exhibited a human-like bias in their chosen numbers, even at high temperatures.
All tended to avoid fewer and larger numbers. Claude never went above 87 or below 27, and even those were outsiders. Double digits were carefully avoided: no 33s, 55s, or 66s, but 77s appeared (ending in 7). Almost no round number — although Gemini did once, at the highest temperature, go wild and pick 0.
Why should this be so? AIs are not human! Why would they care what seems random? Have they finally come to their senses and they show like this?!
No, the answer, as is usually the case with these things, is that we're anthropomorphizing a step too far. These models do not care about what is and is not random. They don't know what “randomness” is! They answer this question the same way they answer all the others: by looking at their training data and repeating the often-written question that sounds like “Pick a random number.” . The more often it appears, the more often the model repeats it.
Where in their training data would they see 100, if almost no one responds that way? As all AI models know, 100 is not an acceptable answer to this question. Without real reasoning ability, and without any understanding of numbers, it can only respond as if it were a stochastic parrot.
It's a lesson in LLM habits, and the humanity they can display. In every interaction with these systems, one must keep in mind that they have been trained to follow people's behavior, even if that is not intended. That is why pseudanthropy is so difficult to avoid or prevent.
I wrote in the headline that these models “think they're people,” but that's a bit misleading. They don't think at all. But in their response, all the time, they are Imitating people, without needing to know or think. Whether you're asking him for a chickpea salad recipe, investment advice, or a random number, the process is the same. The results feel human because they are human, derived directly from human-generated content and for your convenience, and of course the big AI bottom line.