Modern AI is poker, not chess

“AI makes mistakes” is true, but shouldn’t be the end or it’s just lazy criticism.

Using LLMs effectively requires a mindset shift. People generally want AI to behave like classic software - predictable, exact, correct every time. That’s not what it is.

LLMs are not rooted in reality. They don’t understand anything, let alone the real world. They are trained on a huge corpus of human language to predict the next word correctly on average most of the time in most situations. This is powerful.

People are used to software and computers being like chess ♟️ Modern AI is more like poker 🃏

Chess is predictable. You can explore all potential moves, there’s a right move to make, and if you play the right moves you win. Sit across from a grandmaster and you wouldn’t win a single game. We expect machines to behave in this way - if your calculator gave you a wrong answer even once, you’d throw it away.

Poker doesn’t work like that. Play a hand against a professional and you might beat them. Over a few hands you might even win a couple. Over a year, you’d get cleaned out.

A professional doesn’t expect to win every hand. They know that if they keep doing the right things, on average, over the long run, they come out ahead. They don’t dwell on the losses. They focus on process, and they don’t worry too much when a hand goes against them.

Because you can’t control any single hand, you get disciplined about how you play: build the system for the long term, be disciplined in how you play, don’t bet everything on one turn of the cards.

Which is why you wouldn’t stake your life savings on a single hand of poker. Equally you shouldn’t trust a Large Language Model in a high-stakes, one-shot moment either. But a low-stakes, repeatable environment is perfect. There’ll be mistakes along the way, but a good player wins over the long run.

Large language models are powerful but erratic and prone to errors - much more poker than chess.

Thinking in Bets by Annie Duke is a great book on exactly this — and poker is a brilliant game for learning to navigate uncertainty, human behaviour, and much more.

So what’s your game? Poker or chess?