Slot Machine Reward Psychology
This week I attended the Global Gaming Expo Show in Las Vegas. I've been to the show many times over the last few years. It's a trade show for the casino gaming industry ('by the industry, for the industry'). I always enjoy attending.
At this show is an eclectic mix of the products needed to run a casino property: Blackjack tables, chairs, chip counters, card shoes, uniforms, platters, ID, beverage dispensers, Elvis look-a-like performers, and of course slot machines.
Slot machines now account for 70-80% of casino gambling floor revenues. They are the evergreen revenue generators for properties, and there was no shortage of them on display on the show floor this year. Why is this?
How do social media apps take advantage of this dopamine-driven learning strategy? Similar to slot machines, many apps implement a reward pattern optimized to keep you engaged as much as possible. Variable reward schedules were introduced by psychologist B.F. Skinner in the 1930’s. In his experiments, he found that mice respond most. Slot machine manufacturers use psychology to set the winning combination percentages. That's why manufacturers like Konami come up with multi-linked games with bonus screens. And, they know that players need enough tiny payouts to get instant gratification, and enough higher payoffs to keep them interested.
First let’s take look at why people gamble.
Psychology of Gambling
It’s human nature to want to win. It’s part of our DNA, and this is how our DNA gets to propagate and make more DNA. Our brains reward victories with dopamine which makes us feel good and encourages us to want to do more of the same.
In prehistoric society, these dopamine boosts could be rewards for finding tasty and palatable food. Sex feels great to encourage procreation, and parental pride helps us raise children to further our family lineage. Fans follow their favorite football teams and celebrate their wins harkening back to tribal days and clashes between clans. It feels great when your teams scores.
Winning keeps us alive; for this reason, winning feels good.
Winning keeps us alive; for this reason, winning feels good.
Pavlov (and more importantly his dogs!) showed the reward stimuli can be driven through conditioning. Building on this was the work of Jerzy Konorski, and later Burrhus Frederic Skinner, and his research into operant conditioning chambers (now colloquially called “Skinner boxes”).
Skinner boxes
Skinner boxes build on the concepts of Pavlov (who showed that you can condition reactions), to show that you can condition volition (how people make choices).
Imagine you have a box, into which you place a rat. Inside the box is a button and a tray.
If the rat pushes the button, a pellet of food is dispensed into the tray. The dispensing of the food pellet is not reactive (unlike Pavlov); the rat has to take action to get reward. If you can change how the rat pushes the button, you can show that the you have influenced the rat to make a decision; this is called operant conditioning.
Skinner experimented with the how to condition his subjects, and the schedules for how to best train them.
Relevant to this article are two amazing discoveries from Skinner's work:
It works on humans just as much as animals!
The best way to condition is not to always reward the subject!
This later point, at first, seems counter-intuitive. You’d think that constantly rewarding the subject on each action would be the most pleasing to the subject, and thus the best way to condition them. However, this is not the case.
Using a technique called variable ratio reinforcement, the rate of the reinforcement reward is a balanced between expectation (tension), and release.
Research shows that not always dispensing a reward (only giving the reward after they have pushed the button a random number of times), and varying the value of the reward by a random amount, is a better way to train a subject to continue performing the action. I think you can see where this is going …
Just about everyone who plays a slot machine knows that the odds are stacked against them. Even the loosest machines you can find pay out, on average, less than they take in. So why does a sentient person stuff $100 into a machine, when they know that they’d be better off sticking the money in the bank receiving a sum-certain rate of return? (the food pellet on every push). The simple answer is that the reward on a win feels great (especially when it is not predictable or guaranteed).
Going even further, if you could find a slot machine that paid out better than 100% of what it took in (a job!), you would find that people would still prefer to “play” this slot for eight hours a day rather than working on a production line of a factory doing the same task continuously for a known fixed price (even if this production job paid better on average).
Winning feels great!
Solitaire is arguably the most popular card game in the World, and people love to play it (even though there’s no way to always guarantee a win). Remember how good it feels to win? The feeling of pleasure from beating a game that you don’t always win is higher than that of winning a game that is a sure thing.
Slot machines are Skinner boxes. You ‘pull the handle’, but you don’t get a reward every single time. When you do get a reward, sometimes it’s a small win, sometimes a big win. But it’s easy to feel the pressure of ‘just one more pull’. The feeling of euphoria (the dopamine boost) on each win is divine, and this is compelling enough to keep you playing.
Selling the Dream
It’s not just operant conditioning (with the associated dopamine reward boost on each mini-win) that makes gambling so appealing; there is an uber reward for all gambling, and this is the ‘selling of the dream’.
When you buy a $1 lottery ticket, you are not just investing a chance to win a prize, you are buying a dream of potentially winning a huge jackpot and a life changing event.
Winning a million times your initial bet in a game is a life changing event. You know your chances of winning are microscopic, but they’re measurable, and tangible, and you read in the media about others who have won. You can project yourself into the situation. You are buying a dream. You can reconcile this in your mind as part of the cost of the bet.
Is this all bad?
Is this all bad? Is gambling the work of the devil?
If you pay $15 to go see the latest blockbuster at a movie theatre, 120 minutes later you’ll leave with nothing more tangible than the memory of the experience, and the enjoyment you received. Have you received value for money?
Do you begrudge the movie theater once the movie is over?
If, instead of the movie ticket, you invested the same $15 playing penny slots for a couple of hours of entertainment, is this any different? Is one a more ‘legitimate’ entertainment than the other?
How about going to a concert, or reading a book you purchased, playing an online video game, or drinking a glass of vintage champagne …?
I’m not here to judge; if you are aware of what you are doing, I don’t think there is anything wrong/different with getting entertainment from feeding slots (if that is what floats your boat), compared to the experience of watching Britney Spears live on a stage. If nobody is forcing you, and you are getting pleasure from your pastime of choice, good for you. I have no more right to try to convince you what you should be doing for entertainment, than you have to try and persuade me that chicken soup is “more tasty” that tomato soup. Each to their own. It’s your money, spend it, and get enjoyment, however you like.
However, there is a dangerous aspect to gambling, and this is addiction (suffering withdrawal when it can’t be consumed, and becoming dependent). It’s a medical fact that certain people’s brains are wired differently and they lose control. People can become addicted to many different stimuli (drugs, alcohol, attention, food, nicotine, video games, sex, shopping …) and exploiting these behaviors is immoral.
If you are the type of person who suffers any kind of addiction, there is no shame in seeking help. Please do so.
Modern Day Slots
There’s very little in common between a modern slot machine and the “one armed bandit” of times past. In fact, it’s becoming increasingly harder to find a machine that actually uses rotating reels to display the outcome. Most modern slots are computers with buttons, lots of lights, huge screens, sound generating devices, ticket printers, and bill readers. They don’t even take coins or tokens anymore.
Inside a modern slot is software code that runs a Pseudorandom number generator (PRNG), and this is used to render the graphics on the screen depicting the virtual reels. The random numbers generated by the code determine what to display, and if these combinations constitute a win. The casino can configure the average pay-out percentages of the machines to determine how ‘lose’ or ‘tight’ they are. Typically they are configured in the range 92%-96%, so that they pay out, over average, this percentage of the money they take in. The balance is the casino’s profit. The PRNG component is highly regulated, governed, audited, and tested to ensure fairness and credibility.
Still, many casino players have a difficult time understanding odds, and many players have superstitions about how they push the buttons, how they sit, which hand they use, and if they touch anything as the reels are spinning. The random number generator does not care what you do. In fact, it’s already decided if your spin is going to generate a win before it even animates the reels! The spinning of the wheel and the flashing lights are all fireworks and theatrics.
Machines do not “run hot” or “cold”; the chances of a win on the next spin is agnostic of the number of spins since the last win. Machines are not “due for a jackpot soon”. Similarly, a ‘almost win’, where three symbols almost, but not quite, line-up, is just as much a loss as a pile of random symbols (though, as we will see below, this event can amplify the dopamine).
Every spin of a slot machine is an independent event. There is no correlation between prior results an the result of the next spin.
There is no skill involved in playing a slot machine. It's all luck.
Dopamine Amplifiers
Slot machines are colorful, attractive, musical boxes. They play melodies, flash lights, and celebrate wins. The son et lumière shows associated with wins (even small ones) amplify the euphoric feelings we get when we win (and Pavlov showed how important feedback on these can be). When you walk through a busy casino floor it’s impossible not to hear the positive chimes of winning machines and get ‘second-hand’ dopamine boost from the success of others (Inverse schadenfreude!)
I'm sure you've heard this sound as you've walked across a casino floor in Vegas. Does it make you smile?
Slot manufacturers have clever tricks to program our senses. As reels spin, many machines play melodic arpeggio chimes that increase in harmonic complexity as each reel stops. If it looks like there’s a potential win, these sounds build up to a climax, building in anticipation. Often the last reel will slow down just as it’s coming to it’s end point to intensify the climax. As we know from the PRNG, a miss by a inch is the same as a miss by a mile to the computer, but to a person seeing two symbols stop and the third decelerate to just short of the pay-line in slow motion with the chords in their ears can be agony and the catalyst needed for “just one more try”.
It's actually illegal (in the USA), to manufacture machines that artificially generate 'near miss' situations when the player has not won. All symbols are supposed to be rendered just as the PRNG asks them to be displayed (giving the win if appropriate, or showing the exact losing symbols if not), but I don't think it's illegal to celebrate and highlight a natural near-miss if one occurs.
Multiple Win Lines
Most slots these days allow for the playing of multiple win-lines (of course with the associated increase in wager for each line played). This allows the dopamine inducing shows to be played more often, even when the pull has resulted in a net loss. Playing all lines might cost you 100 credits, but you can be rewarded with sounds and lights for prizes of say 20, 30, 30 credits for wins on some of the lines. You feel as though you are winning, even when you are losing. It’s like death from a thousand paper-cuts that you feel good about.
To be honest, anything other than a couple of simple win-lines is just too complex to follow. After the reels stop, I defy anyone to be able to detect winning combinations without the aid of the computer on the more obscure lines.
Flow
When slot player plays a lot, and at speed, they get into what psychologists call 'flow' (that interesting state of the mind in which time seems to move both slowly and quickly at the same time), and can become completely absorbed in what they’re doing. Flow was first investigated by the psychologist Mihaly Csikszentmihalyi. When in a state of flow, a machine can enable a player to work the machine at a rate along the lines of 800-1,200 pulls per hour!
Penny Slots
The ability to play multiple lines (and thus experience lots of wins even if there is a net loss), and the fact that machines take bills instead of coins have, paradoxically, turned ‘penny slots’ (where the elemental unit bet is just $0.01) into some of the casinos highest revenue units. A player can bet multiple win-lines and ratios on each pull making (potentially tens, or hundreds, of wagers on each spin), compared to betting just a single coin on a quarter or dollar machine.
Branding
Whilst there are minor tweaks that can be made to play style, a slot machine is essentially a commodity. All slots pretty much do the same thing; have the same inputs and outputs, and the same controls. Also, as we have seen above, the modern machine starts off as a modular blank cabinet. By changing the physical art assets on a case (if there even are any, as many are now displayed solely on the multiple flat panel screens), then the digital assets in the game (sound effects, pictures, animations, and music), a game can be converted from one design to another very easily.
It's easy to make a generic 'Lucky Slots' game that, absent of any other machine, might be played by a customer, but instead of just making anemic machines, manufactures partner and license recognized brands and build machines and 'experiences' around these. These can have a strong pull, as people like to affinitate themselves with brands they like.
If you are going to 'invest' $100 in a slot machine for entertainment, why stuff it in a generic no-name generic machine, when you could be playing with Joey, Phoebe, Monica, and Chandler from Friends? They are smiling at you, and playing along with you. Walking around the show floor, I found hundreds of brands licensed and turned into slots. There was something for everyone. Here's a short list of some of them I saw:
Simpson's, McGyver, Shark Week, Snoop Dog, Monopoly, Madonna, Farmville, Candy Crush, Ghost-busters, Little Shop of Horrors …
At peak, over 80 Million people a month played Zynga's Farmville on Facebook. That's a lot of people. If any of these players visit Vegas and stand in front of two slot machines, one a generic machine, and another branded with all the logos and symbols they recognize, which do you think they will play?
Machines are multimedia devices that can stimulate all our senses. Our brains latch onto sounds and well as still icons and moving images. A successful branding uses all these to pull in customers.
I'm sure most people are able to identify the branding on this machine from the sound alone.
(Machines branded with the above property sound have been the most popular branded slot for the last two decades!)
What about other casino games?
Slots are not the only games available at casinos, why are they so popular? What about the other games, and the table games?
There are many reasons why slots are so popular, but the first is the low-threshold to play. It takes nothing to walk up to a machine. Even a shy and self-conscious person can do it (or find a machine in far away corner away from others). It's anonymous, and you don't have to talk or interact with anyone. The rules are fixed, and there is no skill. There is no cause of embarrassment for 'doing it wrong' as one is not able to make an incorrect move. It's safe. Playing slots is a personal thing, and you don't often find it a spectator sport. Finally, the odds on slots are fixed and (compared to many casino offerings) are not bad. Anyone can play slots, even introverts. It's the catch-all game.
Next up the 'food chain' is video poker. This requires skill to play optimally so there is a rules barrier and learning curve. However, it's still a personal activity and there is none of the peer pressure or embarrassment of potentially making a bad move in a public arena. Introverts can still play, but we've lost potential audience from those intimidated by the rules/odds.
Next we move to table games like Blackjack. Blackjack requires skill (The absence of skill can make you lose money a lot faster). Blackjack is still random with the shuffled cards, but there is a optimal strategy to play based on the cards you hold and the dealers face-up card. Play stupidly and you can lose money very fast. Play optimally and (depending on the table rules and deck size), you might even have a very slight advantage over the house.
Whilst Blackjack is a social game, played by (potentially) many players at once, the actions of any one player do not affect the game for anyone else. In fact, the opposite is the case; all the players, together, have a common adversary, the dealer. If the dealer busts, it's group hugs all around the table; everyone wins! If the person to your left hits a Blackjack or hits 21 on a double-down you can give them a high-five and be pleased for them. Blackjack is not adversarial between players. Blackjack is a social game.
Blackjack can be played with friends, or with strangers.
Top of the pile are games like Poker. In poker, it's every-person for themselves. You are playing against the other players. A dealer in poker facilitates the game, but does not actively participate (other than taking a cut of the bets). Poker is an adversarial game, it's not social. A game of poker is a game of war. Being aggressive is an essential part of the game (this is not everyone's cup of tea), as it bluffing (some people hate to lie, or are just bad at it). Poker is about winning and beating your opponent. There's a good amount of skill in the game, it's not just luck (as can be seen by the repeated times some of the same characters have won the World Series of Poker events. If it were purely random, it's fairly likely that we'd see different people on the final tables every year, but we don't). It's a small subset of casino guests that enjoy poker.
Rewards
86% of people in the US belong to some sort of loyalty program. Things like frequent-flyer cards, credit card points schemes, coffee shop discount cards, hotel chain programs … Rewards programs keep people loyal to brands.
If you're going to buy a flight, if there's not much in it, you'll probably book it with the airline you have a relationship with. In turn, the airline will award you points and status that you can use for upgrades, free flights, or other rewards. Casinos are the same. They want your business, and they'll incentivize you to patronize their establishment over their competitors.
It's somewhat symbiotic. Casinos want repeat business, and players want to feel special. In the commodity world of casino offerings, each casino wants to encourage a player to exclusively place all their eggs (bets) in one basket, than spread them thin over multiple establishments.
Casinos offer rewards cards that are inserted into a machine before play begins. Wagers placed garner points (which can be exchanged at the casino for tangible benefits such as free food, and even complimentary show tickets and rooms). The casino is trading these prizes for your loyalty, but in addition they are tracking your usage patterns. Anonymously aggregated, this data has use to the casino to help it balance utilization, and how it might want to configure the casino floor layout, but individually it allows for targeting players based on their behaviors, needs, and actions. Casinos called this targeting 'player development' and this is marketing speak for the attempt to groom a player and move them up the value chain.
The correct branding on a machine might get a casino the 'first date' from a player on the machine, but it is probably the reward program that will get them the 'second', 'third' and 'fourth dates'.
Social Casino
What about social casino players? These are people who play casino games on Facebook and other social media.
There is a subtle distinction between social casino players and online casino players. Online players are still gambling with real money, it's just that they are doing it through their browsers and not from a 'brick and mortar' casino establishment. Until recently, in the USA, it was illegal to gamble online. Slowly this is changing, and some states are allowing it in controlled ways. It's fair to say that online gamblers are cut from the same cloth as gamblers who frequent physical casinos. They are risking real money for a chance to win real money; their motivations are the same as physical casino players.
But what about social casino players? At first glance these seem like strange beasts. A social casino player will exchange real money for virtual 'gold', and this is the currency they spend in their games. There is no way to get the money out! Zero. Even if they win billions of virtual coins, they can't cash them out. There's no mechanism (if there was, it would be illegal gambling). They can't get the money out in other way either; their balance can't be exchanged for a prize or anything with monetary value. A social player deposits physical money as a one-way transaction.
A social player deposits physical money as a one-way transaction.
So what motivates a player to pay a social slot machine? Some of the Skinner box principles described above still apply; it's still fun to win, but for social casino players there are different motivations. Social casino game designers have implemented different rewards mechanisms to achieve, obtain, or master. They offer 'badges' for completing certain tasks, levels, or obtaining certain achievements. Humans like to collect things; we like complete sets of things (whether it is baseball cards, toy cars, comic books, thimbles, or Pokemon). Social casino players like to obtain badges; and they also like to chat online. Visit a social casino bingo game and you'll often find bunches of friends chatting whilst playing. (Warning - if you enter one of these rooms, as an outsider, and call a bingo too early, you might be the target of some wrath as the others in the room were playing in a more social way and wanting to wait for others to complete so they could all call bingo at the same time, sharing the badge/prize).
Interestingly, many social casino players are happy playing their virtual casino games, and don't seem motivated to graduate to real casinos and real money. This came as quite a surprise to the industry. Profits at physical casinos has been declining in recent years (correlated, inversely, with the prolific rise of smartphone adoption). There was paranoia in the casino world that players would not need to visit physical casinos and would be able to gamble, without even having to get dressed, from the comfort of their own homes. The slot manufacturers also sensed a possible opportunity to cut out the casinos and deal directly with the players; if they could connect directly with players on their phones, once the online gambling laws crumbled, they'd have the relationship.
These two factors compounded and their was a land grab for social casino companies. High premiums were paid to obtain social casino game developers and their associated customer databases. The theory was that these social casino players could be groomed into profitable clients by converting them over from spending virtual coins into spending real coins. For the reasons mentioned above these conversions haven't been anywhere near as successful as planned, and many of these social gaming companies have changed ownership again.
Technology
The casino industry is ripe for leveraging technology. The tracking of players through the rewards programs discussed above is bread and butter for the Internet industry who have it down to fine art. Vast amounts of data are recorded for these player development programs (and to optimize casinos at both a macro and micro level). For the first time at the show I encountered a Google booth, and they were there offering their cloud, Machine Learning, and AI services to casinos.
I saw plenty of booths displaying heatmaps of floor regions showing where money was being earned more efficiently, and plenty of excel like tables showing sample databases of players and their spending patterns and rates.
Blockchain was another buzzword on the lips of many, but surprisingly I heard and saw no mention of any type of Cryptocurrency.
The dream of blockchain adoption is to help with affiliate marketing. With so many casinos (on and offline) offering similar products, they key to growing a business is bringing in more fresh customers. Casinos will pay a bounty for each new revenue generating customer, and this is significant enough to build entire industries around. Rather than just a finders fee, deals are often structured as affiliate marketing deals. These are revenue sharing deals in which the casino pays a percentage of the revenues from any player delivered for a defined period of time. Obviously, to administer this requires trust and auditing; with the associated complications and costs of both of these. Using blockchain to validate all transactions could provide a simple way for affiliate marketing to correctly attribute and compensate the lead generator.
Other interesting uses of technology seen at the show were facial recognition (to track players for loyalty programs, even if they didn't insert their card, or were playing table games), as well as to detect cheating, suspicious activity, and ticket theft. It's no surprise that casinos have hundreds of cameras already; it's not a leap to imagine these moving to provide a digitally enhanced service.
See you next year Global Gaming Expo! It was a fun show.
Positive reinforcement, using food rewards to increase the likelihood a dog will repeat a desirable behavior, is universally regarded as the most reliable method for teaching commands. While the basic concepts of rewardbased training are easy to understand, people sometimes inadvertently inhibit progress by using too many—or too few —treats.
Let’s say you’re in Las Vegas playing a slot machine, but every time you deposit a quarter and pull the arm, you get your one quarter in return. This wouldn’t keep your attention for long, and you’d probably opt for a different machine.
Now, what if you started feeding your hard-earned quarters into the next machine, but for hours on end got none back? Chances are you’d become equally frustrated and end your short gambling career.
Applied to dog training, both of these extremes—continuous reinforcement or none at all—can lead to lower command compliance.
GET THE BARK IN YOUR INBOX!
Sign up for our newsletter and stay in the know.
“My dog will only sit if I have a treat.” Over the years, I have heard this refrain many times, and it almost always indicates that the dog was rewarded with treats for sitting on cue too often and for too long. Essentially, the dog had learned two things had to be true for him to comply: the sit cue plus a treat. If either were not true, he’d find something more interesting to do.
When initially teaching a new command, “Continuous Reinforcement”—CR in the geeky learning-theory world— is the most effective approach. For instance, when first teaching a puppy to sit, rewarding each successful completion (or “trial”) makes sense because your focus is on clearly pairing the verbal cue and hand gesture with the behavior: put the quarter in (your puppy sits on cue) and the reward appears (treat!).
But acting as your puppy’s loose slot machine for too long causes him to stop working so hard. Why bother sitting quickly, or at all, when a treat invariably appears? CR for too long also causes the dog to become dependent on the food reward: he will refuse to work unless food is presented. Before you get to that point—usually within a few days of teaching a new cue —it’s time to move to a less predictable reinforcement schedule.
Back to the gambling analogy. Once you’re sure your dog has a grasp on what you’re teaching him, it’s time to become a fair and honest slot machine, dispensing small food rewards less frequently for successful trials. (This is also a good time to find soft treats that won’t easily crumble to bits, and to always have a few hidden in your pocket.)
The psychology behind slots— enticing folks to pump coins into machines for hours on end—is that the probability of winning remains constant, even though the number of plays it takes to recoup your money, or better yet, hit the jackpot, changes. The unpredictability makes doing the same mundane activity, over and over, interesting and exciting. You can take advantage of this same psychology to train your dog faster.
When teaching your dog a new command, once you’ve determined that he knows what you’re expecting from him, begin randomly rewarding successful trials using “Variable Ratio” (VR) reinforcement. Start with a low ratio, rewarding roughly one out of every three trials, then increase the ratio over the course of several training sessions.
Slot Machine Reward Psychology Definition
For example, when teaching your puppy to sit, provide a small treat for (successful) trials 2, 7, 9, 15, 18, 19, 20, 23 and 25. Notice that during 25 trials, sometimes he gets three rewards in a row, but sometimes, there’s a longer lag between treats. The idea is to keep him guessing—and working!
Over the course of twice-daily training sessions (two to five minutes each), increase the ratio until he is rewarded for roughly one out of every ten successful trials. The behavior should become a happy habit by then, although, to keep commands fresh, continue to occasionally reward your dog for life. In other words, don’t become the slot machine that never pays a jackpot!
There are other types of reinforcement schedules too involved for our purposes here, but one to take advantage of is “Differential Reinforcement of Excellent Behavior” or DRE. This is just a fancy way of saying “better performance earns bigger rewards.” Once you’ve worked through Continuous Reinforcement (treating every time to teach the command) and Variable Ratio (treating randomly to hone the behavior), you can polish the command by handsomely rewarding only the best trials.
Let’s think about DRE in terms of teaching recalls. Once your dog is largely responding to your “come” command, and you’ve worked through Variable Ratio reinforcement—by sometimes treating and sometimes not— start rewarding with higher-value treats, or more of what you have, only when your dog immediately and enthusiastically answers your call. If he stops and smells the roses (or whatever that was) en route, no reward is given.
Slot Machine Reward Psychology Program
Advancing through these levels is not rigid, and you may combine aspects of more than one as you progress. Be ready to back up a step if you’ve moved too fast—your dog will let you know!