Let me start by asking: Which does your dog like better, the anticipation of something good, or the good thing itself? Counter-intuitively, the research of Jaak Panksepp suggests it’s the anticipation of a reinforcement that is most enjoyable, not the reinforcement itself. Panksepp calls the emotion of anticipation the “seeking or wanting system,” versus the “liking system” that kicks in once an individual gets what it wants. In other words, which gets your dog more excited, hearing the clicker, or getting the treat? Hearing “Let’s go on a walk!” or actually going on the walk? (See here for a speech on this system by Panksepp.)
I thought of this a while ago when walking into the Overture Center in Madison, Wisconsin to see a show by the famous comedian, Jerry Seinfeld. I floated in on a sea of dopamine, barely believing I was going to see in person the man who had left me helpless with laughter in my own living room for so many years. After we got inside we found our seats, the lights finally dimmed and the man himself walked out onto the stage. Wheee, I was so jazzed! And he was good. We laughed a lot. But the giddy, roller coaster-like high I had felt as I walked into the auditorium was gone, and midway through the show I wanted it back.
It was Simon Gadbois’ thoughtful talk at SPARCS a few months ago that got me thinking about this topic again. One of Gadbois’ primary points was that learning is driven by motivation, and motivation is fueled by anticipation. He has done a lot of work with scent detection dogs (fascinating projects related to teaching dogs to find endangered species) and argues that good training capitalizes on the “seeking system,” (the click, if you are a clicker trainer) not the “liking system” (or the treat that comes after the click). For those of you interested in neurobiology, “liking,” or the feeling a dog has while eating a food reward, follows the same neurological pathway as morphine, while “seeking” stimulates the same route as cocaine. (See an interesting paper on neurological distinctions between “liking” and “wanting” by Berridge et. al. 2009.)
Experimental research suggests that it is “seeking” rather than “liking” that best motivates an individual to learn. For example, Gadbois mentioned one of Panksepp’s studies in which cats were always given a reward when they touched one object, but only occasionally when they touched a second object. Guess which object the cats touched most? You got it, the second one. That is why Gadbois argues that clicker trainers should not give a treat every time they click. This all makes great sense to me until I think of chocolate, which I would much rather eat than anticipate eating, thank you very much.
You can imagine that Gadbois’ argument led to a spirited discussion. I was fascinated, because for many years I have wondered why standard clicker training always follows a click with a treat. Karen Pryor strongly advocates for us to reinforce every click [secondary reinforcer] with a treat [primary reinforcer]. Ken Ramirez, one of the best animal trainers in the world in my opinion, always follows a click with a treat. But Gadbois went farther, given the link between motivation and anticipation, suggesting that it was important to balance the “seeking” and “liking” systems, with more emphasis on the former than the later during training. He strongly advocating for not following every click [which creates anticipation] with a treat, far from it, for the reasons described above.
I’ve always argued that family dogs should be reinforced both intermittently (once the behavior is established) and with different types of reinforcement early on. Asking a dog to sit when asked? Decrease the frequency of reinforcement as the dog becomes more reliable (depending, of course, on the level of difficulty) but early on vary the reinforcement, mixing food with play and belly rubs or whatever else the dogs loves. I suspect my inclination has been in part based on what I’ve read in the learning literature and my own experiences over the years. I hereby admit I have never quite understood the importance of reinforcing every click with a treat (or other reinforcement), and so you can imagine I listened to Gadbois’ talk with special interest.
[Full disclosure: I’ve added the following based on some early excellent comments and a chance to reflect on the issue at more length.]
Keep in mind that, based on Pavlov’s research, the click is effective because the trainee has been classically conditioned to respond to the click as a secondary reinforcer. What surprised Pavlov was that the conditioned stimulus (like the click) had the same power, and elicited the same response, as the unconditioned stimulus (the food). Thus, if the click is as powerful as the food, Gadbois asked, why use the food? The answer might be in a talk given by Susan Friedman, (thank you Erin for sending in this link!) that advocates for always pairing a treat with the click, but fading out the click as a means of creating intermittent reinforcement.
Anyone want to pick up the debate? What about you? How do you balance “Seeking” and “Liking?” If you are a clicker trainer, do you always reinforce a click? Always click a correct response? If so, why?
[On a parallel note, Panksepp has always been one of my heroes. Besides being a stellar neurobiologist, he’s the guy who discovered that rats giggle. You can see a video of him initiating and recording their laughter here. Here, also, is his TED talk for those of you who haven’t seen him. And I was fascinated by Gadbois’ work using scent detection dogs to facilitate research on endangered species.]
MEANWHILE, back on the farm. Jim, the Border Collies and I are just back from a heavenly weekend at a Kathy Knox sheepdog clinic in Hinckley, Minnesota. Heavenly is the right word, because it was a perfect weekend. Kathy is an articulate and excellent trainer, the setting was heart-swellingly beautiful (and thank you “other Kathy” and Steve for hosting!), and I learned enough to make my head spin. Willie had a wonderful time, and seemed to love every minute of it, which made me happier than I can say. He did beautifully too, which was gratifying after his strange performance at the last trial.
Maggie struggled with sheep much flightier than she is used to, which provided a great opportunity for me to learn how to help her gain confidence and get control over the flock. I illustrated what I call “infinite trial learning,” and will go to sleep every night repeating: “Watch the sheep, Trisha, watch the sheep.” (I am a split second late when handling Willie because I focus too much on him instead of what the sheep are doing. I know perfectly well to “watch the sheep” because they will tell you what your dog is doing. I just seem to falter on converting that knowledge to practice. Thus the phrase “infinite trial learning….” and the following assigned grades: Willie–A. Maggie–B. Trisha–C, but extra credit for continued effort.
As usual, Jim made it twice as fun by being his usual perfect travel partner, not to mention schlepping Maggie’s crate in and out of motel rooms with nary a complaint. What lucky dogs Willie, Maggie and I are.
Here’s Odi, the good dog of Janet McN, who helped keep the sheep under control for the rest of us. What a guy.
This is Beck and Skye, watching another team of dog and handler the sheep. (For the love of God, Trisha, watch the sheep. Just like the dogs.)