THE SCIENCE OF LEARNING TO LEARN

READING: GET FEEDBACK

The first sign was probably the three pointer. It was an hour or so into the basketball game, and my T-shirt sagged with sweat. Blood thumped in my ears. Adrenaline, the hormonal Red Bull, ran clear and strong. I caught the pass on the wing, and my defender gave me a long moment at the three- point line.

I was not supposed to launch a shot. For years, I had been one of the worst players on the court, barely good enough to make a lay-up. Other players would often overlook me on offense. Opponents would target me for steals, a bit of easy basketball pickings. Even among a group of slow, weak, middle aged basketball players, I was slow, weak, and middle aged. 

Still, I was by myself, wide open, so I tucked my elbow and fired the ball with a brawny sort of aim. The ball lofted towards the basket, an orange orb, hanging in the air, a small slice of eternity, and then, surprisingly, the basketball slipped into the bottom of the net, with a loud snap.

“That was a three,” someone yelled out.

Did that actually go in?, I thought. And then a follow-up notion: Were my basketball classes behind this?

The story of my basketball career is a short one. I played a lot as a kid, and in my middle school days, posters of Magic Johnson hung in my bedroom. But in my late teens, I gave up the sport. Other interests took my attention. School work pulled me into its orbit. For a long while, I stepped onto the court once a year, largely to play a hacking, street ball with my brother.

A few years ago, though, I started playing in a Wednesday night pick-up game, and I fell in love all over again: The exhausting workouts, the pride of a well-placed shot, the post-game trip to the bar. But I was the guy that no one wanted on their squad. On some nights, I might play a couple hours without landing a bucket. My only redeeming quality, it seemed, was the dedication with which I fouled my opponents.

Then, one afternoon, I found basketball instructor Dwane Samuels on Craigslist. During his twenties, Samuels had played basketball at some big name colleges, notching up minutes in summer leagues against NBA All-Stars like Ben Wallace and Jerry Stackhouse. Later, Samuels found a spot playing for the Washington Generals, the Harlem Globetrotters’ perennial opponents.

While Samuels was now retired from any sort of professional basketball, he was still a human velociraptor, large and muscular. During our first session, he had me running sprints and jumping rope and dancing through an agility ladder. Eventually, Samuels tossed me a Spalding and had me review the most basic of basic moves—dribbling through a set of small orange cones, shooting layups, taking two-foot jumpers.

Samuels had moved to the United States from Jamaica as a teenager, and in his Caribbean-tinged accent, he always kept up a slow steady patter of advice and encouragement as I repeated some of the same drills that I had once did in elementary school. “Keep your elbows in,” and "good job, try again” and “now aim for the top of the backboard,” he’d say.

Shy, embarrassed, I didn’t tell anyone about the classes—not friends, not family—certainly not anyone that I played basketball with. A lead-footed forty-something should not be going to basketball classes. Tennis classes? Maybe. Golf lessons? Fine. But basketball is a young person’s game, fast and quick, and most of Samuels’s clients were children who were about the same age as my own.

Within weeks, though, my jumpers started falling regularly. I hit three-pointers. Other people began to notice, too, and a friend asked me for advice on his form. “You were ‘Mr. Lights out!’ with the shooting,” someone else emailed me. Indeed, my game improved so much that a teammate jokingly asked me if I had been shooting steroids.

How could a few lessons have such an impact? Was Samuels some sort of mad-genius instructor? Or was there something about the nature of practice that I had missed after all these years?

The answers bring us to the value of feedback. More exactly, when we learn to learn, we have to enter ourselves into a feedback loop, to hone our skills in a structured sort of way.

"...when we learn to learn, we have to enter ourselves into a feedback loop."

In this sense, what many people call practice isn’t really practice. They’re not engaged in a dedicated way to improve, not using any sort of learning method. There’s a good amount of research on this idea, and the raw amount of time spent practicing often bears little relationship to the actual amount of learning. Indeed, more than forty years worth of research shows that study time does little to shape student outcomes.

Or just consider that some first-year college students will have clear misconceptions about basic physics, even after they’ve solved more than 1,500 basic physics problems. So while the students had banged out Newtonian problem after Newtonian problem in high school—practicing problem after problem—the students still couldn’t really explain Newton’s Third Law.

The Knowledge Effect that we discussed in the reading “Building Knowledge” plays an important role here. Whether it’s playing against the Globetrotters—or improving our Latin—it’s hard to figure how to develop a skill unless we know something about the skill. In this sense, every beginner lacks the metacognitive ability of knowing what understanding that they need to develop. That’s, after all, why they’re beginners.

I can’t get better at something like urban planning because I don’t know enough about the topic. For another example, consider that birding experts can tell the difference between the 300 different types of doves. I’m an amateur, so, frankly, they all look like pigeons to me, and not surprisingly, this limitation makes it hard for me to get better at spotting the difference between a wood dove and a collared dove.

"In fields like medicine, this happens despite the fact that weak handwriting skills causes some 7,000 deaths each year."

There’s a lot of culpability to go around, and we also simply don’t pay enough attention. Handwriting is a great example. After grade school, we tend not to practice our penmanship in any dedicated way, and so our skills get worse. Our “g”s will look more like an “s.” Sentences will look like a set of tiger claw marks. In fields like medicine, this happens despite the fact that weak handwriting skills causes some 7,000 deaths each year.

When it comes to developing a skill, though, the most damaging might be a lack of structure. Because without feedback, without monitoring, without dedicated repetition and engagement, practice is little more than a bit of goofing around. When I reached out to training expert Anders Ericsson, for instance, he argued that when most people practice, “they don’t have a clear idea of what they should improve, and so they’re just wasting their time.”

This type of feedback doesn’t take much effort to be effective. My favorite example comes from Mark Bernstein, a brain surgeon in Toronto. Bernstein began tracking every single mistake that occurred in his operating room over a ten-year period. If a tube fell to the floor, Bernstein made a note of it. If a suture didn’t stick, he’d record it. Even just a bit of miscommunication between Bernstein and a nurse would go into his database, tagged with various details like the date of the event and the age of the patient.  

Later, when Bernstein and his colleagues looked closely at the data, it turned out that the effort to record errors had a tremendous impact. By writing down mistakes—by creating a feedback system—Bernstein and his team made far fewer gaffes. The effects were immediate, a performance steroid, and his team's surgical error rate plummeted over the first year. What’s more, the effect continued to hold for more than a decade, with Bernstein’s error rate eventually declining from over three mistakes per month to just over one and a half errors per month.

It turns out Bernstein was engaged in one of the most basic forms of feedback, a practice known as “monitoring” in the language of psychology. According to researchers, the practice often comes down to a form of awareness. In order to track outcomes, people have to notice what’s going on. In Bernstein’s case, this was about attending to mistakes, looking for slip ups, reviewing gaffes, and his team’s oversights were often pretty obvious. If a machine didn’t work right—or a scalpel clanged to the floor—there was an error.

" In order to track outcomes, people have to notice what’s going on."

But generally speaking, Bernstein’s mistakes were more subtle, small things like a poorly positioned sponge, a delay in anesthesia, a misheard word or command. And it’s in this regard that monitoring is so important. To find errors, we need to track errors, to observe our gaffes, and after each surgery, Bernstein would log each mistake in a database, noting the severity of the error, the type of mistake, and the degree to which it could have been prevented.

To help monitor performance, people might also use journals or diaries. In my case, for instance, I’ve long kept a file in which I’ll ruminate on my writing performance. Like many people, I frequently make grammatical mistakes, often confusing the distinction between “which” and “that,” for example. I’ll make note of my slip-ups and ponder about how to avoid errors—or boost performance—in the future.  

Still others swear by video as a way to track outcomes. In football, for instance, there’s NFL coach Jon Gruden, who has a massive library of football game video. Gruden, formerly with ESPN and now coaching the Oakland Raiders, still keeps up the massive collection of footage, including videos of practices going back more than two decades. As Gruden once told a reporter, “Over there are [footage of] three-step throws. Dropback. Two- and three-jet-slide protection. Split-flow-man protections. Play-action passes. Then over here I have goal line. No-huddle. Wall draws. Green dogs. Effort tapes.”

Part of the benefit of this sort of monitoring is that it pushes us to be more aware. When we tracking our performance, we become more focused on improving, which is quite useful because we often pay almost no attention to our performance. Driving is a good example. Few of us make any effort to get better at navigating a car. Indeed, most of us still park as poorly as we did when we were seventeen, or we brake too much going into curves. I’ve watched people drive straight for miles with their blinker signal still on.

Public speaking is similar. Most of us often have to give talks in front of large groups of colleagues. We’re called on to present in front of bosses or clients. But people frequently make the same mistakes over and over again like a broken wind-up toy. They speak too quickly, or they don’t make eye contact, or they twist a ring on their finger for nervous luck.

What’s easy to forget is that we’re all part automaton. Whether it’s playing football or doing brain surgery, it doesn’t take much for a task to become a mindless habit, an unthinking custom. This helps explain why monitoring can be so powerful. We are stepping out of automatic mode and asking ourselves:

• Am I doing this correctly?
• Did I make a mistake?
• How could I do better?

When we track performance, patterns of behavior also become more clear. In Bernstein’s case, he found that an overwhelming percentage of his team’s surgical errors was preventable, something easily avoided like the contamination of a scalpel. Surprisingly, Bernstein also discovered that seeing more patients generally led to fewer errors, not more. Also, inexperienced nurses and doctors joining Bernstein’s team did not appear to cause any noticeable uptick in mistakes.

Granted, this sort of close monitoring has its downsides. Tracking outcomes can be embarrassing. For me at least, it remains shaming to admit that I still mix up “which” and “that,” even though I’m a professional writer. Worse, Bernstein’s team once dropped a piece of someone’s skull “about the size of a playing card” onto the floor during a surgery. “Mortified” is how Bernstein describes the experience.

To be clear, there’s actually a more powerful form of feedback than monitoring, and it typically requires some sort of external evaluation, some outside criticism. Indeed, more than anything, it was external feedback that boosted my basketball skills.

This fact was clear in my early practices with Dwane Samuels. Like a type of soothsayer, he could see things that I could not see, revealing where I simply had no sense of the scope of my weaknesses.

Take something like facing the basket while taking a jump shot. As basketball goes, the idea is canonical, the first commandment of the shooter’s bible, something I had come across many times before my basketball classes. 

But without even really knowing it, I would fire off the basketball at a diagonal slant to the basket, twisting like a pre-teen ballerina. Samuels pointed out the issue on our first practice, and got me to change my footing. It took a few more weeks, but the change made my shots far more likely to go in.

"We need outsiders to offer targeted criticism, to give outside evaluations."

The value of this sort of feedback goes well beyond basketball, of course. In any endeavor we monitor in an attempt to improve, we can’t uncover all of our errors. This is the nature of learning, the nature of knowledge, another reminder of the value of educators. We need outsiders to offer targeted criticism, to give outside evaluations.

Take writing my book Learn Better as another example. When I submitted a draft to my editor, I had read the text countless times, reviewed each sentence with the devoted care of a 17th century monk. Typos? I thought there weren’t any. Gaps of logic? I assumed that I had addressed them all.

But my editor, Marisa Viglante found rough spots—obvious gaffes, limp reasoning, unsteady prose and structure. Even before I submitted a draft, Viglante told me that this happens with just about every author, famous or not famous, new or experienced. “It’s impossible to edit your own work for this very reason, no matter how smart or skilled you are,” Viglante told me. “I’m your second reader.”

This idea also explains why outside criticism is often a little humiliating. After all, it’s hard to hear that we’re doing something wrong, especially when we know we could be doing better. More exactly, it often pained me to see Viglante’s edits, because they were so on target.

Without question, there are issues with feedback. It can go too far, for one and, even if it’s good feedback, it doesn’t reveal how to correct or improve. People still need to demonstrate their learning, and, generally speaking, helpful feedback shows the way. It gives us a pathway that leads to our development. 

Let’s say, for example, that you thought that the Spanish word for rooster was “pollo.” A weak form of feedback would have just given you the answer (“you got that wrong; the correct answer is gallo”). Or, it might not provide any feedback at all (“Please go to the next question.”) 

The best feedback mixes an observation with a structured way to produce the proper outcome. In the rooster example, for instance, the most effective feedback would indicate that the answer was wrong—and then would provide some slight hints. (“The correct translation for rooster starts with a ‘g.’”) If someone still doesn’t provide the correct answer, then perhaps another tip (“think ‘ga’”) until the correct answer (“gallo”) is produced.

"The best feedback mixes an observation with a structured way to produce the proper outcome."

Recent studies by researcher Neil Heffernan and others have shown that feedback needs to come quickly, the more immediate the better. In many cases, even 24 hours is too long a delay to be useful.

This awareness shifts our idea of what grading should look like. After all, a common teaching scenario is where the students turn in an assignment, the teacher grades that assignment outside of class, and then returns the students’ work the next day, or even a few days later. This type of delayed feedback is ineffective, adds Ken Koedinger, an expert in the science of learning at Carnegie Mellon University. “That’s just too long. Students aren’t even in the mind frame anymore.”

This sort of fast but structured feedback is important early in the learning process, where thoughtful criticism and guidance can have a tremendous impact for beginners. But over time, feedback should fade—and people should do more to produce their own answers, to engage in more mental doing, to create understanding. “Simply re-presenting a fact or concept, whether as feedback after an error or not, is far less effective than having people ‘generate’ the information,” psychologist Bob Bjork told me.

The role of feedback also explains why curriculum is so important, and textbooks and worksheets and other forms of practice turn out to have a huge impact on what we learn. I’ve seen this in my own research with my colleagues Matt Chingos and Chelsea Straus, and the effects of a high-quality curriculum are about the same as the effects of a high-quality instructor, even though high-quality curriculum is often cheaper.

To put it differently, if you’re a student who has a bad teacher and a bad curriculum, you’re better off fighting for a better curriculum. The outcomes are about the same, but the cost is lower—and frankly, it’s often a lot easier to get a new textbook than finding a new teacher.

So what does shoddy curriculum look like? Well, the feedback is bad, and the practice sheets and textbooks will often simply provide answers to students without pushing them to build their thinking. Weak textbooks also tend to be shallow, covering lots of different topics in very cursory ways so that students don’t get extended practice in an area.

Despite the evidence on feedback, it seems that people generally have to come to this conclusion on their own. We need to discover our own desire for outside advice. Not long ago, the doctor and New Yorker writer, Atul Gawande hired a coach to help him develop as a surgeon. It was difficult at first. Like priests, doctors often work behind irreproachable doors, and Gawande felt embarrassed. No one had observed his work in the operating room for almost a decade. “Why should I expose myself to scrutiny and fault-finding?” he asked.

"We need to discover our own desire for outside advice. "

Even for someone at the top of his career like Gawande, though, the effects were clear, and he gained clear insights into his practice, developing new skills and approaches. Over time, Gawande also provided better support for other doctors, letting his residents grapple with key ideas before helping them. But perhaps above all, though, Gwande writes, “I know that I’m learning again.”

It wasn't all the different with Samuels. It was a matter of learning again, and before I signed up for classes with him, for instance, I would go to the local court to train my skills. I’d shoot around for a half hour, throwing up long jumpers. I didn’t really center my efforts, though. I didn’t get any feedback on my footwork or hone certain moves. For a long time, I didn’t even track the number of mid-range jump shots that fell in.

The experience with Samuels, then, was poles apart. We worked on very detailed aspects of the game like a short jumper from the block or a one dribble pull up. Between trainings, I had discrete homework like lying on my back and practicing my shooting form. We once had a conversation about how my middle finger should roll of off the ball—my index finger should dip like it was going into a glass of water.

What’s more, I struggled, and during my practices with Samuels, I was often uncomfortable, uneasy, doing things that I had never really done like dribbling behind my back and setting my feet in a wider stance while shooting. And a lot more than I'd like to admit, the classes felt embarrassing. During a practice session, Samuels once blocked one of my shots with such strength that the ball bounced off the wall of the gym with an echoing thwack.   

Samuels once called it “spoon feeding,” and during our practices, I wasn’t practicing lay-ups as much as I was practicing hitting the ball high off the glass. I wasn’t developing my shot as much as I was making sure that my feet were well-placed. Sometimes Samuels would hold my shoes or move my hips to get me into the proper position, and it all made me improve at a much faster rate. “Details first,” Samuels would often say, “Details first.”

As for my first three pointer, I reached out to Samuels the following day. In the email, I crowed that I had swished a twenty footer. He shared my surprise—and enthusiasm. An email fired back from him within the hour: “The sky is the limit now!”