How To Learn Online

READING: SYSTEMS THINKING PART II

When it comes to learning online, it's crucial to look into the deeper structure of an area of expertise. I learned this myself in an awkward sort of way when I first met up with psychologist Robert Goldstone.

A professor at Indiana University, Bloomington, Goldstone is tall and bald with a wry smile. We met up at a coffee shop in downtown Washington.

“You seem like a smart guy,” Goldstone said after we’d been talking for a bit. “Can I put you in the spot?”

“Sure,” I said, nervously fingering my notepad.  

Goldstone then presented me with a version of the following problem:

An aging king plans to divide his kingdom among his daughters. Each country within the kingdom will be assigned to one of his daughters. (It is possible for multiple countries to be assigned to the same daughter.) In how many different ways can the countries be assigned, if there are five countries and seven daughters?

After Goldstone finished, I wrote down some of the key points, noting the five countries and seven daughters. Then I started to draw the provinces. Would a representation of the situation help me figure out the answer?

“Does it have something to do with factorials? That somehow seems familiar?” I said.

Goldstone scratched his neck. “You’re getting closer.”

I kept working at the problem.

“Can I give you a hint?” Goldstone said. “If the king gives Germany to one daughter. He can still give France to the same daughter.”

I nodded but still struggled, and eventually Goldstone just explained the answer: "If there are seven options, or daughters, for each of the 5 things, or kingdoms, that need to be assigned to an option, there would be 7 X 7 X 7 X 7 X 7 or 7^5 possibilities," he explained.

Goldstone explained that the problem hinged on a math concept known as sampling with replacement. The topic was typically taught in middle school and could be boiled down to formula: “The number of options raised to the power of the number of selections.”

So why did I get the answer wrong? To answer that question, it’s important to first understand the nature of problems, and psychologists like Goldstone describe problems as having both surface and deep features. Surface features are typically the concrete, or superficial elements. In the king problem, for instance, the surface features were the lands and the children and the age of the king.

"Surface features are typically the concrete, or superficial elements."

The deep features tend to be concepts or skills, and in the king problem, the deep features were "the notion of sampling with replacement, the concept of an option and the concept of a selection event," according to Goldstone. In my case, then, I couldn’t see the deep feature. The superficial elements distracted me, and so I couldn't uncover the deeper features.

As we sat next to the coffee shop window, Goldstone argued that people often get distracted by the superficial elements of a problem. He calls it “the greatest cognitive difficulty.” Take a look at this problem for another example:

A homeowner is going to repaint several rooms in her house. She chooses one color of paint for the living room, one for the dining room, one for the family room, and so on. (It is possible for multiple rooms to be painted the same color or for a color never to be used.) In how many different ways can she paint the rooms, if there are 8 rooms and 3 colors?

The problem also comes from Goldstone’s study. But unless you’ve had some experience in sampling with replacement, it’s not immediately clear that the problem is also getting at the same issue of sampling with replacement. In other words, it's hard to grasp that the problem has different superficial features but gets at the same deeper features. "To see this connection you need to see the role that daughters and colors play in their respective scenarios--they are alternatives," Goldstone argues.

So how do people see the deep feature in a problem or area of expertise? Well, one of the easiest ways goes back to the notion of systems, of relationships, and it often pays to mix up our learning. When people see multiple examples with different surface details, they're far more likely to understand the underlying system.

Goldstone has seen this himself in his lab: If people come across a variety of different sampling with replacement problems with different surface features, they're far more likely to understand the core idea. They get a much richer sense of the deeper system.

The Power of Mixed-Up Practice

A library of research provides further support for the value of mixing up our learning. In one study from the 1990s, some young women learned to fire off foul shots. Some practiced only foul shots. Others took more of a jumbled, mixed up approach—they practiced foul shots as well as eight and fifteen footers. The results were remarkable: The jumbled-shot group performed much better, with a deeper sense of the underlying skill.

The same is true in more academic fields, from memory tests to problem solving skills: By mixing up practice, by interweaving different examples, people have a better sense of the underlying relationships. They get a keener sense of the system--and deeper structure of the expertise with outcomes sometimes as much as 40 percent higher. There's some pretty clear take-home lessons for this work, and people should vary their practice--and avoid repetition.

"The ultimate crime is practicing the same thing multiple times in a row. Avoid it like the plague," psychologist Nate Kornell told me. "If you practice for a long chunk of time but don't repeat anything," that works far better.

"'Basic facts' didn’t necessarily need to be in place before deeper analysis could occur; the combination of both was best."

Mental effort plays a big impact here, and in recently published research, researcher Pooja Agarwal looked at how self-quizzing and analogical thinking interact. Specifically, she examined how studying surface, or more “trivia” like information interacts with a deeper understanding of a topic. For example, when and why did welfare programs begin in the United States (surface) versus how would someone who is anti-welfare act if they became unemployed (deep).

“Sometimes we think we have to learn surface features before we can learn deep features and this is exactly what I wanted to look at.” She learned that a combination of surface and deep retrieval practice facilitated the best learning and retention; basically that “basic facts” didn’t necessarily need to be in place before deeper analysis could occur; the combination of both was best.

Mixing Up Practice in Distance Learning

We can do this practice ourselves when learning online. For example, let’s take someone who wants to learn more about American history, and they’re supposed to read two articles about the Revolutionary War articles, two articles about the Civil War articles, and two articles about the Cold War.

The research makes clear that the person would have much deep insights by mixing up the articles: So first a Revolutionary War article, then a Civil War article, and then a Cold War article, and then repeating the process. Why? Because mixing the articles helps people identify links across the different topic areas.  

Granted, people will do this on their own. In skiing, for instance, someone might try to gain experience in different environments—twisty mountains, mogul-filled hills—along with different snow conditions, from powder to icy. In woodworking, people will try different tools and practice on various sorts of wood, oak, pine, fir.

But people don’t typically vary their practice—or examples—nearly enough. To spot deep connections, we need a lot of examples. In Goldstone's experiment, for instance, people only really learned the deeper structure after doing half-dozen problems.

More importantly, we need to mix up those instances in a very direct way. The contrasts between the examples needs to be immediate—and explicit. In the skiing example, for instance, it's not enough to ski a powdery slope one year and an icy hill the next year. The benefits of a more jumbled form of learning come from having the experience directly after each other. So, then, find an icy hill to shoot down right after skiing the powdery one.

To be sure, don't just mix up the learning blindly. As Pooja Agarwal cautions, mixing it up for mixing it up’s sake is not constructive. Interleaving is most beneficial when we are mixing things up that are similar. As she notes, with a laugh, “we don’t put broccoli into fruit salad.”

"Interleaving is most beneficial when we are mixing things up that are similar."

Speculating as a Way to Learn

What's more, interleaving is far from the only way to uncover deep features. Another approach is speculating. As a tool for understanding, as an aspect of the process of learning, speculating is a pretty old practice. Certainly as old as the Bible, and the Good Book is littered with different types of hypotheticals.

“What if there are fifty righteous people in the city?” Abraham asks before the destruction of the city of Gomorrah. Later in the Old Testament, Moses asks God: “What if people don’t believe me?” Jesus of Nazareth also frequently relied on the rhetorical device. “What if you see the Son of Man ascend to where he was before,” he once asked his disciples.

At least in this regard, the Bible isn’t prophetic. The Quran also relies on all sorts of hypotheticals. So do the Confucian Analects. For ancient writers—and most modern ones—the purpose of speculative queries is to push us to consider deep features.

Consider, for instance, the question: What would happen if you could not talk for the rest of your life? There’s not a simple “yes” or “no” answer to the query, and if you spend anytime thinking about your answer, you’d think about the deep feature, noodling over how you communicate with friends, how you network with colleagues, how you ultimately engage with any person that you meet.

Speculation forces a type of reasoning, a manner of scientific inquiry. Albert Einstein knew this, and he used hypotheticals through his career, using much the same approach to discover the theory of general relativity. In that instance, Einstein asked himself: What if someone was falling from a roof? What if the person falling had a toolbox that fell next to them? Later, Einstein called this bit of conjecture “his happiest thought” in large part because it unleashed a wave of new understanding.

There are more recent examples, too. Apple co-founder Steve Jobs understood the value of the approach, and he would ask speculative questions when he wanted to fully grapple with an idea. When Jobs returned to Apple as CEO in the late 1990s, for instance, he wanted to get a better handle on the company. So he pulled in managers, peppering them with queries like: “If money were no object, what would you do?” and: “If you had to cut half of your products, how would do it?”

"If you're working on a tough problem, ask yourself 'what if' questions."

We can do this ourselves. If you're working on a tough problem, ask yourself "what if" questions. What if we had more time? What if we had more people? What if we had more resources? The answers are often provocative--and shed light on the deep feature  

To be sure, deep features are hard. The underlying notions within a field can be slippery. It’s hard to find see the deeper structure. Psychologist Brian Ross recommends, for instance, that people be explicit about the deep structure that they’ve identified. In his research, Ross has found that people solve problems a lot more easily if they write the name of the concept--or deep structure--next to the issue.  

So if someone comes across a question like this:

A skateboarder enters a curved ramp moving pretty quickly, flying along at about a speed of 6.5 miles per second. Then the skateboarder leaves the ramp in a jump, slowing down slightly to 4 miles per second. The skateboarder, and the skateboard have a combined mass of 55 kilograms. What’s the height of the ramp?

The person should be clear about the principle and write down the concept alongside of it. In this case, something along the lines of: the total mechanical energy is the same in the first and final state.

Similarly, I should write down sampling with replacement if I come across another problem featuring a monarch with seven daughters and five lands like I did with Goldstone.