Donate today!
view counter

Researchers see video games as testing, learning tools

By Benjamin Herold for Education Week on Aug 13, 2013 11:15 AM

This is a reprint of an article that originally appeared at Education Week.

Madison, Wis.

Forget No. 2 pencils, or even the new computer-based Common Core exams that have schools across the country scrambling.

Researchers at the University of Wisconsin-Madison are convinced that the tests of the future will look like Crystals of Kaydor, a role-playing video game about aliens.

Designed to measure children’s learning in real time while rewiring their brains to help them be more empathetic, Crystals offers a potentially transformative response to two cutting-edge questions now being debated in the world of testing: whether digital games can effectively blur the line between instruction and assessment and how educators can better gauge children’s social and emotional skills.

“Our job is to provide compelling examples of what assessments can be,” said Constance Steinkuehler, an associate professor of education and former White House policy analyst who co-directs Games+Learning+Society, a center based here that is dedicated to designing and studying video games.

Along with the university’s Center for Investigating Healthy Minds, led by the renowned neuroscientist Richard Davidson, Ms. Steinkuehler and her team hope to demonstrate that successfully playing a video game can itself constitute clear evidence of learning, eliminating the need for after-the-fact assessments.

They also hope to show that video games can strengthen the circuits in children’s brains that regulate empathy, self-control, and the other “noncognitive skills” that researchers increasingly view as the foundation of lifelong academic, financial, physical, and emotional well-being.

Serena Lee, 14, plays a video game developed by University of Wisconsin-Madison researchers. Video games may be the next frontier of student assessment. (Photo: Narayan Mahon/Education Week)

In Crystals, players assume the identity of a damaged robot stranded on a distant planet. To succeed, they must recognize and respond to the nonverbal cues of humanlike aliens, enlisting the creatures’ help through altruistic and “pro-social” behaviors.

Funded by a $1.4 million grant from the Bill & Melinda Gates Foundation, Crystals and a second iPad game, called Tenacity, also developed by Games+Learning+Society, are at the fore of three broad trends reshaping K-12 education: the explosive growth of digital media; the controversial rise of “big data”; and the emergence of new brain research suggesting that critical noncognitive skills are malleable well into adolescence.

(The Gates Foundation also helps support Education Week’s coverage of business and K-12 innovation.)

Assessment experts caution that video games must clear numerous hurdles before they can be considered legitimate testing tools and that efforts to measure social-emotional learning are still in their infancy.

But the notion that games like Crystals and other radically new forms of assessment could soon be used as tests in schools is not fanciful science fiction.

“I would love to see … more reliable, meaningful, and easy-to-administer assessments that help us understand whether we are teaching the noncognitive skills that predict students’ success in college, careers, and life,” U.S. Secretary of Education Arne Duncan said at a conference of the American Educational Research Association in May. “This is the next frontier in assessment.”

‘Cool’ learning

Crystals of Kaydor opens with a serene scene of a space shuttle carrying robotic explorers on their way to a newly discovered planet.

Within seconds, the action kicks into high gear, with sweeping visual effects and pounding electronic music accompanying the robots’ crash landing.

Fourteen-year old Maria Thurow was hooked.

A rising freshman in the 900-student New Glarus school district, outside Madison, Ms. Thurow tested Crystals during one of Games+Learning+Society’s recent “play squads.”

By the time she encountered her first alien, Ms. Thurow, who said she plays digital games ranging from mindless phone apps to complex strategy games like Civilization for up to 10 hours each week, had declared the game “cool.”

Proponents have argued for a decade that video games are powerful learning tools because of their popularity and capacity to engage players in complex problem-solving.

Researchers at the University of Wisconsin-Madison use state-of-the-art MRI scanners to examine whether new video games can change the structure and function of the brain. (Photo: Narayan Mahon/Education Week)

More recently, James Paul Gee, the godfather of video game theory and a co-founder of Games+Learning+Society, has advanced the idea that success in playing video games can offer proof of student mastery of academic content.

Mr. Gee, now a professor at Arizona State University in Tempe, used as an example the popular first-person shooter video game Halo, arguing that it would be nonsensical to give children who successfully complete the game separate, written tests of their Halo knowledge.

“The game is the test,” said Mr. Gee. “If we could design teaching algebra as well as Halo is designed, we’d say the same thing.”

Games+Learning+Society has devoted more resources and talent than most to overcoming two key challenges to Mr. Gee’s vision: creating video games that children actually want to play and capitalizing on the avalanche of information the games generate.

The center’s design studio includes more than a dozen programmers, developers, and artists. Creative director Brian Pelletier spent 18 years overseeing the artistic development of popular commercial games like X-Men Legends.

While not a multimillion-dollar blockbuster, Crystals of Kaydor took Mr. Pelletier’s team eight months and $300,000 to create.

The same attention is being paid to the game’s back end, where researchers and programmers are refining a new system for collecting and analyzing “clickstream data.”

Completing Crystals requires about 3,000 “events,” such as taps on the screen, each of which represents a decision made by the player. The game automatically logs records of every event, along with roughly 15 pieces of related information that it has been programmed to collect.

Soon, the center will have a huge database to mine for evidence that particular patterns of play — how a child solved a particular problem or how long he or she spent trying — are tied to learning outcomes.

Instead of taking isolated, de-contextualized snapshots of student learning, said Mr. Gee, “we can now use digital [games] to assess people in multiple contexts, measure their growth across time, and track different trajectories to mastery. It’s an incredibly threatening moment for more traditional forms of assessment.”

Zander Esh, 13, center, and Isaac Ballwahn, 14, evaluate Crystals of Kaydor, a role-playing video game that challenges students to learn empathy by identifying nonverbal cues. (Photo: Narayan Mahon/Education Week

While playing Crystals, Ms. Thurow initially struggled to identify the emotions displayed on the faces of the aliens she encountered.

Afterward, she said the game was reminiscent of the “culture shock” she experienced when her family moved from Wyoming to the tight-knit Wisconsin community of New Glarus shortly before she started 8th grade.

“Everyone was looking at me, and I had to gauge if they really wanted to get to know me or they were just being nosy,” she explained.

Building blocks

Mr. Davidson, the neuroscientist, said developing the ability to read others’ nonverbal cues is key to navigating many social situations. And the ability to focus your mind — the skill at the heart of Tenacity — is even more important, he argued.

Writer Paul Tough describes multiple strands of research backing those views in his influential book How Children Succeed: Grit, Resilience, and the Hidden Power of Character, including findings by Duke University researchers that individuals who exhibited poor self-control as young children went on to make significantly less money than their peers, were far more likely to suffer from poor physical health and substance abuse, and were far more likely to have been convicted of a crime, regardless of intelligence or social class.

Equally important, said Mr. Davidson, are neuroscientific studies showing that the parts of the brain that regulate important noncognitive skills can be altered through training and experience — and possibly through video games.

“We believe the way to strengthen the circuits of attention and the circuits of empathy is through practice,” he said. “And the way you can make practice fun is by embedding it in a game.”

Adam Wiens, a senior artist at Games+ Learning+Society, works on animating a character for the Crystals of Kaydor video game at the University of Wisconsin in Madison. (Photo: Narayan Mahon/Education Week

To test that theory, Mr. Davidson’s team is using an MRI scanner to peer inside the brains of children who play Crystals and Tenacity. Mr. Davidson said positive findings would signify a “very hopeful” message.

“We’re not accustomed in this culture to thinking about qualities like attention or empathy as skills,” he said. “But neuroscience is teaching us that they’re no different than learning to play a violin.”

For games like Crystals, though, the road from cool prototype to widely used tool will likely be long and rocky.

States and school districts are still struggling with the cost, technical challenges, and politics of implementing the computer-based exams linked to the Common Core State Standards.

And Ms. Steinkuehler acknowledged that the notion of testing students’ empathy or self-control could prove controversial.

“You have to figure out ways to work on [those] skills without it becoming this Orwellian task where Big Brother is constantly watching to make sure that you display the right values,” she said.

Reality check

Before game-based assessment of noncognitive skills can move into the mainstream, said Gregory Cizek, a professor of educational measurement and evaluation at the University of North Carolina at Chapel Hill, there will be debate about whether “it’s appropriate to measure certain things in kids, who should be doing the measuring, and how those skills should count in determining students’ success.”

There will also be psychometric hurdles to clear, Mr. Cizek said.

“A kid might look at these stringy alien things and demonstrate empathy as defined in a gaming situation,” he said, “but does that translate to treating human beings differently?”

Still, it’s not every day that you see 14-year-olds enthusiastically taking tests and bragging about their sensitivity to others’ emotions, as Ms. Thurow and her peers did during the recent Games+Learning+Society play squad.

“Games are a terrifically powerful vehicle for altering the brain in very specific ways,” Mr. Davidson said. “If we can intervene with children and actually strengthen circuits that are beneficial for life outcomes, I think we have a moral obligation as a society to try.”

Click Here
view counter

Comments (8)

Submitted by Christina Puntel (not verified) on August 13, 2013 1:36 pm
For me it is soooo not about using video games as assessments, and much much more about giving students tools to create them. I am sorry the emphasis of this article is on assessment and not on creation, or even co-creation. Shifting the focus onto students as makers means a shift away from the medical model, deficit based education system we find ourselves in the middle of... (I'm more empathetic than you! Ha ha!) Further, thinking about games the way Jane McGonigal asks us to in Reality is Broken and in her TED talk, urges us to look at the ways games work better than society at encouraging collaboration and creative problem solving. We can use games in our classrooms to make real change happen.
Submitted by Anonymous (not verified) on August 13, 2013 2:49 pm
OK, Notebook- I understand that you want to give props to Ben, but please consider a switch to your lineup and put the '$50 million is not enough' ahead of Ben's article. Corbett just announced that he is NOT going to give SDP the grant money until the PFT agrees to some significant concessions. I think that many of your readers, with schools poised to open in a few weeks, are probably more interested in articles that deal with local and state funding of the District. Thanks.
Submitted by Christina Puntel (not verified) on August 13, 2013 8:43 pm
The fact that Corbett and Mark Gleason/PSP's agenda have so easily robbed teachers of talking about practice means they have already succeeded in keeping the haves and have nots in our city very separate, and very unequal. At this time of the year, I am usually steeped in thinking about how games, gaming, digital learning, social justice teaching, and myriad other practices can inform my curriculum planning. This year, the budget, the fight to open schools in some kind of orderly fashion, all of this has robbed public school teachers in Philadelphia of the vital time we need to plan, to dream, to imagine. F$&@ that. I'm working as hard as I can on the inside so I have the mental space to plan and think through my year, but anonymous is right, funding talk comes first. Even though I didnt' love the approach on the video game article, I was happy to have the chance to talk about practice. Musicians make music during wars, poets write poems during storms, and teachers plan curriculum while being scapegoated and targeted, bullied, and lied to. While the video game article didn't situate itself in the current political environment, it certainly could have. I was with teachers this summer from Texas, Arizona, California, and North Carolina. It is not good anywhere for us. Situating curriculum decisions around gaming might have included some discussion around budget cuts, scarcity of technology, closing schools, absence of computers, pushing for bring your own device policies, etc etc etc And one more thing, totally unrelated but its been on my mind.... Ben Herold's tweets about new teacher prof dev at SLA are another fabulous example of divide and conquer, privileging the few, and throwing salt in the wound. Walk in as many shoes as you can.
Submitted by Annonym. (not verified) on August 13, 2013 10:19 pm
Great post! Yes, why do teachers at SLA (and I assume the Workshop and Hill-Freeman) have time to plan together? Because the Phila. School Dictatorship gave them millions and they took it. Meanwhile, most of us are wondering if we'll be able to work with classes of more than 33 students. The Workshop will have 60 9th graders with at least 6 - 7 teachers - now, that is a nice ratio. Inequitable - yes. Immoral - yes. Unjust - yes. But, hey, they are sponsored by the "Partnership!" So much for "progressive" education in Philly!
Submitted by Geoffrey (not verified) on August 13, 2013 11:02 pm
As Ben would say: It's all inside baseball. Except when you get a chance to hit. I can't wait to get back to my four year old laptops. For 90 kids. Let's play games.
Submitted by Benjamin Herold on August 14, 2013 1:55 am


Thanks so much for the thoughtful commentary! Very nice to be able to directly engage Notebook readers again...I miss y'all.

To your points, I thought you might be interested in some additional material from Constance Steinkuehler, the co-director of Games+Learning+Society at Wisconsin.

Ms. Steinkuehler described the types of games - and assessments - her center is developing as a "Trojan horse for progressive pedagogy":

" can do very well on a standardized tests, and it has nothing to do with whether they can actually use physics to solve any problem in their real life.  Performance on the [existing] assessments does not map well to actual activities in your everyday life…I may test highly on mathematics, but that doesn’t predict whether I can leverage mathematics in the everyday world in ways that make my life more beautiful, more efficient, more profound ...The worksheet afterwards isn’t a very strong predictor of life and happiness outside of that worksheet.  And we ought to create assessments that are.  Otherwise, our assessments become gatekeeping, and not diagnostic information that improves my life as the learner....

Right now...we as a nation are acting though what’s measurable is important, rather than figuring out ways to measure what matters. Parents know this.  Teachers know this.  Policymakers know this, principals know this.  Improving assessment is one of the most important things we can do...Part of what we’re trying to do is figure out ways that assessment can actually better serve learners, and be diagnostic and formative, and tell you not just about what you just did, but what you need to do next...

The kind of formative diagnostic assessment [via video games] that we’re talking about, we already know how to do: It’s a teacher and a student sitting down and figuring out where is that child now, what can that child do with supports...But the problem is [that doesn't] scale.  That, and we don’t trust teachers, which is a whole 'nother world of bull---- I don’t want to get into. 

[Video games for assessment] is a...Trojan horse [for] what we know is good pedagogy.  Its based on all the same principles of cognitive science that weve known for 30 years.  The problem is that heretofore, doing all of that rich constructivist pedagogy does not marry well to our assessment systems.  So our task is can we create these compelling [virtual] environments...that can allow teachers and students together to do the activities that we know matter most."  

[She had a lot to say.]

You can also read more about this story here and here.  

Thanks again for engaging, and good luck and godspeed this year.


P.S. - I'll be spending the entire upcoming school year covering the effort to replicate SLA for Education Week.  You raise some of the many questions I'll be exploring.

Submitted by Ms.Cheng (not verified) on August 14, 2013 9:24 am
Hi Ben, many of us are looking forward to the thoughtful evaluation of SLA. I have some opinions, but will wait for your article to see if they are already voiced. The topic of computer gaming as a significant tool to teach empathy is very interesting too. I caught a bit of the WHYY Nightly News that highlighted some of what is being done. Can we speed up real world experiences that we rely on to teach our children lessons of moral consequences? It would appear that in simulated virtual reality, we can. What is groundbreaking is also the correlation between what has so far not been measured, and that is such things as spiritual/moral, sometimes expressed as maturity levels to academic understanding. Just in the exploratory phases, but very intriguing - thank you for your article! (Right now, I just want to pry my teenager away from computer games, because the level at which he is involved seems to me more gratifying than educational:)
Submitted by Jamal (not verified) on June 6, 2014 7:22 am
Well, games are some of the best tools trough you can learn. I've learned English from there.

Post new comment

The content of this field is kept private and will not be shown publicly.

By using this service you agree not to post material that is obscene, harassing, defamatory, or otherwise objectionable. We reserve the right to delete or remove any material deemed to be in violation of this rule, and to ban anyone who violates this rule. Please see our "Terms of Usage" for more detail concerning your obligations as a user of this service. Reader comments are limited to 500 words. You are fully responsible for the content that you post.

Follow Us On

Read the latest print issue

Philly Ed Feed

Recent Comments


Public School Notebook

699 Ranstead St.
Third Floor
Philadelphia, PA 19106
Phone: (215) 839-0082
Fax: (215) 238-2300

© Copyright 2013 The Philadelphia Public School Notebook. All Rights Reserved.
Terms of Usage and Privacy Policy