Meaningful Play 2016 at Michigan State University

Session Information

TitleMeaningful Play Learning Postmortems
Presenter(s)Lindsay Grace, Tanner Jackson and Mark Chen, and Tobi Saulnier
TimeFriday, October 21, 1:00p-2:00p
LocationSuperior Room
DescriptionThis panel will contain the following microtalks:

1) Engaging Game Based Assessments: Design, Development and Evaluation

This lecture provides a post-mortem of 3 of our projects in game based assessment and the lessons learned in collaborating between a university based game studio and the private non-profit Educational Testing Services (ETS). The presentation outlines the two trajectory approach to meeting both organizations' goals, meeting both short term needs for funding cycles and long-term research objectives of academic institutions.

Beyond providing a post-mortem on the design process, effective project management strategies and communication approaches, we discuss how we adapted a client studio production model toward a collaborative, mutually beneficial set of research objectives that yielded published game research in less than 15 months. The collaboration is now beginning its 3rd year and the results are promising.

With nearly 200 study participants preliminary results on the quality of assessment produced from the games is positive. Most interestingly, we are beginning to provide evidence to the claim that engagement in game based assessments is best supported by effective game mechanics, not necessarily game aesthetics. As part of these studies, we compared two games. The first, a game called Robot Sorter, is merely a True-False sort with the aesthetics and other accoutrement of a game. The second, Text Messenger, doesn’t look like a game at all, but plays like one. Using a within subjects design we demonstrate from both pilot study (n=22) and preliminary study (n=100) that player of Text Messenger were more engaged, more likely to continue playing and more attentive than players of Robot Sorter.

The latest game in this research takes the findings from these first games and applies it to the assessment of language use in socio-pragmatics (aka socio-linguistics).

This presentation should be useful to organizational leaders seeking to understand how to effectively shepherd game projects through a combination of institutional investment, collaborative exploration and triad funding scenarios. It is also useful in understanding the challenges of disparate collaboration between teams, iterative pursuit of research goals within funded projects, and adapting research needs to evolving institutional goals.

The presentation will include a variety of concrete and creative heuristics as well as the three game case studies . The projects managed under this relationship represent a significant portion of the American University Game Studio’s ~200K annual portfolio of projects. The studio has been in operation for 3 years, funded through a wide range of clients and partners including The World Bank, Education Testing Services, Deloitte Consulting and the National institutes of Mental Health

2) In-Game Progress Not Necessarily Indicator of Engagement nor Learning

This study argues that progress in a college-readiness game for middle-school students is not necessarily an indicator of engagement nor of learning. Researchers visited an after-school site and video recorded 10 students from local middle schools to document and analyze gameplay, facial expression, posture, and chat utterances. Two extreme cases (one student finished the game while the other struggled) are contrasted to show that engagement can take on many forms and that how far students progressed in the game was not a good measure for this engagement. This study helps researchers see that planning assessments for learning games requires careful thought about what is being measured.

3) The Fiscal Ship: Designing a Civic Game in Partisan Washington - Process and Outcomes

Developed by 1st Playable Productions, the Fiscal Ship game ( is a joint venture of the Hutchins Center and the Serious Games Initiative at the Woodrow Wilson Center. Says Hutchins Center director David Wessel: “We wanted to make a game that was more fun and accessible than the souped-up spreadsheets that are widely available on the web. We wanted to emphasize that decisions about taxes and spending are not only about the deficit and the debt, but also about values, the ones embodied in our governing goals. We wanted to give players the option of increasing spending in one area and cutting it another or paying for it in higher taxes. And we wanted to show that stabilizing the debt is difficult, but possible.” Most importantly the goal was to provide a nonpartisan experience in which players could discover whether beliefs they learned from candidates and social networks were supported by the real policies and data. We hoped that they also would discover that they had fewer differences with the “other side” when it came down to choices that would solve the budget. A worthy goal, however there were substantial design challenges in creating the game. First, the topic, the federal budget, is filled with numbers, projections, and dense descriptions of budget policies that are not accessible to the typical Candy Crush player. Second, the game would not be effective if it appeared to represent a partisan point of view; while at the same time players needed to care about the outcome. Finally, in order to be approachable to a busy voter, the game had to be playable in under thirty minutes. This talk will provide an entertaining and detailed case study as to the design and creative constraints imposed by these challenges, ranging from use of color, narrative and humor, and game objectives. What would be simple design choices, such as the selection of game objectives, had to be finessed such that voters from any point of view could have an experience that was tailored to their belief structure, enabling them to explore the data with an open mind. Color choices for the HUD and iconography in the game had to steer clear of political associations, as well as not appear to casting judgment as to the “safe” levels of debt. The resulting game successfully allows players of all persuasions to gain new insights from real world policies and real world numbers derived from Congressional Budget Office projections. In this era of political gridlock and discourse dominated by sound bites and rhetoric, games like The Fiscal Ship has the potential to be a real factor in improving our society by teaching students and the public at large.

Return to Schedule