Paper Information
| Title | Comparing the Impact of 'Emphasis Frames' on Player Motivation and Performance in a Crowdsourcing Game |
| Presenter(s) | Geoff Kaufman, Mary Flanagan and Sukdith Punjasthitkul |
| Session | Development |
| Time | Friday, October 17, 10:30a-11:30a |
| Location | MSU Room |
| Format | Paper Presentation |
| Description | As crowdsourcing approaches have been increasingly employed as a powerful research tool in virtually every field (Yuen et al., 2011), mounting evidence has demonstrated that masses of voluntary users can be efficiently and effectively mobilized via online platforms to generate vast amounts of new knowledge, particularly for domains in which investment in the same enterprise would not be considered feasible or cost-effective (e.g., Mavandadi, 2012; Rankin et al., 2008; Thorne, Black, & Sykes, 2009; von Ahn, 2006). Moreover, some of the most effective crowdsourcing projects, such as Foldit (Khatib, Kamps, & Milic-Frayling, 2011), Citizen Sort (Prestopnik & Crowston, 2012) and Zooniverse (Borne & Team, 2011; Lintott et al., 2008), have reported great success with utilizing online games that target specific problems (see also Anderson & Rainie, 2012; Cooper et al., 2010). Given their growing ubiquity in everyday life, digital games offer a logical, effective means to foster a widespread and sustained interest in crowdsourcing endeavors (Newton, 2012; Squire & Dikkers, 2012). Indeed, as noted by Ridge (2011):
A well-designed crowdsourcing game can be more fun and more productive than other crowdsourcing interfaces. Not only does good game design entice more people to make their first contribution, but games are also designed to motivate on-going participation. Just as games have been called 'happiness engines', crowdsourcing games could be called 'participation engines.'To date, however, relatively little empirical work has systematically verified such claims or, moreover, identified the key contextual or psychological factors that might augment the inherent motivational draw of crowdsourcing games to encourage greater rates of participation and higher quality input from users. The present research investigated the impact of one such factor - the cognitive "frame" used to describe a game to potential players - as an initial step toward a clearer understanding of how emphasizing particular participatory motivations might impact player engagement and performance. Specifically, two randomized experimental studies tested the impact of several key motivation-related 'emphasis frames' on players of NexTag, a single-player, open-ended media tagging game that is part of Metadata Games, a free and open-source suite of crowdsourcing games that aim to augment digital records by collecting descriptive metadata on image, audio, and film/video artifacts through gameplay (Flanagan & Carini, 2012; Flanagan et al., 2012). The game presents users with a series of four images from the digitized collections of participating libraries, museums, and archives, and invites them to input as many single-word or short-phrase descriptions of the images' content as they wish to provide. Study 1 (N = 97) compared the total number of individual tags contributed by players who were shown, prior to game play, one of three game "frames" emphasizing distinct motivators identified by prior research as among the most commonly cited by crowdsourcing participants (e.g., Brabham, 2008; Kaufmann, Schulze, & Velt, 2011; Tedjamulia et al., 2005; ): (1) personal enjoyment ("Have fun! Play an image-tagging game!"), (2) altruism ("Help the library! Play an image-tagging game!"), and (3) adherence to a perceived social norm of participation ("Join a growing community! Play an image-tagging game!"). In this study, the browser homepage of public computer kiosks in the main library of a private New England college was set to load one of the three versions of the game at random with each new launch. The game framing text appeared at the top of an introductory screen that explained that the game was part of a research study being conducted at the college and that patrons' participation was completely voluntary. In addition, while the game instructions directed players to type in single-word descriptions of the four images presented, participants were not given a set or minimum number of tags to enter (and, thus, were free to type in as many - or as few - tags as they wished). Following a six-week data collection period, analyses of the compiled game data revealed that while the total number of participants who chose to play the game did not significantly differ between the three frame conditions, the average number of tags entered by participants did: on average, players offered significantly fewer tags in the social norm frame condition than in the personal enjoyment frame and altruism frame conditions (which did not significantly differ from one another). In other words, emphasizing the participation of an increasing number of fellow players decreased players' own individual level of input. One explanation for this finding, based on prior work in social psychology, is that the presentation of the social norm regarding crowdsourcing participation may have triggered social loafing, the tendency to exert less effort when activities or work is pooled compared to when one is acting alone (Karau & Williams, 1993; Latane, Williams, & Harkins, 1979). That is, because players were likely to assume that many other players may have already tagged the same images they were presented, they might have been less inclined to exude effort due to the perception that their contributions may be redundant with other players'. Study 2 (N = 148) directly tested this hypothesis by comparing the social norm frame used in Study 1 with two new frames that combined the language of the original frame ("Join a growing community! Play an image-tagging game!") with a second sentence stating either (1) that many other players have tagged the same game images ("Be one among hundreds of players to tag this image!") or (2) that only a few other players have tagged the game images ("Be one of the few players among hundreds to tag this image!). Prior work has shown that the former frame increases the likelihood of social loafing, whereas the latter effectively decreases social loafing (Harkins & Petty, 1982). In addition, to provide a baseline for comparison, a no-frame control condition ("Play an image-tagging game!") was added. In this study, a link inviting users to participate in a voluntary game research study was added to the main library homepage at a large Northeastern university: this link randomly redirected to one of the four versions of the game represented by the four frame conditions (i.e., the original social norm frame, social loafing amplification frame, social loafing reduction frame, and no frame conditions). As in Study 1, the framing language appeared on the introductory game screen, and the same four images used in Study 1 were implemented in the game in Study 2. As an additional indicator of player engagement and motivation, participants in this study were also given the option of replaying NexTag upon completion of the first game. Following a four-week data collection period, analyses of the collected data revealed that compared to participants in the no-frame condition, participants in both the original social norm frame condition and the social loafing amplification frame condition offered significantly fewer tags through gameplay, whereas participants in the social loafing reduction frame condition offered significantly more. In addition, participants in the social loafing reduction frame condition were significantly more likely to elect to replay the game compared to participants in the other three frame conditions: specifically, whereas a slight majority of players in the former condition opted to replay the game, the overwhelming majority (over 75%) of players in the three other conditions opted to decline the opportunity to replay. Taken together, the findings from these studies strongly demonstrate the strong impact of emphasizing distinct motivational factors in presenting a game to players and, moreover, illustrate the potentially detrimental impact of highlighting descriptive norms for participation in crowdsourcing games, particularly in instances when user contributions may be redundant with those of fellow players and, thus, facilitate the occurrence of social loafing. At the same time, the present research suggests that one effective means of counteracting social loafing in crowdsourcing contexts is to make salient the value of users' contributions, which can be achieved by reducing the potential redundancy of those contributions, as shown in the present study, or by stressing the value of users' unique perspectives for contributing valuable input, as other work has demonstrated (e.g., Brickner et al., 1986; Harkins & Petty, 1982). Because the most viable promise of crowdsourcing lies with the sustained participation of contributors who freely volunteer their time and knowledge, it is imperative to understand how games could effectively trigger or direct specific user motivations to encourage higher engagement and better performance. |