Firms often compensate employees based on their relative performance in the most recent business period. These firms need to consider what type of performance information to share with their employees in order to obtain better outcomes in the long run, without diminishing staff motivation. In this paper, we empirically investigate the impact of sharing irrelevant benchmark “information” (e.g., information about the interim winner’s performance) when individuals are making repeated decisions under uncertainty. The decision-making context used is the newsvendor problem, which is a canonical framework for operations management decision making. The newsvendor problem occurs in many business contexts, such as buying fashion goods for retail, setting safety stock levels, setting target inventory levels for perishable goods, selecting the right capacity for a service facility, and overbooking customers. Theoretically, information about the interim winner’s performance has no value for making improved decisions in future rounds, and it might even be misleading. Surprisingly, we find that displaying such irrelevant benchmark information results in significantly improved decisions overall, as compared to a control group; this additional display may motivate participants to perform better. We also identify two personality traits related to impulsivity which moderate this positive information display effect.
Key words: laboratory experiment; repetitive decision making; irrelevant information; newsvendor problem; motivation; impulsivity
“Unless you are Spock, irrelevant things matter in economic behavior.”
— Richard H. Thaler
In Star Trek, if you ask Mr. Spock a question, he will give you a logical answer based on scientific evidence and probabilities. He is a rational character and makes emotionless decisions without being affected by irrelevant information. But we humans are not as calm as Mr. Spock; a single piece of irrelevant information could impact our behavior. For example, Nobel laureate Richard Thaler (2015) notes that students were happier after he raised their exam scores range from 0 to 100 to 0 to 137. The new scores are irrelevant to their course letter grades, which are based on the students’ relative performance in the class, yet they made the students happier.
The effect of irrelevant information on decision making has been studied in the context of psychology (Gaeth and Shanteau 1984, Kahneman and Tversky 1973, Lu and Proctor 2001, Tversky and Kahneman 1974), education (Rice, 1975), accounting (Thaler and Johnson 1990), decision science (Sengupta 1995), and behavioral economics (Thaler 2016). For example, Kahneman and Tversky (1973) find that people respond differently to a probability question (e.g., whether someone is an engineer or a lawyer) when given no description than when given a description that conveys irrelevant information (e.g., this person is a 30-year-old man). Likewise, Tversky and Kahneman (1974) find that even a random number determined by a spinning wheel affects the estimates of unknown quantities. More recently, Thaler (2016) observes that there are many irrelevant factors that can influence choices, such as “the presence of a prior gain,” “the salience of one option over another,” and so forth. He refers to such factors as “supposedly irrelevant factors” (SIFs). It is clear that irrelevant information matters in decision making, but can irrelevant information also help when managers make repeated decisions under uncertainty?
In many business environments, managers need to make repeated decisions under uncertainty (Shapira 2002). Consider, for example, procurement managers who face stochastic demand for a perishable fashion product. With long lead times, they must determine the order quantity well before observing the actual demand for each period. If they order too much, their profits will suffer because they must sell excess inventories at salvage prices. If they order too little, that leads to backorders and to shortage costs. These managers face a classic newsvendor problem, which is a canonical framework for operations management decision making. The newsvendor problem occurs in many other contexts, such as setting safety stock levels, selecting the right capacity for a service facility, and overbooking customers by hotels or airlines. Given the purchase price, sale price, salvage value, and the demand distribution, managers can easily compute the closed form profit-maximizing order quantity which has been discovered years ago (Schweitzer and Cachon 2000). However, recent studies present evidence that individuals responsible for making newsvendor decisions tend to suffer from various decision biases, such as anchoring, demand chasing, and over-confidence, leading to sub-optimal outcomes (e.g., BenZion et al. 2008, Bolton et al. 2012, Bostian et al. 2008, De Vericourt et al. 2013, Ren and Croson 2013, Rudi and Drake 2014, Schweitzer and Cachon 2000).
To improve performance, firms and organizations usually provide performance feedback to their employees. Researchers have shown that benchmark information, or best practice information, can increase performance if used wisely. For example, Barankay (2012) shows that adding benchmark information, which is the sales data on the current performance required to be in the top 10%, 25% and 50%, to rank feedback improves performance on sales as opposed to using rank feedback alone. When hospital emergency departments share best practices for improving productivity, Song et al. (2017) find that public disclosure of physicians’ relative performance with their identities improves productivity when compared to privately disclosing relative performance anonymously. In those studies, the benchmark information is meaningful and relevant so that individuals can learn how far off they are from the top performers in order to improve their own performance. However, other benchmark information such as the highest profit and the associated order quantity among procurement managers from the last period of the newsvendor game, does not provide any useful guidelines for managers in finding the solution that maximizes the expected long-run profit. Moreover, this information is irrelevant to the realized demand quantity for the next period. However, it is possible to test whether such irrelevant benchmark information will actually motivate individuals to improve their repetitive decision making.
The primary goal of this paper is to investigate whether displaying irrelevant benchmark information about the interim winner’s performance impacts individuals’ repeated business decision-making under uncertainty, when relative performance feedback (i.e., rank feedback) is given privately. To the best of our knowledge, our study is the first one that examines the effect of irrelevant information on repetitive business decision making in a competitive environment.
There are two major mechanisms that may impact individuals’ decision making when irrelevant benchmark information is provided: an information effect and a motivation effect.
First, the benchmark information about the interim winner’s performance may lead to improved decisions when demand distribution is unknown, because the benchmark information may serve as a proxy for the actual demand. In that sense, the information is helpful and relevant. However, once the demand distribution is given, such information is no longer informative, and a rational decision maker should stick to the optimal order quantity to maximize the expected profit. To rule out the confounding effect of this information, we provide individuals with the demand distribution in our study. In addition, keeping the benchmark information irrelevant, we provide equivalent information about demand realization after each period. Moreover, since demand follows a Markov process, the benchmark information has nothing to do with the demand for the next period. Therefore, we expect that the information effect is not significant.1
Second, the benchmark information may have a motivating effect that leads to an improvement in ordering decisions. Leveraging social comparison theory, researchers have shown that displaying benchmark information is likely to make games more motivating and enjoyable through elements such as competition, leaderboards, and symbolic awards. Competition among players promotes stonger incentives for effort, output, and efficiency without affecting the quality of the output (Cowgill 2015). Leaderboards have also been shown to improve motivation, engagement, and enjoyment (Halen et al. 2010). Symbolic awards tend to motivate players to pay more attention to the game (Parr 2015).
Displaying benchmark information can motivate individuals to exert more effort and thereby improve performance, but this occurs only if they take action toward their goals. Individuals are more likely to be heterogeneous in characteristics, such as personality, that can potentially modify the effect of displaying irrelevant benchmark information on repetitive decision making under uncertainty. The motivation effect should be larger for individuals who have high self-control (or low impulsivity) because these individuals are more likely to trade short-term gains (e.g., interim wins) for long-term gains (e.g., cumulative long-run profits) by restraining the impulse to pursue the short-term benefit (Ainslie 1975, Martin and Potts 2009). Hence, another goal of this study is to apply theory from psychology to examine the moderating role of self-control, which can be measured by its converse, impulsivity, using the UPPS-P Impulsive Behavior Scale (Whiteside and Lynam 2001). To our knowledge, this paper is the first study that investigates the association between impulsivity and decision making under uncertainty in a competitive setting that mimics a real business environment.
To address these aims, we use the newsvendor problem as the experimental context representing a broad set of operations management issues involving random fluctuations (Fisher and Raman 1996). Numerous studies have looked at various aspects of this problem (e.g., Dana Jr and Petruzzi 2001, Lai and Xiao 2017, Xu et al. 2010), ranging from theoretical modeling to the behavioral aspects (e.g., Lee and Siemsen 2016, Natarajan et al. 2017). Recently, behavioral research has been interested in investigating how and why newsvendors suffer from the so-called “pull-to-center” bias (e.g., Bolton et al. 2012, Bostian et al. 2008, Ockenfels and Selten 2014, Rudi and Drake 2014). This bias is the tendency of individuals to order non-optimal quantities, which typically reside between the profit-maximizing quantity and the mean2 of the demand distribution. Studies involving undergraduate students, managers, and MBA candidates repeatedly reveal this pull-to-center bias – even after the subjects have been taught the closed-form optimal solution (Bolton et al. 2012, Schweitzer and Cachon 2000). Laboratory experiments looking at newsvendors who make two simultaneous decisions per period also reveal this effect (Chen and Li 2016). However, the existing research has not taken into account the competitive setting when relative performance feedback is provided, or to the role of personality traits in the newsvendor decision making context.
Our results show that publicly displaying irrelevant benchmark information about the interim winner’s performance in addition to privately providing individual rankings significantly improves newsvendor decisions, as compared to a control group in which only individual rankings are privately given to each participant. A pull-to-center effect only exists in the treatment group. That is, we observe that individuals order an amount between the profit-maximizing quantity and the median of demand distribution. Furthermore, we find that two personality traits related to impulsivity as measured by the UPPS-P — (lack of) premeditation3 and urgency4 — moderate the treatment effect on ordering decisions. Individuals who score higher on premeditation (i.e., those who think more deliberately before taking an action) and urgency (i.e., those who make impulsive decisions due to negative moods) are more likely to benefit from such information, probably by being motivated to work harder.
This paper contributes to the literature on irrelevant information, performance feedback, behavioral operations, and impulsivity. First, we provide greater insight into the effect of irrelevant information on decision making. We identify a new type of irrelevant information—irrelevant benchmark information—and examine the effect of such information on repeated decisions. Second, we show that displaying such irrelevant benchmark information in addition to the rank feedback could be surprisingly beneficial. Third, we contribute to the behavioral operations management literature by utilizing newsvendor experiments in a competitive environment with asymmetric demand distribution. Finally, we provide insight into when and why displaying irrelevant benchmark information will improve repetitive decision making under uncertainty by investigating the moderating role of impulsivity. Our results suggest that impulsivity traits play an important role in our understanding of the newsvendor decision-making behavior.
The remainder of this paper is organized as follows: Section 2 develops two research hypotheses with supporting literature. In Section 3, we describe the theory about the newsvendor problem underlying our work and present the experimental protocol and the main results of our experiments. In Section 4, we perform a robustness check. Finally, we summarize our main findings and discuss their managerial implications in Section 5.
2. Related Literature and Hypotheses
2.1 Relative performance feedback
Many high-profile business leaders, such as Jack Welch, Marissa Mayer, and Steve Ballmer, have used relative performance feedback to create competition among workers (Cowgill 2015). For example, Welch, the former CEO of GE, is famous for promoting a culture in which every year the bottom 10% of employees were asked to leave the firm. As a pervasive motivating tool in industry, a rank-order tournament has been shown to make individuals’ mean effort levels converge to their theoretical equilibrium levels (Bull et al. 1987).
Carefully chosen feedback information may improve an individual’s overall performance in a competitive environment where relative performance is measured. Although the theoretical work on the impact of various types of feedback information on individual performance is limited (Ederer 2010), there are numerous papers studying related topics in the field or laboratory. Within the education context, Azmat and Iriberri (2010) find that providing students with relative performance feedback information about the class average enhances their grades. In a field study of furniture salespeople, Barankay (2012) shows that including benchmarks in addition to rank feedback can improve performance significantly over only showing the rank feedback. Berger and Pope (2011) manipulate the competitive feedback information provided to participants in randomized groups such that participants were told they were far behind, slightly behind, tied, or slightly ahead of their opponent, or else given no feedback. They find that being slightly behind increases individuals’ effort. Moreover, Charness et al. (2011) show that individuals work harder when they receive feedback with winning and losing symbols on their relative position.
Indeed, sometimes a small change makes a huge difference. Some researchers compare private and public disclosure of relative performance feedback (e.g., rank feedback). For example, using a laboratory experiment, Tafkov (2012) shows that participants solve more multiplication problems correctly when relative performance information is provided publicly rather than privately. Using electronic health records from two emergency departments, Song et al. (2017) find that publicly disclosing relative performance feedback information with physicians’ identities -rather than privately disclosing such information without identities – increases physicians’ productivity on average without a significant reduction in service quality.
1 Individuals are not told ex-ante the information is irrelevant, so it is possible that some individuals still rely on this information while making their inventory decisions. We perform extra analysis in Section 4 to show that individuals’ beliefs about this information do not matter.
2 According to Schweitzer and Cachon (2000), the “center” refers to median of the demand distribution. Prior studies rely on symmetric demand distributions such as uniform and normal, in which the mean and median have the same value.
3 (Lack of) premeditation refers to the inability to take potential consequences into consideration before acting. More detailed explanations appear below.
4 Urgency refers to the tendency to experience strong impulses driven by negative affect. More detailed explanations appear below.