PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of plosonePLoS OneView this ArticleSubmit to PLoSGet E-mail AlertsContact UsPublic Library of Science (PLoS)
 
PLoS One. 2010; 5(6): e10937.
Published online Jun 16, 2010. doi:  10.1371/journal.pone.0010937
PMCID: PMC2886831
Quantifying the Performance of Individual Players in a Team Activity
Jordi Duch,1,2,3 Joshua S. Waitzman,1 and Luís A. Nunes Amaral1,2,4*
1Department of Chemical and Biological Engineering, Northwestern University, Evanston, Illinois, United States of America
2Northwestern Institute on Complex Systems, Northwestern University, Evanston, Illinois, United States of America
3Department of Computer Science and Mathematics, Universitat Rovira i Virgili, Tarragona, Spain
4Howard Hughes Medical Institute, Northwestern University, Evanston, Illinois, United States of America
Enrico Scalas, Editor
University of East Piedmont, Italy
* E-mail: amaral/at/northwestern.edu
Conceived and designed the experiments: JD LANA. Performed the experiments: JD JSW. Analyzed the data: JD JSW LANA. Wrote the paper: JD LANA.
Received December 8, 2009; Accepted March 24, 2010.
Background
Teamwork is a fundamental aspect of many human activities, from business to art and from sports to science. Recent research suggest that team work is of crucial importance to cutting-edge scientific research, but little is known about how teamwork leads to greater creativity. Indeed, for many team activities, it is not even clear how to assign credit to individual team members. Remarkably, at least in the context of sports, there is usually a broad consensus on who are the top performers and on what qualifies as an outstanding performance.
Methodology/Principal Findings
In order to determine how individual features can be quantified, and as a test bed for other team-based human activities, we analyze the performance of players in the European Cup 2008 soccer tournament. We develop a network approach that provides a powerful quantification of the contributions of individual players and of overall team performance.
Conclusions/Significance
We hypothesize that generalizations of our approach could be useful in other contexts where quantification of the contributions of individual team members is important.
The importance of teams is nowadays widely accepted [1], [2]; we know that the composition of teams determines their odds of success [3], [4]. However, it is unclear how team processes lead to greater performance or how individual roles and strengths are combined for optimal results. Indeed, while the contributions of “superstars” are widely acknowledged [5], [6], their impact on the performance of their teams is far from having being established quantitatively. This raises the question: are the large disparities in compensation truly representative of the value that each individual brings to the team?
The main obstacle to answering this question has been our current inability to closely monitor individual actions of team members working together on different events. Team sports offer an extraordinary opportunity to overcome these challenges because interactions between team members are on display for a large number of events.
Soccer is widely viewed as the most popular sport world-wide. Soccer is also one of the most difficult sports to analyze quantitatively due to the complexity of the play and to the nearly uninterrupted flow of the ball during the match. Indeed, unlike baseball or basketball, for which there is a wealth of statistical performance data detailing how each player contributes to the final result, in soccer it is not trivial to define quantitative measures of an individual's contribution. Moreover, because soccer scores tend to be low, simple statistics such as number of assists, number of shots or number of goals only rarely provide a reliable measure of a player's true impact on the match's outcome. Instead, the real measure of the performance of a player is “hidden” in the plays of a team: a player can have tremendous impact by winning the ball from the other team or by passing to a teammate who then makes an assist.
Similarly to many other team activities, this type of information required to quantify in detail the role of a team member on team performance is not usually gathered and analyzed in a systematic way (for exceptions see [7], [8]). In the case of soccer, while the assignment of the credit is usually purely based on the subjective views of commentators and spectators, there typically exists a strong consensus on the quality of team play or of individual performances.
The Euro Cup tournament is second only to the World Cup in terms of general interest, attracting millions of spectators and widespread media coverage. The 2008 tournament was unusual in the amount of statistical information that was collected and published online (see http://euro2008.uefa.com). This wealth of information enabled us to develop a new approach to quantify the performance of players and teams inspired by methods from social network analysis [9], [10].
To capture the influence of a given player on a match, we construct a directed network of “ball flow” among the players of a team. In this network, nodes represent players and arcs are weighted according to the number of passes successfully completed between two players. We also incorporate shooting information by including two non-player nodes, “shots to goal” and “shots wide”. A player's node is connected to these two nodes by arcs weighted according to the number of shots. We refer to the resulting networks as “flow networks”, and we build networks for the two teams in every match of the tournament.
In order to obtain performance information, we start with the observation that a soccer team moves the ball with the opponent's goal in mind, keeping possession and shooting when the opportunity arises. A player's passing accuracy, which represents the fraction of passes initiated by a player that reach a teammate, and his shooting accuracy, which accounts for the fraction of shots that do not miss the goal, describe the capability of a player to move the ball towards the opponent's goal (Figs. 1A and 1B).
Figure 1
Figure 1
Performance statistics for individual players.
Combining the flow network with the passing and shooting accuracy of the players, we obtain the probability that each path definable on the network finishes with a shot. This procedure suggests a natural measure of performance of a player — the betweenness centrality [11] of the player with regard to the opponent's goal, which we denote as flow centrality. The flow centrality captures the fraction of times that a player intervenes in those paths that result in a shot. We take into account defensive efficiency by letting each player start a number of paths proportional to the number of balls that he recovers during the match. We define the match performance An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e001.jpg of player An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e002.jpg in team A as the normalized value of the logarithm of the player's flow centrality in the match (Figs. 1C and 1D).
Team performance
We surmise that the player performance can be extended to the team level by calculating the average performance of a subset of players
A mathematical equation, expression, or formula.
 Object name is pone.0010937.e003.jpg
(1)
where An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e004.jpg. We further assume that performance differences between teams, which we define as
A mathematical equation, expression, or formula.
 Object name is pone.0010937.e005.jpg
(2)
will provide an indicator of which team “deserved” victory in a match (Fig. 2A). In order to test these hypotheses, we first obtain the distribution of differences in performance conditional on outcome
A mathematical equation, expression, or formula.
 Object name is pone.0010937.e006.jpg
(3)
where An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e007.jpg“Win”, “Loss”, “Not Win”An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e008.jpg. Figure 2 shows the cumulative distributions of An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e009.jpg for these three outcomes (see Fig. 3 for a justification for this choice). It is visually apparent that there is a substantially larger mean An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e010.jpg for the cases where the team with the highest performance wins the match.
Figure 2
Figure 2
Validity of the flow centrality metric.
Figure 3
Figure 3
Sensitivity and specificity of the flow centrality metric.
We define An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e024.jpg as
A mathematical equation, expression, or formula.
 Object name is pone.0010937.e025.jpg
(4)
To test the significance of the values of An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e026.jpg obtained, we use bootstrap hypothesis testing [12]. Specifically, we pool the values of An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e027.jpg from all 30 matches in the tournament. We then draw surrogate random samples with replacement from the pooled data. For instance, for the case in Fig. 2B we draw surrogate “Loss” and “Not Win” samples with 9 and 14 data points, respectively, and then determine the difference in means of the two surrogate samples. We repeat this procedure 50,000 times in order to determine the significance of the observed An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e028.jpg. As shown in Figs. 2B, C, and D, we find that there is no significant difference in mean An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e029.jpg between “Loss” and “Not Win” outcomes, while the values of An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e030.jpg and An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e031.jpg are highly significant (An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e032.jpg).
The fact that An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e033.jpg is significantly different for matches in which the team that wins has a better performance, suggests that the value of An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e034.jpg is correlated with the outcome of a match and thus can be used as an objective measure of performance. We thus use the area under the curve (AUC)—sometimes also called the receiver-operator curve (ROC) or the sensitivity-specificity curve—statistic in order to quantify the sensitivity and specificity of An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e035.jpg. Figure 3A shows the AUC for the outcomes “Win” versus “Not Win.” We obtain an AUC of 0.825, which is far outside the 90% confidence band for random samples [0.319, 0.653]. We find that the best AUC value is found when team performance is defined as the average performance of the top two players in a team, although an average of the top 1 to 4 players would also lead to significant discrimination (Fig. 3B).
The AUC analysis enables us to conclude that when An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e036.jpg, the odds that the team with higher performance wins the match are 3[ratio]1 (Fig. 3C). Our team performance metric supports the general consensus that Spain, the winner of Euro 2008, played extremely well during the entire tournament (Table 1 and Fig. 4).
Table 1
Table 1
Best team performances.
Figure 4
Figure 4
Time evolution of the performance of players and teams.
Individual performance
We next rank the performance of all the players of the tournament, and identify players who had influential contributions in a specific match or during the entire tournament. This comparison enables us to answer in an objective manner whether, for example, the most famous players fulfilled the expectations placed on them. We find that our metric provides sensible results that are in agreement with the subjective views of analysts and spectators (Table 2), demonstrating that our quantitative measure of performance captures the consensus opinions.
Table 2
Table 2
Best individual performances.
Eight of the twenty players in our list of best performing players (Table 2) were also selected for the twenty-player team of the tournament. Note that we are excluding goal keepers from this analysis. Since the probability of a player being selected for the tournament team is 1/16 as there were 16 teams in the tournament, the probability of observing a given number of players from the tournament team in our top twenty is given by a binomial with 20 attempts and probability of 1/16. The probability of 4 or more players appearing in both lists by chance is approximately An external file that holds a picture, illustration, etc.
Object name is pone.0010937.e037.jpg. For all practical purposes, the probability of eight players appearing in both lists is zero.
Performance visualization
The success of our performance metric in capturing the quality of play prompts us to develop a graphic representation of the play in a soccer match [13], [14]. We combine the network structure and the information compiled in the different distributions to display several features of a match that summarize the play during the 90 minutes (Fig. 5).
Figure 5
Figure 5
Visualization of the three knockout-phase matches of the Spanish team.
These representations enable us to compare the performance of the two teams in a given match and to identify the players with the most important roles during the match. Moreover, as the individual players' positions remain constant across networks, the different match networks can be easily compared to extract the general features of the play of a team, such as the efficiency of a particular team strategy.
Extensions of our approach
Even though we developed and validated this approach for the case of soccer, we believe that it can be generalized to any team sport (or activity) where the final outcome is the result of a complex pattern of interactions among participants. In particular, the flow centrality metric we introduce may provide a new approach to quantify the contribution of individuals to teams working in other contexts. By combining information about skills, knowledge, and capabilities of the individuals, with information about the strength of the interactions between them —for example, using the number and length of phone calls or the number of e-mails exchanged— and information about completion of specific tasks, one could, potentially, quantitatively assess the individual performance of the team members and their contribution to the team's output.
In order to illustrate how our methodology could be extended to other activities that involve team work, we studied the interactions occurring in the process of completing several scientific projects that resulted in publications involving members of our lab. Specifically, we used email records to reconstruct the exchanges between the co-authors of the papers considered.
We then broke down these exchanges into path on the network of co-authors that terminate with (1) the completion of a task required for the paper, such as performing a calculation, obtaining some data, or writing some portion of the manuscript, (2) the scheduling of a meeting, or (3) the discarding of the task. This procedure enables us to build flow networks for each of the projects considered (Fig. 6). In these networks, a node represents a co-author in the manuscript, and the arcs represent the weighted communication directed from one co-authors to the other.
Figure 6
Figure 6
Visualization interactions between co-authors for three manuscript published by our lab.
Additionally, we assign values to each of the completed task and scheduled meetings and award the corresponding value to each of the co-authors involved in the path. In this way, we are able to determine the flow centrality of each co-author in the project. Our analysis clearly reveals the different inputs and partitioning of responsibilities among co-authors for the different projects.
Our work demonstrates the power of social network analysis methods in providing insight into complex social phenomena. Indeed, whereas there are contexts in which simple measures or statistics may provide a very complete picture of an individual's performance —think of golf, baseball, or a track event— for most situations of interest, objectively quantifying individual performances or individual contributions to team performance is far from trivial.
At least in the context of a soccer, where quantification has always been challenging, we are able to demonstrate that flow centrality provides a powerful objective quantification of individual and team performance. While we cannot demonstrate the power of a similar approach in the context of a scientific collaboration, our preliminary results suggest that flow centrality does provide some insight into the variability in the partitioning of responsibilities among co-authors in a project.
Acknowledgments
We thank E. Altmann, R. Guimerà, D. Malmgren, P. McMullen, A. Salazar, M. Sales-Pardo, E. Sawardecker, S. Seaver, I. Sirer, and M. Stringer for useful comments and suggestions.
Footnotes
Competing Interests: The authors have declared that no competing interests exist.
Funding: This work has been funded through National Science Foundation (NSF) (http://www.nsf.gov/) awards SBE-0830388 and IIS-0838564. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
1. Katzenback JR, Smith DK. The Wisdom of Teams. NY: Harper Bussiness; 1993.
2. Whitfield J. Collaboration: Group theory. Nature. 2008;455:720–723. [PubMed]
3. Guimerà R, Uzzi B, Spiro J, Amaral L. Team assembly mechanisms determine collaboration network structure and team performance. Science. 2005;308:697–702. [PMC free article] [PubMed]
4. Wuchty S, Jones B, Uzzi B. The increasing dominance of teams in production of knowledge. Science. 2007;316:1036–1039. [PubMed]
5. Rosen S. The economics of superstars. American Economic Review. 1981;71:845–858.
6. Lucifora C, Simmons R. Superstar effects in sport: evidence from italian soccer. Journal of Sports Economics. 2003;4:35–55.
7. Brillinger DR. A potential function approach to the flow of play in soccer. Journal of Quantitative Analysis in Sports. 2007;3:3.
8. Hughes M, Franks I. Analysis of passing sequences, shots and goals in soccer. Journal of Sports Science. 2005;23:504–514. [PubMed]
9. Wasserman S, Faust K. Social Network Analysis. Cambridge, UK: Cambridge University Press; 1994.
10. Scott J. Social Network Analysis: A Handbook. London, UK: SAGE Publications Ltd., 2 edition; 2000.
11. Freeman LC. A set of measures of centrality based upon betweenness. Sociometry. 1977;40:35–41.
12. Fawcett T. An introduction to ROC analysis. Pattern Recognition Letters. 2006;27:861–874.
13. Tufte E. The Visual Display of Quantitative Information. Graphics Press; 1983.
14. Tufte E. Visual Explanations: Images and Quantities, Evidence and Narrative. Graphics Press; 1997.
Articles from PLoS ONE are provided here courtesy of
Public Library of Science