Introduction
Patch Size (LoC)
Teams
Locations
Active Reviewers

Sheet 1: Introduction

 This page contains complementary material the paper Investigating the Effectiveness of Peer Code Review in Distributed Software Development, authored by Eduardo Witter dos Santos and Ingrid Nunes. Each section of this page contains data about the influence of a given factor over four code review outcomes, namely: Duration, Participation, Comment Density and Comment Density by Reviewer, totalizing 20 tables. Detailed results for Kruskal-Wallis tests are presented along with descriptive statistics (mean, standard deviation, median, minimum and maximum values) for each influencing factor. Interpretation of Kruskal-Wallis tests deserves a more detailed explanation. In each table, the first two lines contain the outcome and the overall influence of a given factor, verified using a Kruskal-Wallis analysis. This indicates whether there is a statistically significant difference among the groups. Dunn’s test was used to check which pairs of groups actually have a statistically significant difference. The group names are presented in the names of columns and rows, and each cell contains the result of Dunn’s test for a given pair of groups: the z-score result and the adjusted p-value. Statistically significant differences are highlighted. Please notice that we always use p < 0.05, but Dunn’s test actually use adjusted p-values according to the number of size of each group, so it is possible to have p < 0.05 without having a statistically significant difference. The example bellow shows how to interpret the results. As part of our research, we plotted our data in scatterplots and calculated Spearman's correlation coefficient, but this analysis did not lead to any interesting conclusions, due to the data distribution. Our data is clustered in a few groups according to our research questions (e.g., the number of locations), so we actually need to compare groups to verify whether they have differences on code review effectiveness (e.g., reviews with one and two locations). Kruskal-Wallis is nonparametric test used to compare groups of different sizes when data does not fit a normal distribution. This statistic test was used to verify if means of each group have a statistically significant difference, so in our paper, we always present the means for each group (instead of medians).

Sheet 2: Patch Size (LoC)

 Outcome: Duration Statistically significant difference: Yes, H=1761.32, p < 0.05 Patch Size (LoC) 0-20 21-40 41-60 61-80 81-100 101-200 201-400 401-600 601-800 801-1000 1001-2000 2001-3000 21-40 -8.97 0.00 41-60 -9.82 -1.90 0.00 0.04 61-80 -10.81 -3.72 -1.87 0.00 0.00 0.04 81-100 -9.47 -3.21 -1.53 0.20 0.00 0.00 0.07 0.43 101-200 -18.75 -7.28 -4.55 -2.06 -2.13 0.00 0.00 0.00 0.03 0.02 201-400 -23.75 -11.59 -8.48 -5.65 -5.45 -4.56 0.00 0.00 0.00 0.00 0.00 0.00 401-600 -20.25 -11.50 -9.00 -6.55 -6.36 -5.64 -1.82 0.00 0.00 0.00 0.00 0.00 0.00 0.04 601-800 -16.12 -10.01 -8.15 -6.22 -6.12 -5.28 -2.22 -0.69 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.27 801-1000 -14.03 -8.81 -7.21 -5.50 -5.46 -4.53 -1.77 -0.40 0.20 0.00 0.00 0.00 0.00 0.00 0.00 0.05 0.37 0.44 1001-2000 -22.02 -14.13 -11.72 -9.31 -9.01 -8.82 -5.30 -3.26 -2.15 -2.21 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.02 2001-3000 -11.85 -8.30 -7.18 -5.94 -5.93 -5.14 -3.11 -2.04 -1.46 -1.56 0.03 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.08 0.07 0.49 > 3000 -13.81 -8.82 -7.28 -5.63 -5.59 -4.69 -2.02 -0.69 -0.07 -0.25 1.86 1.33 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.27 0.48 0.42 0.04 0.10 Outcome: Participation Statistically significant difference: Yes, H=709.58, p < 0.05 Patch Size (LoC) 0-20 21-40 41-60 61-80 81-100 101-200 201-400 401-600 601-800 801-1000 1001-2000 2001-3000 21-40 5.17 0.00 41-60 7.23 2.38 0.00 0.01 61-80 5.66 1.65 -0.46 0.00 0.06 0.34 81-100 8.10 4.17 2.05 2.32 0.00 0.00 0.03 0.01 101-200 12.58 5.54 2.52 2.76 -0.12 0.00 0.00 0.01 0.00 0.45 201-400 13.65 6.64 3.58 3.71 0.80 1.26 0.00 0.00 0.00 0.00 0.23 0.12 401-600 12.11 7.00 4.43 4.53 1.89 2.52 1.45 0.00 0.00 0.00 0.00 0.04 0.01 0.09 601-800 9.52 5.98 3.99 4.14 1.92 2.37 1.51 0.31 0.00 0.00 0.00 0.00 0.04 0.01 0.08 0.39 801-1000 9.02 5.93 4.14 4.29 2.25 2.67 1.89 0.77 0.43 0.00 0.00 0.00 0.00 0.02 0.01 0.04 0.23 0.35 1001-2000 13.82 9.13 6.66 6.63 4.06 5.08 4.07 2.46 1.81 1.21 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.04 0.13 2001-3000 8.86 6.73 5.42 5.53 3.94 4.33 3.75 2.85 2.46 2.03 1.26 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.03 0.12 > 3000 9.75 6.74 4.98 5.10 3.08 3.59 2.84 1.69 1.27 0.80 -0.28 -1.35 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.06 0.12 0.23 0.40 0.11 Outcome: Comment Density Statistically significant difference: Yes, H=3250.94, p < 0.05 Patch Size (LoC) 0-20 21-40 41-60 61-80 81-100 101-200 201-400 401-600 601-800 801-1000 1001-2000 2001-3000 21-40 12.11 0.00 41-60 14.09 3.24 0.00 0.00 61-80 13.76 4.31 1.29 0.00 0.00 0.11 81-100 15.00 6.28 3.35 2.01 0.00 0.00 0.00 0.03 101-200 26.76 10.92 6.43 4.38 1.72 0.00 0.00 0.00 0.00 0.05 201-400 30.73 14.59 9.84 7.49 4.65 4.01 0.00 0.00 0.00 0.00 0.00 0.00 401-600 27.23 15.44 11.48 9.40 6.78 6.63 3.24 0.00 0.00 0.00 0.00 0.00 0.00 0.00 601-800 21.57 13.33 10.35 8.77 6.62 6.23 3.52 0.83 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.21 801-1000 20.12 12.97 10.32 8.91 6.94 6.54 4.09 1.60 0.75 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.06 0.23 1001-2000 27.44 17.06 13.40 11.39 8.87 9.06 5.95 2.68 1.52 0.60 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.07 0.27 2001-3000 15.91 11.13 9.29 8.32 6.86 6.38 4.58 2.72 2.00 1.35 0.99 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.03 0.10 0.17 > 3000 22.63 15.61 12.96 11.49 9.49 9.42 7.03 4.43 3.32 2.45 2.17 0.67 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.01 0.02 0.25 Outcome: Comment Density by Reviewer Statistically significant difference: Yes, H=3579.57, p < 0.05 Patch Size (LoC) 0-20 21-40 41-60 61-80 81-100 101-200 201-400 401-600 601-800 801-1000 1001-2000 2001-3000 21-40 12.37 0.00 41-60 14.08 3.07 0.00 0.00 61-80 14.57 4.85 1.94 0.00 0.00 0.03 81-100 15.10 6.22 3.44 1.53 0.00 0.00 0.00 0.07 101-200 28.17 11.79 7.40 4.53 2.43 0.00 0.00 0.00 0.00 0.01 201-400 32.55 15.81 11.11 7.94 5.62 4.38 0.00 0.00 0.00 0.00 0.00 0.00 401-600 28.55 16.40 12.52 9.74 7.60 6.87 3.18 0.00 0.00 0.00 0.00 0.00 0.00 0.00 601-800 22.98 14.48 11.57 9.40 7.65 6.81 3.85 1.17 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.13 801-1000 21.20 13.85 11.28 9.35 7.76 6.91 4.23 1.78 0.63 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.04 0.27 1001-2000 28.94 18.23 14.64 11.97 9.89 9.59 6.19 2.95 1.42 0.65 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.08 0.27 2001-3000 16.46 11.57 9.80 8.45 7.29 6.44 4.48 2.65 1.71 1.17 0.76 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.05 0.13 0.23 > 3000 23.11 15.94 13.39 11.43 9.82 9.25 6.65 4.11 2.76 2.03 1.66 0.52 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.02 0.05 0.30 Duration Participation Comment Density Comment Density by Reviewer LOC #Rev Mean StdDev Median Min Max Mean StdDev Median Min Max Mean StdDev Median Min Max Mean StdDev Median Min Max 0 – 20 3836 1.80 6.60 0.00 0.00 173.00 86.10 22.30 100.00 12.50 100.00 99.30 139.60 50.00 0.00 173.00 47.30 60.60 25.00 0.00 900.00 21 – 40 716 3.90 13.00 1.00 0.00 187.00 81.70 22.70 100.00 16.70 100.00 15.30 15.40 11.10 0.00 187.00 5.70 4.10 4.80 0.00 36.60 41 – 60 475 4.40 10.20 1.00 0.00 111.00 78.40 24.00 80.00 14.30 100.00 9.40 8.40 7.10 0.00 111.00 3.50 2.20 3.20 0.00 14.40 61 – 80 342 5.00 10.30 1.00 0.00 89.00 79.50 22.40 81.70 20.00 100.00 7.80 6.80 6.20 0.00 89.00 2.70 1.60 2.50 0.00 11.20 81-100 273 5.80 16.00 1.00 0.00 211.00 74.90 24.00 75.00 20.00 100.00 5.40 4.30 4.20 0.00 211.00 2.00 1.20 1.90 0.00 6.70 101 – 200 762 6.00 12.90 2.00 0.00 193.00 75.20 23.80 75.00 16.70 100.00 4.30 3.70 3.40 0.00 193.00 1.50 0.90 1.30 0.00 5.80 201 – 400 688 8.00 16.80 4.00 0.00 268.00 73.70 23.70 75.00 14.30 100.00 2.60 2.50 1.90 0.00 268.00 0.90 0.70 0.70 0.00 10.00 401 – 600 371 9.40 16.00 5.00 0.00 140.00 71.90 22.30 66.70 9.10 100.00 1.50 1.50 1.10 0.00 140.00 0.50 0.30 0.40 0.00 2.00 601 – 800 203 10.40 16.70 6.00 0.00 139.00 71.20 23.10 71.40 20.00 100.00 1.30 1.50 0.90 0.00 139.00 0.40 0.30 0.30 0.00 2.00 801 – 1000 158 10.00 15.70 6.00 0.00 118.00 69.40 24.60 66.70 10.00 100.00 1.00 0.90 0.70 0.00 118.00 0.30 0.20 0.30 0.00 1.00 1001 – 2000 282 13.40 19.10 8.00 0.00 167.00 66.90 23.20 66.70 16.70 100.00 0.80 0.80 0.50 0.00 167.00 0.20 0.20 0.20 0.00 1.10 2001 – 3000 78 15.10 18.90 9.50 0.00 85.00 62.00 24.70 59.20 16.70 100.00 0.40 0.30 0.30 0.00 85.00 0.10 0.10 0.10 0.00 0.40 >3000 145 12.70 17.00 6.00 0.00 72.00 67.10 24.50 66.70 16.70 100.00 0.20 0.20 0.10 0.00 72.00 0.10 0.10 0.00 0.00 0.50 TOTAL 8329 4.70 12.10 1.00 0.00 268.00 80.10 23.80 100.00 9.10 100.00 48.80 105.80 7.70 0.00 1400.00 22.90 46.90 3.20 0.00 900.00

Sheet 3: Teams

 Outcome: Duration Statistically significant difference: Yes, H=586.72, p < 0.05 Teams 1 2 3 4 5 6 2 -15.55 0.00 3 -17.62 -7.22 0.00 0.00 4 -11.81 -7.03 -3.25 0.00 0.00 0.00 5 -6.69 -4.47 -2.72 -1.02 0.00 0.00 0.01 0.23 6 -1.65 -1.11 -0.69 -0.28 0.00 0.09 0.22 0.32 0.43 0.50 7 -1.98 -1.43 -1.01 -0.60 -0.32 -0.23 0.05 0.13 0.22 0.34 0.44 0.43 Outcome: Participation Statistically significant difference: Yes, H=226.72, p < 0.05 Teams 1 2 3 4 5 6 2 11.36 0.00 3 10.28 2.85 0.00 0.01 4 6.37 2.92 1.41 0.00 0.01 0.18 5 3.04 1.43 0.74 0.04 0.01 0.20 0.30 0.49 6 1.35 0.95 0.79 0.61 0.58 0.19 0.30 0.32 0.32 0.31 7 1.51 1.11 0.94 0.76 0.74 0.11 0.20 0.25 0.28 0.31 0.29 0.48 Outcome: Comment Density Statistically significant difference: Yes, H=184.71, p < 0.05 Teams 1 2 3 4 5 6 2 -10.88 0.00 3 -8.88 -1.82 0.00 0.12 4 -5.52 -2.21 -1.23 0.00 0.06 0.33 5 -2.50 -0.95 -0.51 0.10 0.03 0.45 0.40 0.46 6 0.28 0.66 0.76 0.91 0.87 0.46 0.38 0.39 0.42 0.41 7 0.13 0.51 0.62 0.77 0.73 -0.10 0.49 0.38 0.38 0.42 0.38 0.48 Outcome: Comment Density by Reviewer Statistically significant difference: Yes, H=91.94, p < 0.05 Teams 1 2 3 4 5 6 2 -9.29 0.00 3 -3.66 2.12 0.00 0.12 4 -1.45 1.32 0.25 0.38 0.33 0.44 5 -0.07 1.24 0.73 0.56 0.47 0.25 0.29 0.34 6 0.77 1.09 0.97 0.93 0.76 0.31 0.29 0.27 0.26 0.29 7 1.09 1.41 1.29 1.25 1.07 0.22 0.26 0.33 0.30 0.28 0.25 0.43 Duration Participation Comment Density Comment Density by Reviewer Teams #Rev Mean StdDev Median Min Max Mean StdDev Median Min Max Mean StdDev Median Min Max Mean StdDev Median Min Max 1 4934 3.00 8.30 0.00 0.00 173.00 82.70 24.10 100.00 9.10 100.00 42.20 92.00 5.90 0.00 173.00 22.10 45.10 2.60 0.00 600.00 2 2440 5.80 13.50 2.00 0.00 211.00 76.80 24.00 78.90 12.50 100.00 55.10 112.10 10.00 0.00 211.00 25.50 51.10 4.20 0.00 900.00 3 766 9.40 19.30 3.00 0.00 268.00 75.50 20.10 75.00 25.00 100.00 66.40 140.50 11.50 0.00 268.00 22.10 46.30 3.10 0.00 350.00 4 152 12.40 19.00 5.00 0.00 132.00 73.50 17.70 72.70 30.80 100.00 77.30 177.80 13.20 0.00 132.00 17.80 39.70 3.20 0.00 260.00 5 33 24.10 39.10 6.00 0.00 156.00 73.50 15.80 71.40 50.00 100.00 50.50 79.80 17.90 0.00 156.00 8.90 14.00 3.10 0.00 60.00 6 2 21.50 20.50 21.50 1.00 42.00 63.00 8.40 63.00 54.50 71.40 8.10 7.10 8.10 0.90 42.00 1.60 1.40 1.60 0.20 3.00 7 2 19.00 16.00 19.00 3.00 35.00 58.50 21.50 58.50 37.00 80.00 5.20 0.90 5.20 4.30 35.00 0.60 0.00 0.60 0.50 0.60 TOTAL 8329 4.70 12.10 1.00 0.00 268.00 80.10 23.80 100.00 9.10 100.00 48.80 105.80 7.70 0.00 1400.00 22.90 46.90 3.20 0.00 900.00

Sheet 4: Locations

 Outcome: Duration Statistically significant difference: Yes, H=158.00, p < 0.05 Locations 1 2 2 -11.60 0.00 3 -5.12 -2.87 0.00 0.00 Outcome: Participation Statistically significant difference: Yes, H=86.69, p < 0.05 Locations 1 2 2 9.13 0.00 3 2.06 0.32 0.03 0.38 Outcome: Comment Density Statistically significant difference: Yes, H=134.05, p < 0.05 Locations 1 2 2 -11.12 0.00 3 -3.51 -1.37 0.00 0.08 Outcome: Comment Density by Reviewer Statistically significant difference: Yes, H=56.12, p < 0.05 Locations 1 2 2 -7.23 0.00 3 -2.12 -0.74 0.03 0.23 Duration Participation Comment Density Comment Density by Reviewer Locations #Rev Mean StdDev Median Min Max Mean StdDev Median Min Max Mean StdDev Median Min Max Mean StdDev Median Min Max 1 7219 4.10 10.80 0.00 0.00 211.00 80.80 24.10 100.00 9.10 100.00 45.50 98.40 6.70 0.00 211.00 22.50 46.80 2.90 0.00 900.00 2 1076 8.00 17.40 2.00 0.00 268.00 75.20 21.60 75.00 20.00 100.00 68.70 140.20 14.20 0.00 268.00 25.10 47.70 4.80 0.00 400.00 3 34 22.20 31.70 5.50 0.00 126.00 75.90 15.30 75.00 42.90 100.00 132.80 206.60 54.30 0.00 126.00 35.40 52.10 12.10 0.00 180.00 TOTAL 8329 4.70 12.10 1.00 0.00 268.00 80.10 23.80 100.00 9.10 100.00 48.80 105.80 7.70 0.00 1400.00 22.90 46.90 3.20 0.00 900.00

Sheet 5: Active Reviewers

 Outcome: Duration Statistically significant difference: Yes, H=1652.94, p < 0.05 Active Reviewers 1 2 3 4 5 6 7 8 9 10 11 2 -15.61 0.00 3 -28.21 -13.70 0.00 0.00 4 -27.00 -15.95 -5.37 0.00 0.00 0.00 5 -23.56 -15.60 -8.00 -3.71 0.00 0.00 0.00 0.00 6 -17.53 -12.27 -7.30 -4.48 -1.73 0.00 0.00 0.00 0.00 0.08 7 -12.54 -9.14 -5.95 -4.18 -2.35 -1.04 0.00 0.00 0.00 0.00 0.02 0.23 8 -7.46 -5.43 -3.54 -2.51 -1.44 -0.67 0.02 0.00 0.00 0.00 0.02 0.13 0.31 0.49 9 -6.26 -4.79 -3.42 -2.68 -1.90 -1.32 -0.77 -0.69 0.00 0.00 0.00 0.01 0.06 0.16 0.30 0.32 10 -2.31 -1.87 -1.46 -1.23 -1.00 -0.83 -0.67 -0.66 -0.40 0.02 0.06 0.13 0.18 0.23 0.28 0.32 0.31 0.38 11 -2.12 -1.67 -1.26 -1.04 -0.81 -0.64 -0.48 -0.47 -0.22 0.14 0.04 0.09 0.17 0.22 0.29 0.31 0.36 0.36 0.44 0.47 12 -2.20 -1.76 -1.34 -1.12 -0.89 -0.72 -0.56 -0.55 -0.30 0.08 -0.06 0.03 0.08 0.16 0.21 0.27 0.31 0.34 0.34 0.42 0.48 0.48 Outcome: Participation Statistically significant difference: Yes, H=268.49, p < 0.05 Active Reviewers 1 2 3 4 5 6 7 8 9 10 11 2 10.93 0.00 3 11.81 1.59 0.00 0.23 4 11.40 3.63 2.34 0.00 0.00 0.06 5 9.82 4.24 3.31 1.46 0.00 0.00 0.00 0.25 6 5.46 1.77 1.19 0.05 -0.89 0.00 0.19 0.28 0.53 0.39 7 4.07 1.69 1.31 0.56 -0.11 0.46 0.00 0.22 0.26 0.54 0.57 0.54 8 2.65 1.23 1.00 0.56 0.15 0.51 0.19 0.03 0.28 0.34 0.53 0.58 0.53 0.62 9 1.16 -0.13 -0.03 0.35 -0.64 -0.35 -0.55 -0.62 0.29 0.57 0.52 0.56 0.52 0.57 0.52 0.52 10 1.91 1.60 1.55 1.45 1.36 1.45 1.37 1.30 1.49 0.15 0.24 0.23 0.24 0.25 0.23 0.26 0.26 0.25 11 0.39 0.07 -0.03 0.07 -0.16 -0.08 -0.14 -0.19 0.03 -1.08 0.58 0.54 0.51 0.54 0.60 0.55 0.57 0.61 0.53 0.31 12 0.38 0.07 -0.02 0.08 -0.17 -0.08 -0.15 -0.20 0.03 -1.08 0.00 0.57 0.53 0.50 0.56 0.61 0.57 0.59 0.63 0.51 0.32 0.50 Outcome: Comment Density Statistically significant difference: Yes, H=660.89, p < 0.05 Active Reviewers 1 2 3 4 5 6 7 8 9 10 11 2 -21.71 0.00 3 -19.31 1.01 0.00 0.37 4 -15.87 -0.42 -1.15 0.00 0.50 0.32 5 -11.47 -0.36 -0.90 -0.05 0.00 0.47 0.39 0.49 6 -7.98 -0.65 -1.00 -0.43 -0.36 0.00 0.46 0.36 0.54 0.47 7 -5.68 -0.95 -1.18 -0.80 -0.75 -0.45 0.00 0.38 0.31 0.41 0.43 0.54 8 -2.96 -0.14 -0.28 -0.06 -0.05 0.10 0.37 0.01 0.48 0.44 0.49 0.48 0.49 0.48 9 -3.62 -1.58 -1.68 -1.52 -1.49 -1.35 -1.08 -1.20 0.00 0.34 0.31 0.36 0.34 0.32 0.35 0.33 10 -0.32 0.30 0.27 0.31 0.32 0.35 0.42 0.32 0.74 0.47 0.44 0.44 0.44 0.45 0.46 0.49 0.46 0.42 11 -2.01 -1.39 -1.42 -1.37 -1.37 -1.33 -1.26 -1.33 -0.88 -1.19 0.16 0.36 0.37 0.35 0.33 0.32 0.31 0.30 0.39 0.32 12 -0.21 0.40 0.37 0.42 0.42 0.46 0.52 0.42 0.84 0.08 1.27 0.46 0.48 0.49 0.49 0.52 0.55 0.52 0.53 0.40 0.49 0.32 Outcome: Comment Density by Reviewer Statistically significant difference: Yes, H=275.55, p < 0.05 Active Reviewers 1 2 3 4 5 6 7 8 9 10 11 2 -15.94 0.00 3 -8.58 6.37 0.00 0.00 4 -4.78 6.58 1.69 0.00 0.00 0.23 5 -1.64 6.53 3.00 1.61 0.24 0.00 0.01 0.22 6 -0.67 4.71 2.41 1.52 0.36 0.33 0.00 0.05 0.25 0.38 7 -0.18 3.29 1.81 1.25 0.48 0.21 0.44 0.00 0.19 0.35 0.36 0.44 8 0.45 2.52 1.64 1.31 0.84 0.66 0.48 0.36 0.04 0.22 0.33 0.33 0.33 0.36 9 -1.02 0.48 -0.16 -0.39 -0.71 -0.81 -0.87 -1.09 0.34 0.35 0.44 0.38 0.33 0.33 0.34 0.35 10 0.60 1.05 0.86 0.79 0.69 0.65 0.62 0.49 0.87 0.33 0.35 0.34 0.32 0.34 0.33 0.33 0.37 0.35 11 -1.14 -0.69 -0.88 -0.95 -1.05 -1.08 -1.11 -1.21 -0.80 -1.23 0.35 0.33 0.36 0.33 0.34 0.34 0.35 0.32 0.33 0.34 12 0.76 1.21 1.02 0.95 0.85 0.81 0.78 0.65 1.02 0.11 1.34 0.32 0.34 0.32 0.34 0.34 0.34 0.32 0.32 0.33 0.45 0.33 Duration Participation Comment Density Comment Density by Reviewer Active Reviewers #Rev Mean StdDev Median Min Max Mean StdDev Median Min Max Mean StdDev Median Min Max Mean StdDev Median Min Max 1 2431 1.30 4.50 0.00 0.00 79.00 84.30 27.40 100.00 9.10 100.00 26.60 54.40 1.90 0.00 79.00 26.60 54.40 1.90 0.00 900.00 2 2502 3.20 7.60 0.00 0.00 140.00 78.60 24.80 100.00 14.30 100.00 54.50 100.50 11.30 0.00 140.00 27.20 50.20 5.60 0.00 600.00 3 1940 6.00 11.90 2.00 0.00 211.00 78.70 21.20 75.00 21.40 100.00 57.10 116.30 9.00 0.00 211.00 19.00 38.80 3.00 0.00 400.00 4 840 7.90 12.60 4.00 0.00 103.00 77.60 18.50 80.00 28.60 100.00 68.80 155.80 10.10 0.00 103.00 17.20 39.00 2.50 0.00 350.00 5 372 14.00 26.40 6.00 0.00 268.00 76.50 15.90 71.40 35.70 100.00 61.50 151.10 9.00 0.00 268.00 12.30 30.20 1.80 0.00 260.00 6 149 18.70 33.40 6.00 0.00 193.00 78.90 14.50 75.00 46.20 100.00 59.40 116.90 10.00 0.00 193.00 9.90 19.50 1.70 0.00 108.30 7 60 18.90 29.30 11.00 0.00 156.00 78.70 13.70 77.80 43.80 100.00 44.00 73.60 14.20 0.60 156.00 6.30 10.50 2.00 0.10 42.90 8 21 17.20 19.50 9.00 0.00 71.00 77.90 12.70 80.00 50.00 100.00 31.40 48.70 13.00 0.00 71.00 3.90 6.10 1.60 0.00 27.50 9 11 33.20 30.10 21.00 0.00 96.00 81.80 15.30 81.80 52.90 100.00 192.10 321.80 15.80 3.30 96.00 21.30 35.80 1.80 0.40 122.20 10 1 35.00 0.00 35.00 35.00 35.00 37.00 0.00 37.00 37.00 37.00 6.20 0.00 6.20 6.20 35.00 0.60 0.00 0.60 0.60 0.60 11 1 15.00 0.00 15.00 15.00 15.00 84.60 0.00 84.60 84.60 84.60 262.50 0.00 262.50 262.50 15.00 23.90 0.00 23.90 23.90 23.90 12 1 20.00 0.00 20.00 20.00 20.00 85.70 0.00 85.70 85.70 85.70 5.00 0.00 5.00 5.00 20.00 0.40 0.00 0.40 0.40 0.40 TOTAL 8329 4.70 12.10 1.00 0.00 268.00 80.10 23.80 100.00 9.10 100.00 48.80 105.80 7.70 0.00 1400.00 22.90 46.90 3.20 0.00 900.00