Overview

Introduction
Patch Size (LoC)
Teams
Locations
Active Reviewers

Sheet 1: Introduction

This page contains complementary material the paper Investigating the Effectiveness of Peer Code Review in Distributed Software Development, authored by Eduardo Witter dos Santos and Ingrid Nunes.

Each section of this page contains data about the influence of a given factor over four code review outcomes, namely: Duration, Participation, Comment Density and Comment Density by Reviewer, totalizing 20 tables. Detailed results for Kruskal-Wallis tests are presented along with descriptive statistics (mean, standard deviation, median, minimum and maximum values) for each influencing factor.

Interpretation of Kruskal-Wallis tests deserves a more detailed explanation. In each table, the first two lines contain the outcome and the overall influence of a given factor, verified using a Kruskal-Wallis analysis. This indicates whether there is a statistically significant difference among the groups. Dunn’s test was used to check which pairs of groups actually have a statistically significant difference.

The group names are presented in the names of columns and rows, and each cell contains the result of Dunn’s test for a given pair of groups: the z-score result and the adjusted p-value. Statistically significant differences are highlighted. Please notice that we always use p < 0.05, but Dunn’s test actually use adjusted p-values according to the number of size of each group, so it is possible to have p < 0.05 without having a statistically significant difference.

The example bellow shows how to interpret the results.

As part of our research, we plotted our data in scatterplots and calculated Spearman's correlation coefficient, but this analysis did not lead to any interesting conclusions, due to the data distribution. Our data is clustered in a few groups according to our research questions (e.g., the number of locations), so we actually need to compare groups to verify whether they have differences on code review effectiveness (e.g., reviews with one and two locations).

Kruskal-Wallis is nonparametric test used to compare groups of different sizes when data does not fit a normal distribution. This statistic test was used to verify if means of each group have a statistically significant difference, so in our paper, we always present the means for each group (instead of medians).

Sheet 2: Patch Size (LoC)

Outcome: Duration
Statistically significant difference: Yes, H=1761.32, p < 0.05
Patch Size (LoC)	0-20	21-40	41-60	61-80	81-100	101-200	201-400	401-600	601-800	801-1000	1001-2000	2001-3000
21-40	-8.97
21-40	0.00
41-60	-9.82	-1.90
41-60	0.00	0.04
61-80	-10.81	-3.72	-1.87
61-80	0.00	0.00	0.04
81-100	-9.47	-3.21	-1.53	0.20
81-100	0.00	0.00	0.07	0.43
101-200	-18.75	-7.28	-4.55	-2.06	-2.13
101-200	0.00	0.00	0.00	0.03	0.02
201-400	-23.75	-11.59	-8.48	-5.65	-5.45	-4.56
201-400	0.00	0.00	0.00	0.00	0.00	0.00
401-600	-20.25	-11.50	-9.00	-6.55	-6.36	-5.64	-1.82
401-600	0.00	0.00	0.00	0.00	0.00	0.00	0.04
601-800	-16.12	-10.01	-8.15	-6.22	-6.12	-5.28	-2.22	-0.69
601-800	0.00	0.00	0.00	0.00	0.00	0.00	0.02	0.27
801-1000	-14.03	-8.81	-7.21	-5.50	-5.46	-4.53	-1.77	-0.40	0.20
801-1000	0.00	0.00	0.00	0.00	0.00	0.00	0.05	0.37	0.44
1001-2000	-22.02	-14.13	-11.72	-9.31	-9.01	-8.82	-5.30	-3.26	-2.15	-2.21
1001-2000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.02	0.02
2001-3000	-11.85	-8.30	-7.18	-5.94	-5.93	-5.14	-3.11	-2.04	-1.46	-1.56	0.03
2001-3000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.03	0.08	0.07	0.49
> 3000	-13.81	-8.82	-7.28	-5.63	-5.59	-4.69	-2.02	-0.69	-0.07	-0.25	1.86	1.33
> 3000	0.00	0.00	0.00	0.00	0.00	0.00	0.03	0.27	0.48	0.42	0.04	0.10


Outcome: Participation
Statistically significant difference: Yes, H=709.58, p < 0.05
Patch Size (LoC)	0-20	21-40	41-60	61-80	81-100	101-200	201-400	401-600	601-800	801-1000	1001-2000	2001-3000
21-40	5.17
21-40	0.00
41-60	7.23	2.38
41-60	0.00	0.01
61-80	5.66	1.65	-0.46
61-80	0.00	0.06	0.34
81-100	8.10	4.17	2.05	2.32
81-100	0.00	0.00	0.03	0.01
101-200	12.58	5.54	2.52	2.76	-0.12
101-200	0.00	0.00	0.01	0.00	0.45
201-400	13.65	6.64	3.58	3.71	0.80	1.26
201-400	0.00	0.00	0.00	0.00	0.23	0.12
401-600	12.11	7.00	4.43	4.53	1.89	2.52	1.45
401-600	0.00	0.00	0.00	0.00	0.04	0.01	0.09
601-800	9.52	5.98	3.99	4.14	1.92	2.37	1.51	0.31
601-800	0.00	0.00	0.00	0.00	0.04	0.01	0.08	0.39
801-1000	9.02	5.93	4.14	4.29	2.25	2.67	1.89	0.77	0.43
801-1000	0.00	0.00	0.00	0.00	0.02	0.01	0.04	0.23	0.35
1001-2000	13.82	9.13	6.66	6.63	4.06	5.08	4.07	2.46	1.81	1.21
1001-2000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.04	0.13
2001-3000	8.86	6.73	5.42	5.53	3.94	4.33	3.75	2.85	2.46	2.03	1.26
2001-3000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.03	0.12
> 3000	9.75	6.74	4.98	5.10	3.08	3.59	2.84	1.69	1.27	0.80	-0.28	-1.35
> 3000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.06	0.12	0.23	0.40	0.11


Outcome: Comment Density
Statistically significant difference: Yes, H=3250.94, p < 0.05
Patch Size (LoC)	0-20	21-40	41-60	61-80	81-100	101-200	201-400	401-600	601-800	801-1000	1001-2000	2001-3000
21-40	12.11
21-40	0.00
41-60	14.09	3.24
41-60	0.00	0.00
61-80	13.76	4.31	1.29
61-80	0.00	0.00	0.11
81-100	15.00	6.28	3.35	2.01
81-100	0.00	0.00	0.00	0.03
101-200	26.76	10.92	6.43	4.38	1.72
101-200	0.00	0.00	0.00	0.00	0.05
201-400	30.73	14.59	9.84	7.49	4.65	4.01
201-400	0.00	0.00	0.00	0.00	0.00	0.00
401-600	27.23	15.44	11.48	9.40	6.78	6.63	3.24
401-600	0.00	0.00	0.00	0.00	0.00	0.00	0.00
601-800	21.57	13.33	10.35	8.77	6.62	6.23	3.52	0.83
601-800	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.21
801-1000	20.12	12.97	10.32	8.91	6.94	6.54	4.09	1.60	0.75
801-1000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.06	0.23
1001-2000	27.44	17.06	13.40	11.39	8.87	9.06	5.95	2.68	1.52	0.60
1001-2000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.07	0.27
2001-3000	15.91	11.13	9.29	8.32	6.86	6.38	4.58	2.72	2.00	1.35	0.99
2001-3000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.03	0.10	0.17
> 3000	22.63	15.61	12.96	11.49	9.49	9.42	7.03	4.43	3.32	2.45	2.17	0.67
> 3000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.01	0.02	0.25


Outcome: Comment Density by Reviewer
Statistically significant difference: Yes, H=3579.57, p < 0.05
Patch Size (LoC)	0-20	21-40	41-60	61-80	81-100	101-200	201-400	401-600	601-800	801-1000	1001-2000	2001-3000
21-40	12.37
21-40	0.00
41-60	14.08	3.07
41-60	0.00	0.00
61-80	14.57	4.85	1.94
61-80	0.00	0.00	0.03
81-100	15.10	6.22	3.44	1.53
81-100	0.00	0.00	0.00	0.07
101-200	28.17	11.79	7.40	4.53	2.43
101-200	0.00	0.00	0.00	0.00	0.01
201-400	32.55	15.81	11.11	7.94	5.62	4.38
201-400	0.00	0.00	0.00	0.00	0.00	0.00
401-600	28.55	16.40	12.52	9.74	7.60	6.87	3.18
401-600	0.00	0.00	0.00	0.00	0.00	0.00	0.00
601-800	22.98	14.48	11.57	9.40	7.65	6.81	3.85	1.17
601-800	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.13
801-1000	21.20	13.85	11.28	9.35	7.76	6.91	4.23	1.78	0.63
801-1000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.04	0.27
1001-2000	28.94	18.23	14.64	11.97	9.89	9.59	6.19	2.95	1.42	0.65
1001-2000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.08	0.27
2001-3000	16.46	11.57	9.80	8.45	7.29	6.44	4.48	2.65	1.71	1.17	0.76
2001-3000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.05	0.13	0.23
> 3000	23.11	15.94	13.39	11.43	9.82	9.25	6.65	4.11	2.76	2.03	1.66	0.52
> 3000	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.02	0.05	0.30


		Duration					Participation					Comment Density					Comment Density by Reviewer
LOC	#Rev	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max
0 – 20	3836	1.80	6.60	0.00	0.00	173.00	86.10	22.30	100.00	12.50	100.00	99.30	139.60	50.00	0.00	173.00	47.30	60.60	25.00	0.00	900.00
21 – 40	716	3.90	13.00	1.00	0.00	187.00	81.70	22.70	100.00	16.70	100.00	15.30	15.40	11.10	0.00	187.00	5.70	4.10	4.80	0.00	36.60
41 – 60	475	4.40	10.20	1.00	0.00	111.00	78.40	24.00	80.00	14.30	100.00	9.40	8.40	7.10	0.00	111.00	3.50	2.20	3.20	0.00	14.40
61 – 80	342	5.00	10.30	1.00	0.00	89.00	79.50	22.40	81.70	20.00	100.00	7.80	6.80	6.20	0.00	89.00	2.70	1.60	2.50	0.00	11.20
81-100	273	5.80	16.00	1.00	0.00	211.00	74.90	24.00	75.00	20.00	100.00	5.40	4.30	4.20	0.00	211.00	2.00	1.20	1.90	0.00	6.70
101 – 200	762	6.00	12.90	2.00	0.00	193.00	75.20	23.80	75.00	16.70	100.00	4.30	3.70	3.40	0.00	193.00	1.50	0.90	1.30	0.00	5.80
201 – 400	688	8.00	16.80	4.00	0.00	268.00	73.70	23.70	75.00	14.30	100.00	2.60	2.50	1.90	0.00	268.00	0.90	0.70	0.70	0.00	10.00
401 – 600	371	9.40	16.00	5.00	0.00	140.00	71.90	22.30	66.70	9.10	100.00	1.50	1.50	1.10	0.00	140.00	0.50	0.30	0.40	0.00	2.00
601 – 800	203	10.40	16.70	6.00	0.00	139.00	71.20	23.10	71.40	20.00	100.00	1.30	1.50	0.90	0.00	139.00	0.40	0.30	0.30	0.00	2.00
801 – 1000	158	10.00	15.70	6.00	0.00	118.00	69.40	24.60	66.70	10.00	100.00	1.00	0.90	0.70	0.00	118.00	0.30	0.20	0.30	0.00	1.00
1001 – 2000	282	13.40	19.10	8.00	0.00	167.00	66.90	23.20	66.70	16.70	100.00	0.80	0.80	0.50	0.00	167.00	0.20	0.20	0.20	0.00	1.10
2001 – 3000	78	15.10	18.90	9.50	0.00	85.00	62.00	24.70	59.20	16.70	100.00	0.40	0.30	0.30	0.00	85.00	0.10	0.10	0.10	0.00	0.40
>3000	145	12.70	17.00	6.00	0.00	72.00	67.10	24.50	66.70	16.70	100.00	0.20	0.20	0.10	0.00	72.00	0.10	0.10	0.00	0.00	0.50
TOTAL	8329	4.70	12.10	1.00	0.00	268.00	80.10	23.80	100.00	9.10	100.00	48.80	105.80	7.70	0.00	1400.00	22.90	46.90	3.20	0.00	900.00

Sheet 3: Teams

Outcome: Duration
Statistically significant difference: Yes, H=586.72, p < 0.05
Teams	1	2	3	4	5	6
2	-15.55
2	0.00
3	-17.62	-7.22
3	0.00	0.00
4	-11.81	-7.03	-3.25
4	0.00	0.00	0.00
5	-6.69	-4.47	-2.72	-1.02
5	0.00	0.00	0.01	0.23
6	-1.65	-1.11	-0.69	-0.28	0.00
6	0.09	0.22	0.32	0.43	0.50
7	-1.98	-1.43	-1.01	-0.60	-0.32	-0.23
7	0.05	0.13	0.22	0.34	0.44	0.43


Outcome: Participation
Statistically significant difference: Yes, H=226.72, p < 0.05
Teams	1	2	3	4	5	6
2	11.36
2	0.00
3	10.28	2.85
3	0.00	0.01
4	6.37	2.92	1.41
4	0.00	0.01	0.18
5	3.04	1.43	0.74	0.04
5	0.01	0.20	0.30	0.49
6	1.35	0.95	0.79	0.61	0.58
6	0.19	0.30	0.32	0.32	0.31
7	1.51	1.11	0.94	0.76	0.74	0.11
7	0.20	0.25	0.28	0.31	0.29	0.48


Outcome: Comment Density
Statistically significant difference: Yes, H=184.71, p < 0.05
Teams	1	2	3	4	5	6
2	-10.88
2	0.00
3	-8.88	-1.82
3	0.00	0.12
4	-5.52	-2.21	-1.23
4	0.00	0.06	0.33
5	-2.50	-0.95	-0.51	0.10
5	0.03	0.45	0.40	0.46
6	0.28	0.66	0.76	0.91	0.87
6	0.46	0.38	0.39	0.42	0.41
7	0.13	0.51	0.62	0.77	0.73	-0.10
7	0.49	0.38	0.38	0.42	0.38	0.48


Outcome: Comment Density by Reviewer
Statistically significant difference: Yes, H=91.94, p < 0.05
Teams	1	2	3	4	5	6
2	-9.29
2	0.00
3	-3.66	2.12
3	0.00	0.12
4	-1.45	1.32	0.25
4	0.38	0.33	0.44
5	-0.07	1.24	0.73	0.56
5	0.47	0.25	0.29	0.34
6	0.77	1.09	0.97	0.93	0.76
6	0.31	0.29	0.27	0.26	0.29
7	1.09	1.41	1.29	1.25	1.07	0.22
7	0.26	0.33	0.30	0.28	0.25	0.43




		Duration					Participation					Comment Density					Comment Density by Reviewer
Teams	#Rev	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max
1	4934	3.00	8.30	0.00	0.00	173.00	82.70	24.10	100.00	9.10	100.00	42.20	92.00	5.90	0.00	173.00	22.10	45.10	2.60	0.00	600.00
2	2440	5.80	13.50	2.00	0.00	211.00	76.80	24.00	78.90	12.50	100.00	55.10	112.10	10.00	0.00	211.00	25.50	51.10	4.20	0.00	900.00
3	766	9.40	19.30	3.00	0.00	268.00	75.50	20.10	75.00	25.00	100.00	66.40	140.50	11.50	0.00	268.00	22.10	46.30	3.10	0.00	350.00
4	152	12.40	19.00	5.00	0.00	132.00	73.50	17.70	72.70	30.80	100.00	77.30	177.80	13.20	0.00	132.00	17.80	39.70	3.20	0.00	260.00
5	33	24.10	39.10	6.00	0.00	156.00	73.50	15.80	71.40	50.00	100.00	50.50	79.80	17.90	0.00	156.00	8.90	14.00	3.10	0.00	60.00
6	2	21.50	20.50	21.50	1.00	42.00	63.00	8.40	63.00	54.50	71.40	8.10	7.10	8.10	0.90	42.00	1.60	1.40	1.60	0.20	3.00
7	2	19.00	16.00	19.00	3.00	35.00	58.50	21.50	58.50	37.00	80.00	5.20	0.90	5.20	4.30	35.00	0.60	0.00	0.60	0.50	0.60
TOTAL	8329	4.70	12.10	1.00	0.00	268.00	80.10	23.80	100.00	9.10	100.00	48.80	105.80	7.70	0.00	1400.00	22.90	46.90	3.20	0.00	900.00

Sheet 4: Locations

Outcome: Duration
Statistically significant difference: Yes, H=158.00, p < 0.05
Locations	1	2
2	-11.60
2	0.00
3	-5.12	-2.87
3	0.00	0.00


Outcome: Participation
Statistically significant difference: Yes, H=86.69, p < 0.05
Locations	1	2
2	9.13
2	0.00
3	2.06	0.32
3	0.03	0.38


Outcome: Comment Density
Statistically significant difference: Yes, H=134.05, p < 0.05
Locations	1	2
2	-11.12
2	0.00
3	-3.51	-1.37
3	0.00	0.08


Outcome: Comment Density by Reviewer
Statistically significant difference: Yes, H=56.12, p < 0.05
Locations	1	2
2	-7.23
2	0.00
3	-2.12	-0.74
3	0.03	0.23



		Duration					Participation					Comment Density					Comment Density by Reviewer
Locations	#Rev	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max
1	7219	4.10	10.80	0.00	0.00	211.00	80.80	24.10	100.00	9.10	100.00	45.50	98.40	6.70	0.00	211.00	22.50	46.80	2.90	0.00	900.00
2	1076	8.00	17.40	2.00	0.00	268.00	75.20	21.60	75.00	20.00	100.00	68.70	140.20	14.20	0.00	268.00	25.10	47.70	4.80	0.00	400.00
3	34	22.20	31.70	5.50	0.00	126.00	75.90	15.30	75.00	42.90	100.00	132.80	206.60	54.30	0.00	126.00	35.40	52.10	12.10	0.00	180.00
TOTAL	8329	4.70	12.10	1.00	0.00	268.00	80.10	23.80	100.00	9.10	100.00	48.80	105.80	7.70	0.00	1400.00	22.90	46.90	3.20	0.00	900.00

Sheet 5: Active Reviewers

Outcome: Duration
Statistically significant difference: Yes, H=1652.94, p < 0.05
Active Reviewers	1	2	3	4	5	6	7	8	9	10	11
2	-15.61
2	0.00
3	-28.21	-13.70
3	0.00	0.00
4	-27.00	-15.95	-5.37
4	0.00	0.00	0.00
5	-23.56	-15.60	-8.00	-3.71
5	0.00	0.00	0.00	0.00
6	-17.53	-12.27	-7.30	-4.48	-1.73
6	0.00	0.00	0.00	0.00	0.08
7	-12.54	-9.14	-5.95	-4.18	-2.35	-1.04
7	0.00	0.00	0.00	0.00	0.02	0.23
8	-7.46	-5.43	-3.54	-2.51	-1.44	-0.67	0.02
8	0.00	0.00	0.00	0.02	0.13	0.31	0.49
9	-6.26	-4.79	-3.42	-2.68	-1.90	-1.32	-0.77	-0.69
9	0.00	0.00	0.00	0.01	0.06	0.16	0.30	0.32
10	-2.31	-1.87	-1.46	-1.23	-1.00	-0.83	-0.67	-0.66	-0.40
10	0.02	0.06	0.13	0.18	0.23	0.28	0.32	0.31	0.38
11	-2.12	-1.67	-1.26	-1.04	-0.81	-0.64	-0.48	-0.47	-0.22	0.14
11	0.04	0.09	0.17	0.22	0.29	0.31	0.36	0.36	0.44	0.47
12	-2.20	-1.76	-1.34	-1.12	-0.89	-0.72	-0.56	-0.55	-0.30	0.08	-0.06
12	0.03	0.08	0.16	0.21	0.27	0.31	0.34	0.34	0.42	0.48	0.48


Outcome: Participation
Statistically significant difference: Yes, H=268.49, p < 0.05
Active Reviewers	1	2	3	4	5	6	7	8	9	10	11
2	10.93
2	0.00
3	11.81	1.59
3	0.00	0.23
4	11.40	3.63	2.34
4	0.00	0.00	0.06
5	9.82	4.24	3.31	1.46
5	0.00	0.00	0.00	0.25
6	5.46	1.77	1.19	0.05	-0.89
6	0.00	0.19	0.28	0.53	0.39
7	4.07	1.69	1.31	0.56	-0.11	0.46
7	0.00	0.22	0.26	0.54	0.57	0.54
8	2.65	1.23	1.00	0.56	0.15	0.51	0.19
8	0.03	0.28	0.34	0.53	0.58	0.53	0.62
9	1.16	-0.13	-0.03	0.35	-0.64	-0.35	-0.55	-0.62
9	0.29	0.57	0.52	0.56	0.52	0.57	0.52	0.52
10	1.91	1.60	1.55	1.45	1.36	1.45	1.37	1.30	1.49
10	0.15	0.24	0.23	0.24	0.25	0.23	0.26	0.26	0.25
11	0.39	0.07	-0.03	0.07	-0.16	-0.08	-0.14	-0.19	0.03	-1.08
11	0.58	0.54	0.51	0.54	0.60	0.55	0.57	0.61	0.53	0.31
12	0.38	0.07	-0.02	0.08	-0.17	-0.08	-0.15	-0.20	0.03	-1.08	0.00
12	0.57	0.53	0.50	0.56	0.61	0.57	0.59	0.63	0.51	0.32	0.50


Outcome: Comment Density
Statistically significant difference: Yes, H=660.89, p < 0.05
Active Reviewers	1	2	3	4	5	6	7	8	9	10	11
2	-21.71
2	0.00
3	-19.31	1.01
3	0.00	0.37
4	-15.87	-0.42	-1.15
4	0.00	0.50	0.32
5	-11.47	-0.36	-0.90	-0.05
5	0.00	0.47	0.39	0.49
6	-7.98	-0.65	-1.00	-0.43	-0.36
6	0.00	0.46	0.36	0.54	0.47
7	-5.68	-0.95	-1.18	-0.80	-0.75	-0.45
7	0.00	0.38	0.31	0.41	0.43	0.54
8	-2.96	-0.14	-0.28	-0.06	-0.05	0.10	0.37
8	0.01	0.48	0.44	0.49	0.48	0.49	0.48
9	-3.62	-1.58	-1.68	-1.52	-1.49	-1.35	-1.08	-1.20
9	0.00	0.34	0.31	0.36	0.34	0.32	0.35	0.33
10	-0.32	0.30	0.27	0.31	0.32	0.35	0.42	0.32	0.74
10	0.47	0.44	0.44	0.44	0.45	0.46	0.49	0.46	0.42
11	-2.01	-1.39	-1.42	-1.37	-1.37	-1.33	-1.26	-1.33	-0.88	-1.19
11	0.16	0.36	0.37	0.35	0.33	0.32	0.31	0.30	0.39	0.32
12	-0.21	0.40	0.37	0.42	0.42	0.46	0.52	0.42	0.84	0.08	1.27
12	0.46	0.48	0.49	0.49	0.52	0.55	0.52	0.53	0.40	0.49	0.32


Outcome: Comment Density by Reviewer
Statistically significant difference: Yes, H=275.55, p < 0.05
Active Reviewers	1	2	3	4	5	6	7	8	9	10	11
2	-15.94
2	0.00
3	-8.58	6.37
3	0.00	0.00
4	-4.78	6.58	1.69
4	0.00	0.00	0.23
5	-1.64	6.53	3.00	1.61
5	0.24	0.00	0.01	0.22
6	-0.67	4.71	2.41	1.52	0.36
6	0.33	0.00	0.05	0.25	0.38
7	-0.18	3.29	1.81	1.25	0.48	0.21
7	0.44	0.00	0.19	0.35	0.36	0.44
8	0.45	2.52	1.64	1.31	0.84	0.66	0.48
8	0.36	0.04	0.22	0.33	0.33	0.33	0.36
9	-1.02	0.48	-0.16	-0.39	-0.71	-0.81	-0.87	-1.09
9	0.34	0.35	0.44	0.38	0.33	0.33	0.34	0.35
10	0.60	1.05	0.86	0.79	0.69	0.65	0.62	0.49	0.87
10	0.33	0.35	0.34	0.32	0.34	0.33	0.33	0.37	0.35
11	-1.14	-0.69	-0.88	-0.95	-1.05	-1.08	-1.11	-1.21	-0.80	-1.23
11	0.35	0.33	0.36	0.33	0.34	0.34	0.35	0.32	0.33	0.34
12	0.76	1.21	1.02	0.95	0.85	0.81	0.78	0.65	1.02	0.11	1.34
12	0.32	0.34	0.32	0.34	0.34	0.34	0.32	0.32	0.33	0.45	0.33



		Duration					Participation					Comment Density					Comment Density by Reviewer
Active Reviewers	#Rev	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max	Mean	StdDev	Median	Min	Max
1	2431	1.30	4.50	0.00	0.00	79.00	84.30	27.40	100.00	9.10	100.00	26.60	54.40	1.90	0.00	79.00	26.60	54.40	1.90	0.00	900.00
2	2502	3.20	7.60	0.00	0.00	140.00	78.60	24.80	100.00	14.30	100.00	54.50	100.50	11.30	0.00	140.00	27.20	50.20	5.60	0.00	600.00
3	1940	6.00	11.90	2.00	0.00	211.00	78.70	21.20	75.00	21.40	100.00	57.10	116.30	9.00	0.00	211.00	19.00	38.80	3.00	0.00	400.00
4	840	7.90	12.60	4.00	0.00	103.00	77.60	18.50	80.00	28.60	100.00	68.80	155.80	10.10	0.00	103.00	17.20	39.00	2.50	0.00	350.00
5	372	14.00	26.40	6.00	0.00	268.00	76.50	15.90	71.40	35.70	100.00	61.50	151.10	9.00	0.00	268.00	12.30	30.20	1.80	0.00	260.00
6	149	18.70	33.40	6.00	0.00	193.00	78.90	14.50	75.00	46.20	100.00	59.40	116.90	10.00	0.00	193.00	9.90	19.50	1.70	0.00	108.30
7	60	18.90	29.30	11.00	0.00	156.00	78.70	13.70	77.80	43.80	100.00	44.00	73.60	14.20	0.60	156.00	6.30	10.50	2.00	0.10	42.90
8	21	17.20	19.50	9.00	0.00	71.00	77.90	12.70	80.00	50.00	100.00	31.40	48.70	13.00	0.00	71.00	3.90	6.10	1.60	0.00	27.50
9	11	33.20	30.10	21.00	0.00	96.00	81.80	15.30	81.80	52.90	100.00	192.10	321.80	15.80	3.30	96.00	21.30	35.80	1.80	0.40	122.20
10	1	35.00	0.00	35.00	35.00	35.00	37.00	0.00	37.00	37.00	37.00	6.20	0.00	6.20	6.20	35.00	0.60	0.00	0.60	0.60	0.60
11	1	15.00	0.00	15.00	15.00	15.00	84.60	0.00	84.60	84.60	84.60	262.50	0.00	262.50	262.50	15.00	23.90	0.00	23.90	23.90	23.90
12	1	20.00	0.00	20.00	20.00	20.00	85.70	0.00	85.70	85.70	85.70	5.00	0.00	5.00	5.00	20.00	0.40	0.00	0.40	0.40	0.40
TOTAL	8329	4.70	12.10	1.00	0.00	268.00	80.10	23.80	100.00	9.10	100.00	48.80	105.80	7.70	0.00	1400.00	22.90	46.90	3.20	0.00	900.00