A Complete Example

10.8 A Complete Example

Learning Objective

To see a complete linear correlation and regression analysis, in a practical setting, as a cohesive whole.

In the preceding sections numerous concepts were introduced and illustrated, but the analysis was broken into disjoint pieces by sections. In this section we will go through a complete example of the use of correlation and regression analysis of data from start to finish, touching on all the topics of this chapter in sequence.

In general educators are convinced that, all other factors being equal, class attendance has a significant bearing on course performance. To investigate the relationship between attendance and performance, an education researcher selects for study a multiple section introductory statistics course at a large university. Instructors in the course agree to keep an accurate record of attendance throughout one semester. At the end of the semester 26 students are selected a random. For each student in the sample two measurements are taken: x, the number of days the student was absent, and y, the student’s score on the common final exam in the course. The data are summarized in Table 10.4 "Absence and Score Data".

Table 10.4 Absence and Score Data

Absences	Score	Absences	Score
x	y	x	y
2	76	4	41
7	29	5	63
2	96	4	88
7	63	0	98
2	79	1	99
7	71	0	89
0	88	1	96
0	92	3	90
6	55	1	90
6	70	3	68
2	80	1	84
2	75	3	80
1	63	1	78

A scatter plot of the data is given in Figure 10.13 "Plot of the Absence and Exam Score Pairs". There is a downward trend in the plot which indicates that on average students with more absences tend to do worse on the final examination.

Figure 10.13 Plot of the Absence and Exam Score Pairs

The trend observed in Figure 10.13 "Plot of the Absence and Exam Score Pairs" as well as the fairly constant width of the apparent band of points in the plot makes it reasonable to assume a relationship between x and y of the form

y = β_{1} x + β_{0} + ε

where $β_{1}$ and $β_{0}$ are unknown parameters and ε is a normal random variable with mean zero and unknown standard deviation σ. Note carefully that this model is being proposed for the population of all students taking this course, not just those taking it this semester, and certainly not just those in the sample. The numbers $β_{1}$ , $β_{0}$ , and σ are parameters relating to this large population.

First we perform preliminary computations that will be needed later. The data are processed in Table 10.5 "Processed Absence and Score Data".

Table 10.5 Processed Absence and Score Data

x	y	x²	xy	y²	x	y	x²	xy	y²
2	76	4	152	5776	4	41	16	164	1681
7	29	49	203	841	5	63	25	315	3969
2	96	4	192	9216	4	88	16	352	7744
7	63	49	441	3969	0	98	0	0	9604
2	79	4	158	6241	1	99	1	99	9801
7	71	49	497	5041	0	89	0	0	7921
0	88	0	0	7744	1	96	1	96	9216
0	92	0	0	8464	3	90	9	270	8100
6	55	36	330	3025	1	90	1	90	8100
6	70	36	420	4900	3	68	9	204	4624
2	80	4	160	6400	1	84	1	84	7056
2	75	4	150	5625	3	80	9	240	6400
1	63	1	63	3969	1	78	1	78	6084

Adding up the numbers in each column in Table 10.5 "Processed Absence and Score Data" gives

Σ x = 71, Σ y = 2001, Σ x^{2} = 329, Σ x y = 4758, and Σ y^{2} = 161511 .

Then

\begin{array}{l} S S_{x x} & = & Σ x^{2} - \frac{1}{n} {(Σ x)}^{2} = 329 - \frac{1}{26} {(71)}^{2} = 135.1153846 \\ S S_{x y} & = & Σ x y - \frac{1}{n} (Σ x) (Σ y) = 4758 - \frac{1}{26} (71) (2001) = − 706.2692308 \\ S S_{y y} & = & Σ y^{2} - \frac{1}{n} {(Σ y)}^{2} = 161511 - \frac{1}{26} {(2001)}^{2} = 7510.961538 \end{array}

and

\bar{x} = \frac{Σ x}{n} = \frac{71}{26} = 2.730769231 and \bar{y} = \frac{Σ y}{n} = \frac{2001}{26} = 76.96153846

We begin the actual modelling by finding the least squares regression line, the line that best fits the data. Its slope and y-intercept are

\begin{array}{l} {\hat{β}}_{1} & = & \frac{S S_{x y}}{S S_{x x}} = \frac{− 706.2692308}{135.1153846} = − 5.227156278 \\ {\hat{β}}_{0} & = & \bar{y} - {\hat{β}}_{1} \bar{x} = 76.96153846 - (− 5.227156278) (2.730769231) = 91.23569553 \end{array}

Rounding these numbers to two decimal places, the least squares regression line for these data is

\hat{y} = − 5.23 x + 91.24 .

The goodness of fit of this line to the scatter plot, the sum of its squared errors, is

S S E = S S_{y y} - {\hat{β}}_{1} S S_{x y} = 7510.961538 - (− 5.227156278) (− 706.2692308) = 3819.181894

This number is not particularly informative in itself, but we use it to compute the important statistic

s_{ε} = \sqrt{\frac{S S E}{n − 2}} = \sqrt{\frac{3819.181894}{24}} = 12.11988495

The statistic $s_{ε}$ estimates the standard deviation σ of the normal random variable ε in the model. Its meaning is that among all students with the same number of absences, the standard deviation of their scores on the final exam is about 12.1 points. Such a large value on a 100-point exam means that the final exam scores of each sub-population of students, based on the number of absences, are highly variable.

The size and sign of the slope ${\hat{β}}_{1} = − 5.23$ indicate that, for every class missed, students tend to score about 5.23 fewer points lower on the final exam on average. Similarly for every two classes missed students tend to score on average $2 \times 5.23 = 10.46$ fewer points on the final exam, or about a letter grade worse on average.

Since 0 is in the range of x-values in the data set, the y-intercept also has meaning in this problem. It is an estimate of the average grade on the final exam of all students who have perfect attendance. The predicted average of such students is ${\hat{β}}_{0} = 91.24 .$

Before we use the regression equation further, or perform other analyses, it would be a good idea to examine the utility of the linear regression model. We can do this in two ways: 1) by computing the correlation coefficient r to see how strongly the number of absences x and the score y on the final exam are correlated, and 2) by testing the null hypothesis $H_{0} : β_{1} = 0$ (the slope of the population regression line is zero, so x is not a good predictor of y) against the natural alternative $H_{a} : β_{1} < 0$ (the slope of the population regression line is negative, so final exam scores y go down as absences x go up).

The correlation coefficient r is

r = \frac{S S_{x y}}{\sqrt{S S_{x x} S S_{y y}}} = \frac{− 706.2692308}{\sqrt{(135.1153846) (7510.961538)}} = − 0.7010840977

a moderate negative correlation.

Turning to the test of hypotheses, let us test at the commonly used 5% level of significance. The test is

\begin{array}{l} H_{0} : β_{1} = 0 \\ vs . H_{a} : β_{1} < 0 @ α = 0.05 \end{array}

From Figure 12.3 "Critical Values of ", with $d f = 26 - 2 = 24$ degrees of freedom $t_{0.05} = 1.711$ , so the rejection region is $(− \infty, − 1.711] .$ The value of the standardized test statistic is

t = \frac{{\hat{β}}_{1} - B_{0}}{s_{ε} ∕ \sqrt{S S_{x x}}} = \frac{− 5.227156278 - 0}{12.11988495 ∕ \sqrt{135.1153846}} = − 5.013

which falls in the rejection region. We reject H₀ in favor of H_a. The data provide sufficient evidence, at the 5% level of significance, to conclude that $β_{1}$ is negative, meaning that as the number of absences increases average score on the final exam decreases.

As already noted, the value $β_{1} = − 5.23$ gives a point estimate of how much one additional absence is reflected in the average score on the final exam. For each additional absence the average drops by about 5.23 points. We can widen this point estimate to a confidence interval for $β_{1} .$ At the 95% confidence level, from Figure 12.3 "Critical Values of " with $d f = 26 - 2 = 24$ degrees of freedom, $t_{α ∕ 2} = t_{0.025} = 2.064 .$ The 95% confidence interval for $β_{1}$ based on our sample data is

{\hat{β}}_{1} \pm t_{α ∕ 2} \frac{s_{ε}}{\sqrt{S S_{x x}}} = − 5.23 \pm 2.064 \frac{12.11988495}{\sqrt{135.1153846}} = − 5.23 \pm 2.15

or $(− 7.38, − 3.08) .$ We are 95% confident that, among all students who ever take this course, for each additional class missed the average score on the final exam goes down by between 3.08 and 7.38 points.

If we restrict attention to the sub-population of all students who have exactly five absences, say, then using the least squares regression equation $\hat{y} = − 5.23 x + 91.24$ we estimate that the average score on the final exam for those students is

\hat{y} = − 5.23 (5) + 91.24 = 65.09

This is also our best guess as to the score on the final exam of any particular student who is absent five times. A 95% confidence interval for the average score on the final exam for all students with five absences is

\begin{array}{l} {\hat{y}}_{p} \pm t_{α ∕ 2} s_{ε} \sqrt{\frac{1}{n} + \frac{{(x_{p} - \bar{x})}^{2}}{S S_{x x}}} & = & 65.09 \pm (2.064) (12.11988495) \sqrt{\frac{1}{26} + \frac{{(5 - 2.730769231)}^{2}}{135.1153846}} \\ = & 65.09 \pm 25.01544254 \sqrt{0.0765727299} \\ = & 65.09 \pm 6.92 \end{array}

which is the interval $(58 . 17,72 . 01) .$ This confidence interval suggests that the true mean score on the final exam for all students who are absent from class exactly five times during the semester is likely to be between 58.17 and 72.01.

If a particular student misses exactly five classes during the semester, his score on the final exam is predicted with 95% confidence to be in the interval

\begin{array}{l} {\hat{y}}_{p} \pm t_{α ∕ 2} s_{ε} \sqrt{1 + \frac{1}{n} + \frac{{(x_{p} - \bar{x})}^{2}}{S S_{x x}}} & = & 65.09 \pm 25.01544254 \sqrt{1.0765727299} \\ = & 65.09 \pm 25.96 \end{array}

which is the interval $(39 . 13,91 . 05) .$ This prediction interval suggests that this individual student’s final exam score is likely to be between 39.13 and 91.05. Whereas the 95% confidence interval for the average score of all student with five absences gave real information, this interval is so wide that it says practically nothing about what the individual student’s final exam score might be. This is an example of the dramatic effect that the presence of the extra summand 1 under the square sign in the prediction interval can have.

Finally, the proportion of the variability in the scores of students on the final exam that is explained by the linear relationship between that score and the number of absences is estimated by the coefficient of determination, r². Since we have already computed r above we easily find that

r^{2} = {(− 0.7010840977)}^{2} = 0.491518912

or about 49%. Thus although there is a significant correlation between attendance and performance on the final exam, and we can estimate with fair accuracy the average score of students who miss a certain number of classes, nevertheless less than half the total variation of the exam scores in the sample is explained by the number of absences. This should not come as a surprise, since there are many factors besides attendance that bear on student performance on exams.

Key Takeaway

It is a good idea to attend class.

Exercises

The exercises in this section are unrelated to those in previous sections.

The data give the amount x of silicofluoride in the water (mg/L) and the amount y of lead in the bloodstream (μg/dL) of ten children in various communities with and without municipal water. Perform a complete analysis of the data, in analogy with the discussion in this section (that is, make a scatter plot, do preliminary computations, find the least squares regression line, find $S S E$ , $s_{ε}$ , and r, and so on). In the hypothesis test use as the alternative hypothesis $β_{1} > 0$ , and test at the 5% level of significance. Use confidence level 95% for the confidence interval for $β_{1} .$ Construct 95% confidence and predictions intervals at $x_{p} = 2$ at the end.
$\begin{matrix} x & 0.0 & 0.0 & 1.1 & 1.4 & 1.6 \\ y & 0.3 & 0.1 & 4.7 & 3.2 & 5.1 \end{matrix}$ $\begin{matrix} x & 1.7 & 2.0 & 2.0 & 2.2 & 2.2 \\ y & 7.0 & 5.0 & 6.1 & 8.6 & 9.5 \end{matrix}$
The table gives the weight x (thousands of pounds) and available heat energy y (million BTU) of a standard cord of various species of wood typically used for heating. Perform a complete analysis of the data, in analogy with the discussion in this section (that is, make a scatter plot, do preliminary computations, find the least squares regression line, find $S S E$ , $s_{ε}$ , and r, and so on). In the hypothesis test use as the alternative hypothesis $β_{1} > 0$ , and test at the 5% level of significance. Use confidence level 95% for the confidence interval for $β_{1} .$ Construct 95% confidence and predictions intervals at $x_{p} = 5$ at the end.
$\begin{matrix} x & 3.37 & 3.50 & 4.29 & 4.00 & 4.64 \\ y & 23.6 & 17.5 & 20.1 & 21.6 & 28.1 \end{matrix}$ $\begin{matrix} x & 4.99 & 4.94 & 5.48 & 3.26 & 4.16 \\ y & 25.3 & 27.0 & 30.7 & 18.9 & 20.7 \end{matrix}$

Large Data Set Exercises

Large Data Sets 3 and 3A list the shoe sizes and heights of 174 customers entering a shoe store. The gender of the customer is not indicated in Large Data Set 3. However, men’s and women’s shoes are not measured on the same scale; for example, a size 8 shoe for men is not the same size as a size 8 shoe for women. Thus it would not be meaningful to apply regression analysis to Large Data Set 3. Nevertheless, compute the scatter diagrams, with shoe size as the independent variable (x) and height as the dependent variable (y), for (i) just the data on men, (ii) just the data on women, and (iii) the full mixed data set with both men and women. Does the third, invalid scatter diagram look markedly different from the other two?

http://www.gone.2012books.lardbucket.org/sites/all/files/data3.xls

http://www.gone.2012books.lardbucket.org/sites/all/files/data3A.xls
Separate out from Large Data Set 3A just the data on men and do a complete analysis, with shoe size as the independent variable (x) and height as the dependent variable (y). Use $α = 0.05$ and $x_{p} = 10$ whenever appropriate.

http://www.gone.2012books.lardbucket.org/sites/all/files/data3A.xls
Separate out from Large Data Set 3A just the data on women and do a complete analysis, with shoe size as the independent variable (x) and height as the dependent variable (y). Use $α = 0.05$ and $x_{p} = 10$ whenever appropriate.

http://www.gone.2012books.lardbucket.org/sites/all/files/data3A.xls

Answers

$Σ x = 14.2$ , $Σ y = 49.6$ , $Σ x y = 91.73$ , $Σ x^{2} = 26.3$ , $Σ y^{2} = 333.86 .$

$S S_{x x} = 6.136$ , $S S_{x y} = 21.298$ , $S S_{y y} = 87.844 .$

$\bar{x} = 1.42$ , $\bar{y} = 4.96 .$

${\hat{β}}_{1} = 3.47$ , ${\hat{β}}_{0} = 0.03 .$

$S S E = 13.92 .$

$s_{ε} = 1.32 .$

r = 0.9174, r² = 0.8416.

$d f = 8$ , T = 6.518.

The 95% confidence interval for $β_{1}$ is: $(2 . 24,4 . 70) .$

At $x_{p} = 2$ , the 95% confidence interval for $E (y)$ is $(5 . 77,8 . 17) .$

At $x_{p} = 2$ , the 95% prediction interval for y is $(3 . 73,10 . 21) .$

The positively correlated trend seems less profound than that in each of the previous plots.
The regression line: $\hat{y} = 3.3426 x + 138.7692 .$ Coefficient of Correlation: r = 0.9431. Coefficient of Determination: r² = 0.8894. $S S E = 283.2473 .$ $s_{e} = 1.9305 .$ A 95% confidence interval for $β_{1}$ : $(3 . 0733,3 . 6120) .$ Test Statistic for $H_{0} : β_{1} = 0$ : T = 24.7209. At $x_{p} = 10$ , $\hat{y} = 172.1956$ ; a 95% confidence interval for the mean value of y is: $(171 . 5577,172 . 8335)$ ; and a 95% prediction interval for an individual value of y is: $(168 . 2974,176 . 0938) .$

Absences	Score	Absences	Score
x	y	x	y
2	76	4	41
7	29	5	63
2	96	4	88
7	63	0	98
2	79	1	99
7	71	0	89
0	88	1	96
0	92	3	90
6	55	1	90
6	70	3	68
2	80	1	84
2	75	3	80
1	63	1	78

Absences	Score	Absences	Score
x	y	x	y
2	76	4	41
7	29	5	63
2	96	4	88
7	63	0	98
2	79	1	99
7	71	0	89
0	88	1	96
0	92	3	90
6	55	1	90
6	70	3	68
2	80	1	84
2	75	3	80
1	63	1	78

Absences	Score	Absences	Score
x	y	x	y
2	76	4	41
7	29	5	63
2	96	4	88
7	63	0	98
2	79	1	99
7	71	0	89
0	88	1	96
0	92	3	90
6	55	1	90
6	70	3	68
2	80	1	84
2	75	3	80
1	63	1	78