Part 3 - Confidence Interval and Hypothesis Testing

Inferences for mean/proportion

Let’s recap what a confidence interval is,

When dealing with confidence intervals, we will often encounter different “types” of situations. Let’s review these.

Confidence Interval on the mean of a Normal Distribution, variance known

Suppose we have a normal distribution with unknown mean $\mu$ and known variance $\sigma^2$.

We have thus a random sample $X_1, X_2, \ldots, X_n$, such that, for all $i$,

with $\mu$ unknown and $\sigma$ a known constant.

We would like a confidence interval for $\mu$.

We know that,

we can standardize this to,

So, because $Z \sim N(0, 1)$,

Thus,

Re-arranging this, we get,

Let’s call them $L$ and $U$ for lower and upper, finally, the interval is,

Confidence Interval on the mean of an arbitrary distribution, variance known

Let us recap the central limit theorem.

The Central Limit Theorem (CLT) implies if $n$ is large enough,

Thus,

Which yields the (same) interval,

Confidence interval on the mean of a normal distribution, variance unknown

In the case we do not have the variance, we need to sample it!

Recall the sample variance,

A natural procedure is thus to,

In a normal population, the exact distribution of $T$ is $T \sim t_{n - 1}$.

We can write,

or,

Which yields us the interval,

In large samples,

Consequently, an approximate confidence interval of level $100(1 - \alpha)\%$ for $\mu$ is,

Confidence Interval on the mean: Summary

Is the population normal?

• If yes, is $\sigma$ known?
  • If yes, use an exact $z$-confidence interval:
    • $[L, U] = \left[\bar{x} - z_{1 - \alpha/2} \frac{\sigma}{\sqrt{n}}, \bar{x} + z_{1 - \alpha/2} \frac{\sigma}{\sqrt{n}}\right]$
  • If no, use an exact $t$-confidence interval:
    • $[L, U] = \left[\bar{x} - t_{n - 1;1 - \alpha/2} \frac{s}{\sqrt{n}}, \bar{x} + t_{n - 1;1 - \alpha/2} \frac{s}{\sqrt{n}}\right]$
• If no, use an approximate large sample confidence interval:
  • $[L, U] = \left[\bar{x} - z_{1 - \alpha/2} \frac{\sigma}{\sqrt{n}}, \bar{x} + z_{1 - \alpha/2} \frac{\sigma}{\sqrt{n}}\right]$
  • or
  • $[L, U] = \left[\bar{x} - z_{1 - \alpha/2} \frac{s}{\sqrt{n}}, \bar{x} + z_{1 - \alpha/2} \frac{s}{\sqrt{n}}\right]$

Confidence Interval on the proportion

The random variable to study is,

$X_1, X_2, \ldots, X_n$ is a set of $n$ independent Bern($p$) random variables.

Thus,

and the sample proportion is,

We also know that,

if $n$ is large.

Approximate two-sided confidence interval of level $100(1 - \alpha)\%$ for $p$ is given by,

Hypotheses testing for the mean

Let’s recall hypothesis testing. The null hypothesis is usually of the form,

The alternative hypothesis can be a two-sided alternative,

or one-sided alternatives,

Remember, we have two important cases

1. Rejecting $H_0$ when it is true: type I error.
2. Failing to reject $H_0$ when it is false: type II error.

Note that $\beta$ depends on the (unknown) value of $\mu$ under the alternative.

Assume for the moment that the population is normal with known $\sigma$.

At significance level $\alpha$, we are after two constants $\ell$ and $u$ such that,

Thus,

and,

This yields,

The decision rule is then,

Hypotheses testing for the mean: $p$-value

The $p$-value is the probability that the test statistic will take on a value that is at least as extreme as the observed value when $H_0$ is true. (extreme is to be understood in the direction of the alternative).

When testing $H_0 : \mu = \mu_0$ against $H_a : \mu \neq \mu_0$, the $p$-value will be the probability of finding the random variable $\bar{X}$ more different to $\mu_0$ than the observed $\bar{x}$, that is,

Let us define,

as the “observed value of the test statistic”.

As we know that $Z = \sqrt{n} \frac{\bar{X} - \mu_0}{\sigma} \sim N(0, 1)$, we can write,

Operationally, since a $p$-value is computed, we typically compare it to a predefined significance level $\alpha$ to make a decision:

Hypotheses testing for the mean: one-sided

With $H_a : \mu > \mu_0$ we are after a constant $u$ such that,

As we know that $Z = \sqrt{n} \frac{\bar{X} - \mu_0}{\sigma} \sim N(0, 1)$, the decision rule is reject $H_0$ if $\bar{x} > \mu_0 + z_{1 - \alpha} \frac{\sigma}{\sqrt{n}}$.

Again with $z_0 = \sqrt{n} \frac{\bar{x} - \mu_0}{\sigma}$, the $p$-value is,

With $H_a : \mu < \mu_0$, we are after a constant $\ell$ such that,

The decision rule is reject $H_0$ if $\bar{x} < \mu_0 - z_{1 - \alpha} \frac{\sigma}{\sqrt{n}}$.

The $p$-value is,

Hypotheses testing for the mean: other cases

Say we have a normal population with unknown standard deviation.

Specifically, for the two-sided test $H_0 : \mu = \mu_0$ against $H_a : \mu \neq \mu_0$, the decision rule is,

and from the observed value of the test statistic,

we can compute the $p$-value,

If we have non-normal populations with known or unknown standard deviation,

or,

The associated approximate $p$-value will be given by,

with $z_0 = \sqrt{n} \frac{\bar{x} - \mu_0}{\sigma}$ or $z_0 = \sqrt{n} \frac{\bar{x} - \mu_0}{s}$.

Hypotheses testing for the proportion

Large-sample confidence interval for $p$,

The decision rule at (approximate) level $\alpha$ is,

The (approximate) $p$-value for this test is,

where $z_0 = \sqrt{n} \frac{\hat{p} - p_0}{\sqrt{p_0(1 - p_0)}}$.

R code for finding $z_{1 - \alpha/2}$

1
alpha<-0.05
2
zstar<-qnorm(1-alpha/2)

Example

Suppose 53 people among 100 surveyed is for the proposition, find the 95% confidence interval for the proportion of people in favor of the proposition.

1
n<-100
2
phat<-53/n
3
SE<-sqrt(phat*(1-phat)/n)
4
alpha<-0.05
5
zstar<-qnorm(1 - alpha/2)
6
> c(phat-zstar*SE, phat+zstar*SE)
7
[1] 0.432178 0.6278216

Built-in functions in R for finding the CI of proportion estimate

1
prop.test(x=53, n=100, conf.level=0.95)
2
>   1-sample proportions test with continuity correction
3
>
4
> data:  53 out of 100, null probability 0.5
5
> X-squared = 0.25, df = 1, p-value = 0.6171
6
> alternative hypothesis: true p is not equal to 0.5
7
> 95 percent confidence interval:
8
>   0.4280225 0.6296465
9
> sample estimates:
10
>   p
11
> 0.53

Built-in functions in R for finding the CI of mean estimate

1
x<-c(175, 185, 170, 184, 175)
2
t.test(x, conf.level=0.90)
3
>   One Sample t-test
4
>
5
> data:  x
6
> t = 61.567, df = 4, p-value = 4.169e-07
7
> alternative hypothesis: true mean is not equal to 0
8
> 90 percent confidence interval:
9
>   171.6434 183.9566
10
> sample estimates:
11
> mean of x
12
>  177.8
13

14

15
t.test(x, conf.level=0.90, alt="less")
16
>   One Sample t-test
17
>
18
> data:  x
19
> t = 61.567, df = 4, p-value = 1
20
> alternative hypothesis: true mean is less than 0
21
> 90 percent confidence interval:
22
>   -Inf 182.2278
23
> sample estimates:
24
> mean of x
25
>  177.8

Inferences for difference of means

In many situations it is quite common to be interested in *comparing two ‘populations’ in regard to a parameter of interest.

The two ‘populations’ may be:

• Produced items using an existing and a new technique.
• Success rates in two groups of individuals.
• Health test results for patients who received a drug and for patients who received a placebo.

Two-sample test

$X_{11}, X_{12}, \ldots, X_{1n_1}$ is a sample from population 1. $X_{21}, X_{22}, \ldots, X_{2n_2}$ is a sample from population 2.

The samples are independent (i.e., observations in sample 1 are by no means linked to the observations in sample 2, they concern different individuals).

What we would like to know is whether $\mu_1 = \mu_2$ or not.

So,

We compute the sample means $\bar{x}_1$ and $\bar{x}_2$.

• If $\bar{x}_1 \simeq \bar{x}_2$, then $H_0$ is probably acceptable.
• If $\bar{x}_1$ is considerably different to $\bar{x}_2$, that is evidence that $H_0$ is not true and we are tempted to reject it.

Note that the alternative hypothesis can be,

or,

We know that,

and,

we deduce the sampling distribution of $\bar{X}_1 - \bar{X}_2$,

Now, testing for $H_0 : \mu_1 = \mu_2$ exactly amounts to for $H_0 : \mu_1 - \mu_2 = 0$, with $\bar{X}_1 - \bar{X}_2$ as an estimator for $\mu_1 - \mu_2$.

Two-sample test: known variances

Suppose that $\sigma_1$ and $\sigma_2$ are known.

For the two-sided test (with $H_a : \mu_1 - \mu_2 \neq 0$), at significance level $\alpha$, the decision rule is,

$p$-value is,

where $z_0 = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}}$.

Similarly, for the one-sided test $H_a : \mu_1 > \mu_2$, the decision rule is,

and the $p$-value is,

while for the one-sided test $H_a : \mu_1 < \mu_2$, the decision rule is,

and the $p$-value is,

$100(1 - \alpha)\%$ two-sided confidence interval for $\mu_1 - \mu_2$ is,

$100(1 - \alpha)\%$ one-sided confidence intervals for $\mu_1 - \mu_2$ are,

and,

Two-sample test: unknown equal variances

Assume now $\sigma_1 = \sigma_2 = \sigma$, but $\sigma$ is unknown.

We can estimate $\sigma^2$ by the pooled variance estimator,

Where $S_1^2$ and $S_2^2$ are the sample variances of the two samples,

and,

For the two-sided test (with $H_a : \mu_1 - \mu_2 \neq 0$), at significance level $\alpha$, the decision rule is,

The $p$-value is given by,

with $T \sim t_{n_1 + n_2 - 2}$, and where $t_0$ is the observed value of the test statistic,

This test is known as the two-sample $t$-test.

A $100(1 - \alpha)\%$ two-sided confidence interval for $\mu_1 - \mu_2$ is,

Two-sample test: unknown unequal variances

There is no exact result available. An approximate result can be applied,

where the number of degrees of freedom is,

(rounded down to the nearest integer).

This is called Welch’s two sample $t$-test.

Example

6 subjects were given a drug (treatment group, $\mu_2$) and an additional 6 subjects a placebo (control group, $\mu_1$). Their reaction time to a stimulus was measured (in ms). Perform a two sample $t$-test for comparing the means of the treatment and control groups

Let’s use a one-sided test.

1
control<-c(91, 87, 99, 77, 88, 91)
2
treat<-c(101, 110, 103, 93, 99, 104)
3
t.test(control, treat, alternative="less", var.equal=TRUE)
4
>  Two Sample t-test
5
>
6
> data:  control and treat
7
> t = -3.4456, df = 10, p-value = 0.003136
8
> alternative hypothesis: true difference in means is less than 0
9
> 95 percent confidence interval:
10
>   -Inf -6.082744
11
> sample estimates:
12
> mean of x mean of y
13
> 88.83333 101.66667

We can also do a Welch test,

1
t.test(control, treat, alternative="less")
2
>  Welch Two Sample t-test
3
>
4
> data:  control and treat
5
> t = -3.4456, df = 9.4797, p-value = 0.003391
6
> alternative hypothesis: true difference in means is less than 0
7
> 95 percent confidence interval:
8
>   -Inf -6.044949
9
> sample estimates:
10
> mean of x mean of y
11
> 88.83333 101.66667

Paired data for difference of means

Two sample $t$-test cannot be used when we deal with “before and after” data, and numerous situations where the data are naturally paired (and thus, not independent).

Let $(X_{11}, X_{21}), (X_{12}, X_{22}), \ldots, (X_{1n}, X_{2n})$ be a random sample of $n$ pairs of observations drawn from two subpopulations $X_1$ and $X_2$, with respective means $\mu_1$ and $\mu_2$.

An easy way is just to consider the differences.

We have just a sample $D_1, D_2, \ldots, D_n$ from a distribution with mean,

Testing for $H_0 : \mu_1 = \mu_2$ is just $H_0 : \mu_D = 0$. This can be accomplished by performing the usual one sample test for mean.

So, in R we can,

1
t.test(x,y, paired=TRUE)
2
# or
3
t.test(x-y)

Inferences for variance

We know that,

and is a natural estimator for the population variance $\sigma^2$.

In general, little can be said about the distribution of $S^2$. However, when the population is normal, that is $X \sim N(\mu, \sigma^2)$, then,

Let $\chi^2_{\nu; \alpha}$ be the value such that,

for $X \sim \chi^2_{\nu}$.

As we know that,

we can write,

which can be re-arranged to,

Which gives us the interval,

Now, $H_0 : \sigma^2 = \sigma_0^2$ against $H_a : \sigma^2 \neq \sigma_0^2$. It is natural to reject $H_0$ whenever $s^2$ is too distant from $\sigma_0^2$.

We are after two constants $\ell$ and $u$ such that,

This yields,

The decision rule is then,

One-sided CI,

One-sided test,

The $p$-value is,

Inferences for ratio of variances/test of equality of variances

Let $X_{11}, X_{12}, \ldots, X_{1n_1}$ be a sample from population 1. Let $X_{21}, X_{22}, \ldots, X_{2n_2}$ be a sample from population 2.

The samples are independent, and we would like to know whether $\sigma_1^2 = \sigma_2^2$ or not.

Define the sample variances,

We can use the test statistic,

In general, there is no known exact distribution for $F$. Fortunately, if $X_{1i} \sim N(\mu_1, \sigma_1^2)$ and $X_{2i} \sim N(\mu_2, \sigma_2^2)$, $F$ has an $F(n_1 - 1, n_2 - 1)$ distribution if the null ($\sigma_1^2 = \sigma_2^2$) is true.

$H_0 : \sigma_1^2 = \sigma_2^2$ against $H_a : \sigma_1^2 \neq \sigma_2^2$.

It is natural to reject $H_0$ whenever $F$ is too big or too small,

The decision rule is,

Two-sided CI for $\frac{\sigma_1^2}{\sigma_2^2}$,

Which yields us the interval,

One-sided CI,

One-sided test,