Rowan-Classes/5th-Semester-Fall-2023/Prob-and-Stat-for-ECEs/Notes/Chapter-05.md

Chapter 5

Statistical Inference

Statistical inference is the process of drawing conclusions about the entire population based on information from a sample.

Parameter vs. Statistic

A parameter is a number that summarizes data from an entire population.

A statistic is a number that summarizes data from a sample.

	parameter	statistic
mean	\mu	\bar{x}
standard deviation	\sigma	s
variance	\sigma^2	s^2

Example

Suppose you were interested in the number of hours that Rowan students spend studying on Sundays. You take a random sample of n = 100 students and the average time they study on Sunday is $\bar{x}= 3.2$[hrs].

We use $\bar{x} = 3.2$[hrs] as our best estimate for \mu.

Variability of Sample Statistics

We normally think of a parameter as a fixed value. Sample statistics vary from sample to sample.

Sampling Distribution

A sampling distribution is the distribution of sample statistics computed for different samples of the same sample size from the same population.

The mean of the sample means is \mu. For a random sample of size, n, the standard error is given by:

\text{var}(\bar{x}) = {\sigma^2 \over n}

Central Limit Theorem

If \bar{x} is the mean of a random sample of size, n, taken from a population with mean, \mu, and finite variance, \sigma^2, then the limiting form of the distribution.

z = {\sqrt{n} (\bar{x} - \mu )\over \sigma}

As n \to \infty, is the standard normal distribution. This generally holds for n \ge 30. If n < 30, the approximation is good so long as the population is not too different from a normal distribution.

Unbiased Estimator

A statistic, \hat{\theta}, is said to be an unbiased estimator of the parameter, \theta, if:

E[\hat{\theta}] = \theta

or

E[\hat{\theta} - \theta] =0

The mean:

\bar{x} = {1\over n} \sum_{i=1}^{n} x_i

is an unbiased estimator of \mu.

Proof:

E[\bar{x}] = E\left[ {1\over n} \sum_{i=1}^n x_i\right]

= {1\over n} E[x_1 + x_2 + x_3 + \cdots + x_n]

= {1\over n} \left[ E[x_1] + E[x_2] + \cdots + E[x_n]\right]

= {1\over n} [\mu + \mu + \cdots + \mu]

= {1\over n} [n\mu] = \mu

Confidence Interval for \mu if \sigma is known:

If our sample size is "large", then the CLT tells us that:

{\sqrt{n} (\bar{x} - \mu) \over \sigma} \sim N(0,1) \text{ as } n \to \infty

1 - \alpha = P(-z_{\alpha \over 2} \le {\bar{x} - \mu \over \sigma/\sqrt{n}} \le z_{\alpha \over2}

A (1 - \alpha)% confidence interval for \mu is:

\bar{x} \pm z_{\alpha \over 2} {\sigma \over \sqrt{n}}

90% CI: z_{\alpha \over 2} = 1.645

95% CI: $z_{\alpha \over 2} = 1.96$$

99% CI: z_{\alpha \over 2} = 2.576

Example

In a random sample of 75 Rowan students, the sample mean height was 67 inches. Suppose the population standard deviation is known to be \sigma = 7 inches. Construct a 95% confidence interval for the mean height of all rowan students.

\bar{x} \pm z_{\alpha \over 2} {\sigma \over \sqrt{n}}

\bar{x} = 67

z_{\alpha \over 2} = 1.96

\sigma = 7

n = 75

A 95% CI for \mu:

67 \pm 1.96 \left({7\over\sqrt{75}}\right) = (65.4, 68.6)

Interpretation

95% confident that the mean height of all Rowan students is somewhere between 65.4 and 68.6 inches.

From the sample, we found that \bar{x} = 67 inches. Using the confidence interval, we are saying that we are 95% confident that \mu is somewhere between 65.4 and 68.6 inches.

A limitation of z confidence interval is that \sigma is unlikely to be known.

Confidence interval for \mu if \sigma is unknown:

If \sigma is unknown, we then estimate the standard error, {\sigma \over \sqrt{n}} as {s \over \sqrt{n}}.

When we estimate the standard error, the distribution is not normal. Instead, it follows a t-distribution with n-1 degrees of freedom. The new distribution is given as:

{\bar{x} - \mu \over {s \over \sqrt{n}}}

A (1 - \alpha)% confidence interval for \mu when \sigma is unknown is:

\bar{x} \pm t^* {s\over \sqrt{n}}

Where t^* is an end point chosen from the t-distribution. t^* varies based on sample size and desired confidence level.

Example

A research engineer for a time manufacturer is investigating tire life for a new rubber compound and has built 115 tires and tested them to end-of-life in a road test. The sample mean and standard deviation are 60139.7, and 3645.94 kilometers.

Find a 90% confidence interval for the mean life of all such tires.

\bar{x} \pm t^* {s\over\sqrt{n}}

\bar{x} = 60139.7

s = 3645.94

n = 115

t^* = \texttt{t\_crit\_value(115, 0.90)} = 1.658

60139.7 \pm 1.658 {3645.94 \over \sqrt{115}} = (59567.1, 60703.3)

Width of a Confidence Interval

\bar{x} \pm t_{\alpha \over 2} {s \over \sqrt{n}}

As sample size increases the width of the confidence interval decreases, and \bar{x} becomes a better approximation of \mu.

\lim_{n\to\infty} {s \over \sqrt{n}} = 0

\lim_{n\to\infty} P(|\bar{x} - \mu| < \varepsilon) = 1

Where \varepsilon > 0.

One-Sided Confidence Intervals

\left(-\infty, \bar{x} + t_\alpha {s \over \sqrt{n}}\right)

Confidence Intervals in Python

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import scipy.stats

conf_levels = []
iterations = 100

def tvalue(sample_size, conf_level):
    return stat.t.ppf(1 - (1 - conf_level)/2, sample_size - 1)

for i in range(iterations)
    sample = np.random.chisquare(df=10, size=100)
    sample_mean = np.mean(sample)
    std = np.std(sample)
    t_value = tvalue(100, .95)
    lb = sample_mean - t_value*(std / np.sqrt(100))
    ub = sample_mean + t_value*(std / np.sqrt(100))
    conf_levels.append((lb, ub))

plt.figure(figsize=(15,5))

for j, (lb, ub) in enumerate(conf_levels):
    if 10 < lb or 10 > ub:
        plt.plot([j,j], [lb,ub], 'ro-', color='red')
    else:
        plt.plot([j,j], [lb,ub], 'ro-', color='green')

plt.show()

Hypothesis Testing

Many problems require that we decide whether to accept or reject a statement about some parameter.

Hypothesis

A claim that we want to test or investigate

Hypothesis Test

A statistical test that is used to determine whether results from a sample are convincing enough to allow us to conclude something about the population.

Use sample evidence to back up claims about a population

Null Hypothesis

The claim that there is no effect or no difference (H_0).

Alternative Hypothesis

The claim for which we seek evidence (H_a).

Using H_0 and H_a

Does the average Rowan student spend more than $300 each semester on books?

In a sample of 226 Rowan students, the mean cost of a students textbook was $344 with a standard deviation of $106.

H_0: \mu = 300.

H_a: \mu > 300.

H_0 and H_a are statements about population parameters, not sample statistics.

In general, the null hypothesis is a statement of equality (=), while the alternative hypothesis is a statement of inequality (<, >, \ne).

Possible outcomes of a hypothesis test

1. Reject the null hypothesis
   • Rejecting H_0 means we have enough evidence to support the alternative hypothesis
2. Fail to reject the null hypothesis
   • Not enough evidence to support the alternative hypothesis

Figuring Out Whether Sample Data is Supported

If we assume that the null hypothesis is true, what is the probability of observing sample data that is as extreme or more extreme than what we observed.

In the Rowan example, we found that \bar{x} = 344.

One-Sample T-test for a Mean

To test a hypothesis regarding a single mean, there are two main parametric options:

1. z-test
2. t-test

The z-test requires knowledge of the population standard deviation. Since \sigma is unlikely to be known, we will use a t-test.

To test H_0: \mu = \mu_0 against its alternative H_a: \mu \ne \mu_0, use the t-statistic.

t^* = {\bar{x} - \mu_0 \over {s \over \sqrt{n}}}

P-Value

A measure of inconsistency between the null hypothesis and the sample data.

Significance Level (\alpha)

\alpha for a test of hypothesis is a boundary below which we conclude that a p-value shows statistically significant evidence against the null.

Common \alpha levels are 0.01, 0.05, 0.10.

The lower the \alpha, the stronger the evidence required to reject H_0. If the p-value is less than \alpha, reject H_0, but if the p-value is greater than \alpha, fail to reject H_0.

Steps of a Hypothesis Test

1. State the H_0 and H_a
2. Calculate the test statistic
3. Find the p-value
4. Reject or fail to reject H_0
5. Write conclusion in the context of the problem

Example

A researcher is interested in testing a particular brand of batteries and whether its battery life exceeds 40 hours.

A random sample of n=70 batteries has a mean life of \bar{x} = 40.5 hours with s = 1.75 Let \alpha = 0.05.

H_0: \mu = 40

H_a \mu > 40

t^* = {\bar{x} - \mu_0 \over {s \over \sqrt{n}}}

t^* = {40.5 - 40 \over {1.75 \over \sqrt{70}}} = 2.39

Find the p-value:

P(t_{\alpha \over 2} \ge |t^*|) = 0.0097

>>> from scipy.stats import t

>>> t = 2.39    # The t-score
>>> s = 70      # The sample size

>>> t.sf(abs(t), s)
0.009772027372500908

If in fact H_0 is true, the probability of observing a test statistic that is as extreme or more extreme than t^* = 2.39 is about 0.0097. That is to say, the sample is very unlikely to occur under H_0. Since the p-value is less than \alpha, H_0 is rejected.

Sample evidence suggests that the mean battery life of this particular brand exceeds 40 hours.

Type 1 Error

When H_0 is rejected despite it being true.

The probability that a type 1 error occurs is \alpha

Type 2 Error

When H_0 is not rejected despite it being false.

NOTE:

Our group of subjects should be representative of the entire population of interests.

Because we cannot impose an experiment on an entire population, we are often forced to examine a small, and we hope that the sample statistics, \bar{x} and s^2, are good estimates of the population parameters, \mu and \sigma^2.

Example

The effects of caffeine on the body have been well studied. In one experiment, a group of 20 male college students were trained in a particular tapping movement and to tap a rapid rate. They were randomly divided into caffeine and non-caffeine groups and given approximately 2 cups of coffee (Either 200[mg] of caffeine or decaf). After a two hour period, the tapping rate was measured.

The population of interest is male college-aged students.

The question of interest: is the mean tap rate of the caffeinated group different than that of the non-caffeinated group.

Let \mu_c be the mean of the caffeinated group, and \mu_d be the mean of the caffeinated group.

H_0: \mu_c = \mu_d

H_a: \mu_c \ne \mu_d

Two-Sample T-Test

To test:

H_0: \mu_1 = \mu_2

H_a: \mu_1 \ne \mu_2

Use the following statistic:

t^* = {(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2) \over s_p \sqrt{{1\over n_1} + {1\over n_2}}}

Where:

s_p^2 = {(n_1 -1)s_1^2 + (n_2 - 1)s_2^2 \over n_1 + n_2 -2}

Where t^* follows a t-distribution with n_1 + n_2 -2 degrees of freedom under H_0. Thus, the p-value is P(t_{n_1 + n_2 -2} \ge |t^*|) for a one sided test, and twice that for a two sided test.

Assumptions:

The two populations are independently normally distributed with the same variance.

Example

H_0: \mu_c = \mu_d

H_a: \mu_c = \mu_d

s_p^2 = {(n_1 -1)s_1^2 + (n_2 - 1)s_2^2 \over n_1 + n_2 -2}

s_p^2 = {(10 -1)(5.73) + (10 - 1)(4.9) \over 18} = 5.315

s_p = \sqrt{5.315}

Find the p-value:

2P(t_{n_1 + n_2 - 2} \ge |3.394|) = 0.00326

Since the p-value < \alpha, we reject H_0.

Sample evidence suggests that the mean tap rate for the caffeinated group is different than that for the con-caffeinated group.

Example

The thickness of a plastic film in mils on a substrate material is thought to be influenced by the temperature at which the coating is applied. A completely randomized experiment is carried out. 11 substrates are coated at 125$^\circ$F, resulting in sample mean coating thickness of \bar{x}_1 = 103.5, and sample standard deviation of s_1 = 10.2. Another 13 substrates are coated at 150$^\circ$F, where \bar{x}_2 = 99.7 and s_2 = 15.1. It is suspected that raising the process temperature would reduce the mean coating thickness. Does the data support this claim? Use \alpha = 0.01.

	125$^\circ$F	150$^\circ$F
\bar{x}	103.5	99.7
s	10.2	15.1
n	11	13

H_0: \mu_1 = \mu_2

H_a: \mu_1 < \mu_2

t^* = {(\bar{x}_1 - \bar{x}_2) \over s_p \sqrt{{1\over n_1} + {1\over n_2}}}

s_p^2 = {(11 - 1)(10.2)^2 + (13-1)(15.1)^2 \over 11 + 13 - 2} = 171.66

s_p = 13.1

t^* = {(99.7 - 103.5) \over 13.1 \sqrt{{1\over11} + {1\over13}}} = -0.71

Find the p-value:

P(t_{n_1 + n_2 - 2} > |-0.71|) = 0.243

Since the p-value is greater than \alpha, we fail to reject H_0. That is to say sample evidence does not suggest that raising the process temperature would reduce the mean coating thickness.

Practical vs. Statistical Significance

More samples is not always better.

• Waste of resources
• Statistical significance \ne practical significance

Example

Consider an SAT score improvement study.

$600 study plan: x_{11}, x_{12}, \cdots, x_{1n}

Traditional study plan: x_{21}, x_{22}, \cdots, x_{2n}

Test for H_0: \mu_1 = \mu_2

H_a: \mu_1 \ne \mu_2

Test statistic:

t^* = {\bar{x}_1 - \bar{x}_2 \over s_p \sqrt{{1\over n_1} + {1\over n_2}}}

Suppose that \mu_1 - \mu_2 = 1 point. When n \to \infty, \bar{x}_1 - \bar{x}_2 \xrightarrow{p} \mu_1 - \mu_2, s_p^2 \to \sigma^2 as n\to \infty.