2.2 Regression and Correlation

• What does simple linear regression model?

  Relationship between two variables
• In the formula $y =$$a +$$bx$, $a$ represents the y-intercept
• What is the primary goal of the least squares method?

  Minimize squared residuals
• The least squares method ensures the regression line is as close as possible to all data points.
• Under certain conditions, the least squares method yields unbiased estimators
• What does $b$ represent in the formula $y =$$a +$$bx$?


  Slope
• The least squares method calculates values of $a$ and $b$ in $y =$$a +$$bx$.
• Steps for using the least squares method

  1️⃣ Define the model y = a + bx</latex>

  2️⃣ Calculate the sum of squared residuals

  3️⃣ Minimize the sum of squared residuals

  4️⃣ Find the values of $a$ and $b$
• What is a key benefit of the least squares method in regression analysis?

  Clear selection criterion
• The least squares method is easily implemented using standard statistical software
• The least squares method always yields unbiased estimators for $a$ and $b$.


  False
• What is the formula to calculate the slope $b$ in simple linear regression?


  $b =$$\frac{\sum (x_{i} - \bar{x})(y_{i} - \bar{y})}{\sum (x_{i} - \bar{x})^{2}}$
• In the formula a = \bar{y} - b\bar{x}</latex>, $\bar{y}$ represents the mean of the response variable.
• The mean of the x-values in the example dataset is 2.5.
• Which hypothesis test is used to determine if the slope $b$ is significantly different from zero?


  t-test
• The standard error of $b$ is calculated using the sum of squared residuals.
• What is the alternative hypothesis for testing the significance of the slope in regression analysis?

  $H_{1}: b \neq 0$
• A t-test is used to compare the estimated slope to zero
• The standard error of the slope SE(b)</latex> measures the variability of the estimated slope $b$ around zero.
• How is the standard error of the slope $SE(b)$ calculated?


  $\sqrt{\frac{\sum (y_{i} - \hat{y}_{i})^{2}}{(n - 2) \sum (x_{i} - \bar{x})^{2}}}$
• When testing the significance of the regression, the null hypothesis is that the slope is equal to zero
• Steps to test the significance of a regression

  1️⃣ Calculate the slope $b$

  2️⃣ Calculate the standard error $SE(b)$

  3️⃣ Compute the t-statistic

  4️⃣ Find the p-value

  5️⃣ Compare the p-value to $\alpha$
• If $p < \alpha$, we reject the null hypothesis and conclude that the relationship is statistically significant.
• In an example where $t \approx 4.95$ with 2 degrees of freedom, is the relationship statistically significant if $\alpha =$$0.05$?


  Yes
• Simple linear regression models the relationship between a response variable and a single explanatory variable.
• What does the slope $b$ represent in the formula $y =$$a +$$bx$?


  The change in $y$ for a unit change in $x$
• The least squares method minimizes the sum of the squared residuals between observed and predicted values.
• The least squares method provides a well-defined best-fit line by minimizing the sum of squared residuals.
• Under what conditions does the least squares method yield unbiased estimators for $a$ and $b$?


  Certain conditions
• What is the least squares method used for?

  Finding the best-fit line
• The least squares method minimizes the sum of the squared residuals
• The least squares method yields unbiased estimators for $a$ and $b$ under certain conditions.
• What are the formulas to calculate the regression coefficients $a$ and $b$?


  b = \frac{\sum (x_{i} - \bar{x})(y_{i} - \bar{y})}{\sum (x_{i} - \bar{x})^{2}}</latex>, $a =$$\bar{y} - b\bar{x}$
• In the regression coefficient formulas, $x_{i}$ and $y_{i}$ represent individual data points
• Steps to calculate regression coefficients

  1️⃣ Calculate the means $\bar{x}$ and $\bar{y}$

  2️⃣ Calculate the covariance $\sum (x_{i} - \bar{x})(y_{i} - \bar{y})$

  3️⃣ Calculate the sum of squared differences for $x$: $\sum (x_{i} - \bar{x})^{2}$

  4️⃣ Calculate the slope $b$

  5️⃣ Calculate the y-intercept $a$
• For the example dataset, the mean of $x$ is $\bar{x} =$$2.5$ and the mean of y</latex> is $\bar{y} =$$4$.
• What is the resulting regression equation for the example dataset?

  $y =$$0.5 +$$1.4x$
• The standard error of $b$ is calculated using the formula $SE(b) =$$\sqrt{\frac{\sum (y_{i} - \hat{y}_{i})^{2}}{(n - 2) \sum (x_{i} - \bar{x})^{2}}}$.
• Steps to test the significance of the regression

  1️⃣ Calculate the slope $b$

  2️⃣ Calculate the standard error $SE(b)$

  3️⃣ Compute the t-statistic

  4️⃣ Find the p-value corresponding to the t-statistic

  5️⃣ Compare the p-value to the significance level $\alpha$
• What is the t-statistic for the example dataset used in the significance test?

  t \approx 4.95</latex>