Machine Learning Cheat Sheet Part 2 - Linear Regression with One Variable

1. Training set: $ (x^{(1)}, y^{(1)}), (x^{(2)}, y^{(2)}), ... (x^{(m)}, y^{(m)}) $

2. Hypothesis: $ h_\theta(x)=\theta_0+\theta_1x $

3. Parameters: $ \theta_0, \theta_1 $

4. Cost function: $J(\theta_0, \theta_1)$ uses parameters $\theta_0$ and $\theta_1$ to check the difference between hypothesis values $h_\theta(x)$ and given values $y$ from training example $(x,y)$:

\[ J(\theta_0, \theta_1) = \frac{1}{2m} \sum_{i=1}^{m} (h_\theta(x^{(i)}) - y^{(i)})^2 \]
5. Goal: minimise cost function

$ min_{\theta_0, \theta_1} J(\theta_0, \theta_1) $

6. Gradient descent algorithm (minimisation of cost function):

$ \alpha $ - learning rate

repeat until convergence:
\[
{
\{
\\
 \theta_0 = \theta_0 - \alpha \frac{1}{m} \sum_{i=1}^{m} (h_0(x^{(i)} - y^{(i)})
\\
\theta_1 = \theta_1 - \alpha \frac{1}{m} \sum_{i=1}^{m} (h_0(x^{(i)} - y^{(i)})\times x^{(i)}
\\
\}
}
\]
(update $ \theta_0 $ and $ \theta_1 $ simultaneously!)