Machine Learning Cheat Sheet Part 2 - Linear Regression with One Variable

1. Training set: $(x^{(1)}, y^{(1)}), (x^{(2)}, y^{(2)}), ... (x^{(m)}, y^{(m)})$

2. Hypothesis: $h_\theta(x)=\theta_0+\theta_1x$

3. Parameters: $\theta_0, \theta_1$

4. Cost function: $J(\theta_0, \theta_1)$ uses parameters $\theta_0$ and $\theta_1$ to check the difference between hypothesis values $h_\theta(x)$ and given values $y$ from training example $(x,y)$:

$$J(\theta_0, \theta_1) = \frac{1}{2m} \sum_{i=1}^{m} (h_\theta(x^{(i)}) - y^{(i)})^2$$
5. Goal: minimise cost function

$min_{\theta_0, \theta_1} J(\theta_0, \theta_1)$

6. Gradient descent algorithm (minimisation of cost function):

$\alpha$ - learning rate

repeat until convergence:
$${ \{ \\  \theta_0 = \theta_0 - \alpha \frac{1}{m} \sum_{i=1}^{m} (h_0(x^{(i)} - y^{(i)}) \\ \theta_1 = \theta_1 - \alpha \frac{1}{m} \sum_{i=1}^{m} (h_0(x^{(i)} - y^{(i)})\times x^{(i)} \\ \} }$$
(update $\theta_0$ and $\theta_1$ simultaneously!)