Levenberg Marquardt

In fitting a function $\hat{y}(\beta)$ of an independent variable $ t $ and a vector of n parameters $ \beta $ to a set of m data points $(t_i, y_i)$, it is customary and convenient to minimize the sum of the weighted squares of the errors (or weighted residuals) between the measured data $ y_i $ and thecurve-fit function $ \hat{y}(\beta) $.

Least square error $$ \beta_{opt} = \underset{\beta}{\arg \min} \sum_{i=1}^{m}(y(\beta) - \hat{y}(\beta))^2 = (y - \hat{y})^T(y - \hat{y}) $$

For a small $\delta_{\beta}$, a Taylor espansion series leads to the approximation

$$ f(\beta + \delta_{\beta}) \approx f(\beta) + J\delta_{\beta} $$

where $J$ is the Jacobian Matrix $J = \frac{\partial f(\beta)}{\partial \beta}$