10-3-2024 - Curve Fitting

$$f(\vec{x})=a_1{f}_1(\vec{x})+a_2{f}_2(\vec{x})+\cdots +a_k{f}_k(\vec{x})$$ $$({\vec{x}}_1,{\vec{y}}_1),\dots ,({\vec{x}}_m,{\vec{y}}_m)$$

Find best fit

$$\underset{{\mathbf{A}}_{m\times k}}{\underbrace{\left[f_j({\vec{x}}_i)\right]}}\underset{{\hat{x}}_{k\times 1}}{\underbrace{\left[\begin{array}{c}{a}_1\\ ⋮\\ a_k\end{array}\right]}}=\underset{{\vec{b}}_{m\times 1}}{\underbrace{\left[\begin{array}{c}{y}_1\\ ⋮\\ y_k\end{array}\right]}}$$ $$\begin{array}{r}\mathbf{A}\hat{\mathbf{x}}=\vec{\mathbf{b}}\\ {\mathbf{A}}^T\mathbf{A}\hat{\mathbf{x}}={\mathbf{A}}^T\vec{\mathbf{b}}\\ \Vert \mathbf{A}\hat{\mathbf{x}}-\vec{\mathbf{b}}\Vert \text{ Minimum possible error}\\ \hat{\mathbf{x}}=({\mathbf{A}}^T\mathbf{A}{)}^{-1}{\mathbf{A}}^T\vec{\mathbf{b}}\end{array}$$

Do Gaussian elimination of $\mathbf{A}$ to go to RREF if not invertible

Since these are high-rank matrices, inverting them is very very hard - so it’s useful to have techniques to efficiently compute these things “pseudo-inverse” problems - low-rank approximation of inverse - SVD turns out to be very good at solving these problems, by taking the top few singular values

• If you just invert that part, you get a pseudo-inverse which is a good enough approximation