Function Approximation

In [1]:
N = 1e3;
x = linspace(0, 10, N);
y = 2.0 * x - 5.0 + 3.0 * randn(1, N);
In [2]:
figure(1);
scatter(x, y);
hold on;
In [3]:
th = polyfit(x, y, 1);
yp = th(1) * x + th(2);

th

th =

   1.9808  -4.8830
In [4]:
scatter(x, y);
hold on;
plot(x, yp, 'LineWidth', 7);
In [5]:
% pick two points at random

x1 = x(345);
x2 = x(401);

y1 = y(345);
y2 = y(401);
In [6]:
%%latex

Solve

\[ m x_1 + b = y_1 \]
\[ m x_2 + b = y_2 \]

for $m$ and $b$, by forming the matrix equation

\[
\begin{bmatrix}
1 & x_1 \\
1 & x_2
\end{bmatrix} \begin{bmatrix}
b \\
m
\end{bmatrix} =  \begin{bmatrix}
y_1 \\
y_2
\end{bmatrix} \]
Solve \[ m x_1 + b = y_1 \] \[ m x_2 + b = y_2 \] for $m$ and $b$, by forming the matrix equation \[ \begin{bmatrix} 1 & x_1 \\ 1 & x_2 \end{bmatrix} \begin{bmatrix} b \\ m \end{bmatrix} = \begin{bmatrix} y_1 \\ y_2 \end{bmatrix} \]
In [7]:
A = [1, x1;
     1, x2];
c = [y1; 
     y2];
     
% A th = c
th = A \ c;
m = th(2)
b = th(1)

m =  4.3968
b = -9.9094
In [8]:
figure();
scatter(x, y);
hold on;
plot(x, m * x + b, 'LineWidth', 7);
hold on;
plot(x1, y1, 'MarkerSize', 40, 'Color', 'black');
plot(x2, y2, 'MarkerSize', 40, 'Color', 'black');
text(0.1, 25, 'Passes through two points, but is not a good data approximation', 'FontSize', 18);
In [9]:
%%latex

Solve the same equation for all values of $x_i$ and $y_i$
\[
\begin{bmatrix}
1 & x_1 \\
1 & x_2 \\
1 & x_3 \\
\vdots \\
1 & x_{n - 1} \\
1 & x_n
\end{bmatrix} \begin{bmatrix}
b \\
m
\end{bmatrix} = \begin{bmatrix}
y_1 \\
y_2 \\
y_3 \\
\vdots \\
y_{n - 1} \\
y_n
\end{bmatrix}
\]
Solve the same equation for all values of $x_i$ and $y_i$ \[ \begin{bmatrix} 1 & x_1 \\ 1 & x_2 \\ 1 & x_3 \\ \vdots \\ 1 & x_{n - 1} \\ 1 & x_n \end{bmatrix} \begin{bmatrix} b \\ m \end{bmatrix} = \begin{bmatrix} y_1 \\ y_2 \\ y_3 \\ \vdots \\ y_{n - 1} \\ y_n \end{bmatrix} \]
In [10]:
A = [ones(length(x), 1), x(:)];
c = zeros(length(x), 1);

% do x(:) to make sure vector is vertical

% A th = c
th = A \ y(:) 
b = th(1)
m = th(2)

th =

  -4.8830
   1.9808

b = -4.8830
m =  1.9808
In [11]:
figure()
scatter(x, y);
hold on;
plot(x, m * x + b, 'LineWidth', 7);

Approximating a quadratic

In [12]:
N = 1e3;
x = linspace(-1, 3, N);
p = [0.5, -2.0, -3.0];
y = p(1) * x.^2 + p(2) * x + p(3) + 0.5 * randn(1, N);
In [13]:
figure();
scatter(x, y);

Try solving

$\begin{bmatrix} 1 & x_1 & x_1^2 \\ 1 & x_2 & x_2^2 \\ 1 & x_3 & x_3^2 \\ \vdots \\ 1 & x_n & x_n^2 \end{bmatrix} \begin{bmatrix} p(3) \\ p(2) \\ p(1) \end{bmatrix} = \begin{bmatrix} y_1 \\ y_2 \\ y_3 \\ \vdots \\ y_n \end{bmatrix}$

In [14]:
A = [ones(length(x), 1), x(:), x(:).^2];

th = A \ y(:)

th =

  -3.01095
  -1.96906
   0.49583
In [15]:
yp = A * th;
figure();
scatter(x, y);
hold on;
plot(x, yp, 'LineWidth', 7);

Approximating sine

$f(x) = c \sin(x) + d$

In [16]:
c = 0.1;
d = 2.0;
N = 1e3;
x = linspace(0, 10, N);
y = c * sin(x) + d + 0.1 * rand(1, N);
In [17]:
figure();
scatter(x, y);

Try solving

$\begin{bmatrix} 1 & sin(x_1) \\ 1 & sin(x_2) \\ 1 & sin(x_3) \\ \vdots & \vdots \\ 1 & sin(x_n) \\ \end{bmatrix} \begin{bmatrix} \theta_1 \\ \theta_2 \\ \end{bmatrix} = \begin{bmatrix} y_1 \\ y_2 \\ y_3 \\ \vdots \\ y_n \\ \end{bmatrix}$

In [18]:
A = [ones(length(x), 1), sin(x(:)), x(:).^2];

% A th = y
th = A \ y(:)

th =

   2.0494e+00
   1.0071e-01
  -2.1672e-06
In [19]:
figure();
scatter(x, y);
hold on;
xn = linspace(-2, 12, 100);
An = [ones(length(xn), 1), sin(xn(:)), xn(:).^2];
yn = An * th;

plot(xn, yn, 'LineWidth', 7);