First-order Ordinary Differential Equations cheatsheet
Star

By Afshine Amidi and Shervine Amidi

Introduction

Differential Equations A differential equation is an equation containing derivatives of a dependent variable $y$ with respect to independent variables $x$. In particular,

- Ordinary Differential Equations (ODE) are differential equations having one independent variable.

- Partial Differential Equations (PDE) are differential equations having two or more independent variables.


Order An ODE is said to be of order $n$ if the highest derivative of the unknown function in the equation is the $n^{th}$ derivative with respect to the independent variable.


Linearity An ODE is said to be linear only if the function $y$ and all of its derivatives appear by themselves. Thus, it is of the form:

\[\boxed{a_n(x)y^{(n)}+a_{n-1}(x)y^{(n-1)}+...+a_1(x)y'+a_0(x)y+b(x)=0}\]

Direction Field Method

Implicit form The implicit form of an ODE is where $y'$ is not separated from the remaining terms of the ODE. It is of the form:

\[\boxed{F(x,y,y')=0}\]

Remark: Sometimes, $y'$ cannot be separated from the other terms and the implicit form is the only one that we can write.


Explicit form The explicit form of an ODE is where $y'$ is separated from the remaining terms of the ODE. It is of the form:

\[\boxed{y'=f(x,y)}\]

Direction field method The direction field method is a graphical representation for the solution of ODE $y'=f(x,y)$ without actually solving for $y(x)$. Here is the procedure:

1. Determine the values $(x_i,y_i)$ that form the grid.

2. Compute the slope $f(x_i,y_i)$ for each point of the grid.

3. Report the associated vector for each point of the grid.


Separation of Variables

Separable An ODE is said to be separable if it can be written in the form:

\[\boxed{f(x,y)=g(x)h(y)}\]

Reduction to separable form The following table sums up the variable changes that allow us to change the ODE $y'=f(x,y)$ to $u'=g(x,u)$ that is separable.

Original form Change of variables New form
$y'=f\left(\frac{y}{x}\right)$ $u\triangleq\frac{y}{x}$ $u'x+u=f(u)$
$y'=f\left(ax+by+c\right)$ $u\triangleq ax+by+c$ $\frac{u'-a}{b}=f(u)$

Equilibrium

Characterization In order for an ODE to have equilibrium solutions, it must be (1) autonomous and (2) have a value $y^*$ that makes the derivative equal to 0, i.e:

\[(1)\quad\boxed{\frac{dy}{dt}=f(\require{cancel} \xcancel{t},y)=f(y)}\quad\textrm{ and }\quad(2)\quad\boxed{\exists\textrm{ }y^*, \frac{dy^*}{dt} = f(y^*)=0}\]

Stability Equilibrium solutions can be classified into 3 categories:

- Unstable: solutions run away with any small change to the initial conditions.
- Stable: any small perturbation leads the solutions back to that solution.
- Semi-stable: a small perturbation is stable on one side and unstable on the other.


Linear first-order ODE technique

Standard form The standard form of a first-order linear ODE is expressed with $p(x), r(x)$ known functions of $x$, such that:

\[\boxed{y'+p(x)y=r(x)}\]

Remark: If $r=0$, then the ODE is homogenous, and if $r\neq0$, then the ODE is inhomogeneous.


General solution The general solution $y$ of the standard form can be decomposed into a homogenous part $y_h$ and a particular part $y_p$ and is expressed in terms of $p(x), r(x)$ such that:

\[\boxed{y=y_h+y_p}\quad\quad\textrm{with}\quad\quad\boxed{y_h=Ce^{-\int pdx}}\quad\textrm{and}\quad\boxed{y_p=e^{-\int pdx}\times\int\left[r e^{\int pdx}\right]dx}\]

Remark 1: Here, for any function $p$, the notation $\displaystyle\int pdx$ denotes the primitive of $p$ without additive constant.

Remark 2: The term $e^{-\int pdx}$ is called the basis of the ODE and $e^{\int pdx}$ is called the integrating factor.


Reduction to linear form The one-line table below sums up the change of variables that we apply in order to have a linear form:

Name, setting Original form Change of variables New form
Bernoulli, $n\in\mathbb{R}\backslash\{0,1\}$ $y'+p(x)y=q(x)y^n$ $u\triangleq y^{1-n}$ $u'+(1-n)p(x)u=(1-n)q(x)$

Existence and uniqueness of an ODE

Here, we are given an ODE $y'=f(x,y)$ with initial conditions $y(x_0)=y_0$.

Existence theorem If $f(x,y)$ is continuous at all points in a rectangular region containing $(x_0,y_0)$, then $y'=f(x,y)$ has at least one solution $y(x)$ passing through $(x_0,y_0)$.

Remark: If the condition does not apply, then we cannot say anything about existence.


Uniqueness theorem If both $f(x,y)$ and $\frac{\partial f}{\partial y}(x,y)$ are continuous at all points in a rectangular region containing $(x_0,y_0)$, then $y'=f(x,y)$ has a unique solution $y(x)$ passing through $(x_0,y_0)$.

Remark: If the condition does not apply, then we cannot say anything about uniqueness.


Numerical methods for ODE - Initial value problems

In this section, we would like to find $y(t)$ for the interval $[0,t_f]$ that we divide into $N+1$ equally-spaced points $t_0< t_1 < ... < t_N = t_f$, such that:

\[\frac{dy}{dt}=f(t,y)\quad\textrm{with}\quad y(0)=y_0\]

Error In order to assess the accuracy of a numerical method, we define its local and global errors $\epsilon_{\textrm{local}}, \epsilon_{\textrm{global}}$ as follows:

\[\boxed{\epsilon_{\textrm{local}}=|y^{\textrm{exact}}(t_n)-y^{\textrm{numerical}}(t_n)|}\quad\textrm{and}\quad\boxed{\epsilon_{\textrm{global}}=\sqrt{\frac{\displaystyle\sum_{n=1}^N|y^{\textrm{exact}}(t_n)-y^{\textrm{numerical}}(t_n)|^2}{N}}}\]

Remark 1: If $\epsilon_{\textrm{local}}=O(h^k)$, then $\epsilon_{\textrm{global}}=O(h^{k-1})$.

Remark 2: When we talk about the 'error' of a method, we refer to its global error.


Taylor series The Taylor series giving the exact expression of $y_{n+1}$ in terms of $y_n$ and its derivatives is:

\[\boxed{y_{n+1}=y_n+hy_n'+\frac{h^2}{2}y_n''+\frac{h^3}{6}y_n'''+...=\sum_{k=0}^{+\infty}\frac{h^k}{k!}y_n^{(k)}}\]

We can also have an expression of $y_n$ in terms of $y_{n+1}$ and its derivatives:

\[\boxed{y_{n}=y_{n+1}-hy_{n+1}'+\frac{h^2}{2}y_{n+1}''-\frac{h^3}{6}y_{n+1}'''+...=\sum_{k=0}^{+\infty}\frac{(-h)^k}{k!}y_{n+1}^{(k)}}\]

Stability The stability analysis of any ODE solver algorithm is performed on the model problem, defined by:

\[\boxed{y'=\lambda y\quad\textrm{ with }\quad y(0)=y_0\quad\textrm{and}\quad\lambda < 0}\]

which gives $y_n=y_0\sigma^n$, for which $h$ verifies the condition $|\sigma(h)| < 1$.


Euler methods The Euler methods are numerical methods that aim at estimating the solution of an ODE:


Type Update formula Error Stability condition
Forward Euler $y_{n+1}=y_n+hf(t_n,y_n)$ $O(h)$ $h<\frac{2}{|\lambda|}$
Backward Euler $y_{n+1}=y_n+hf(t_{n+1},y_{n+1})$ $O(h)$ None

Runge-Kutta methods The table below sums up the most commonly used Runge-Kutta methods:


Method Update formula Error Stability condition
RK1 Euler's $y_{n+1}=y_n+hk_1$
where $k_1=f(t_n,y_n)$
$O(h)$ $\displaystyle h<\frac{2}{|\lambda|}$
RK2 Heun's $y_{n+1}=y_n+h\left(\frac{1}{2}k_1+\frac{1}{2}k_2\right)$
where $k_1=f(t_n,y_n)$

and $k_2=f(t_n+h,y_n+hk_1)$
$O(h^2)$ $\displaystyle h<\frac{2}{|\lambda|}$
RK3 Kutta's $y_{n+1}=y_n+h\left(\frac{1}{6}k_1+\frac{2}{3}k_2+\frac{1}{6}k_3\right)$
where $k_1=f(t_n,y_n)$

and $k_2=f(t_n+\frac{1}{2}h,y_n+\frac{1}{2}hk_1)$

and $k_3=f(t_n+h,y_n-hk_1+2hk_2)$
$O(h^3)$ $\displaystyle h<\frac{2.5}{|\lambda|}$
RK4 Classic $y_{n+1}=y_n+h\left(\frac{1}{6}k_1+\frac{1}{3}k_2+\frac{1}{3}k_3+\frac{1}{6}k_4\right)$
where $k_1=f(t_n,y_n)$

and $k_2=f(t_n+\frac{1}{2}h,y_n+\frac{1}{2}hk_1)$

and $k_3=f(t_n+\frac{1}{2}h,y_n+\frac{1}{2}hk_2)$

and $k_4=f(t_n+h,y_n+hk_3)$
$O(h^4)$ $\displaystyle h<\frac{2.8}{|\lambda|}$

System of Linear ODEs

Definition A system of $n$ first order linear ODEs

\[\begin{cases} y_1'=a_{11}y_1+...+a_{1n}y_n\\ \vdots\\ y_n'=a_{n1}y_1+...+a_{nn}y_n\\ \end{cases}\]

can be written in matrix form as:

\[\boxed{\vec{y}'=A\vec{y}}\]

where $A=\left(\begin{array}{ccc}a_{11}& \cdots& a_{1n}\\\vdots& \ddots & \vdots\\a_{n1}& \cdots& a_{nn}\end{array}\right)$ and $\vec{y}=\left(\begin{array}{c}y_1\\\vdots\\y_n\end{array}\right)$


Characteristic equation The characteristic equation of a linear system of $n$ equations represented by $A$ is given by:

\[\boxed{\textrm{det}(A-\lambda I)=0}\]

For $n=2$, this equation can be written as:

\[\boxed{\lambda^2-(a_{11}+a_{22})\lambda+(a_{11}a_{22}-a_{12}a_{21})=0}\]

Eigenvector, eigenvalue The roots $\lambda$ of the characteristic equation are the eigenvalues of $A$. The solutions $\vec{v}$ of the equation $A\vec{v}=\lambda I$ are called the eigenvectors associated with the eigenvalue $\lambda$.


System of homogeneous ODEs The resolution of the system of 2 homogeneous linear ODEs $\vec{y}'=A\vec{y}$ is detailed in the following table:

Case Eigenvalues $\leftrightarrow$ Eigenvectors Solution
Real distinct eigenvalues $\lambda_1\leftrightarrow\vec{\eta}_{\lambda_1}$
$\lambda_2\leftrightarrow\vec{\eta}_{\lambda_2}$
$\vec{y}=C_1\vec{\eta}_{\lambda_1}e^{\lambda_1t}+C_2\vec{\eta}_{\lambda_2}e^{\lambda_2t}$
Double root eigenvalues $\lambda\leftrightarrow\vec{\eta}$
$\vec{\rho}\textrm{ s.t. }(A-\lambda I)\vec{\rho}=\vec{\eta}$
$\vec{y}=[(C_1+C_2t)\vec{\eta}+C_2\vec{\rho}]e^{\lambda t}$
Complex conjugate eigenvalues $\alpha+i\beta\leftrightarrow\vec{\eta}_{R}+i\vec{\eta}_{I}$
$\alpha-i\beta\leftrightarrow\vec{\eta}_{R}-i\vec{\eta}_{I}$
$\vec{y}=C_1(\cos(\beta t)\vec{\eta}_R-\sin(\beta t)\vec{\eta}_I)e^{\alpha t}$
$\quad+ C_2(\cos(\beta t)\vec{\eta}_I+\sin(\beta t)\vec{\eta}_R)e^{\alpha t}$