First-order Ordinary Differential Equations cheatsheet
By Afshine Amidi and Shervine Amidi
Introduction
Differential Equations A differential equation is an equation containing derivatives of a dependent variable $y$ with respect to independent variables $x$. In particular,
- Ordinary Differential Equations (ODE) are differential equations having one independent variable.
- Partial Differential Equations (PDE) are differential equations having two or more independent variables.
Order An ODE is said to be of order $n$ if the highest derivative of the unknown function in the equation is the $n^{th}$ derivative with respect to the independent variable.
Linearity An ODE is said to be linear only if the function $y$ and all of its derivatives appear by themselves. Thus, it is of the form:
Direction Field Method
Implicit form The implicit form of an ODE is where $y'$ is not separated from the remaining terms of the ODE. It is of the form:
Remark: Sometimes, $y'$ cannot be separated from the other terms and the implicit form is the only one that we can write.
Explicit form The explicit form of an ODE is where $y'$ is separated from the remaining terms of the ODE. It is of the form:
Direction field method The direction field method is a graphical representation for the solution of ODE $y'=f(x,y)$ without actually solving for $y(x)$. Here is the procedure:
Separation of Variables
Separable An ODE is said to be separable if it can be written in the form:
Reduction to separable form The following table sums up the variable changes that allow us to change the ODE $y'=f(x,y)$ to $u'=g(x,u)$ that is separable.
Original form | Change of variables | New form |
$y'=f\left(\frac{y}{x}\right)$ | $u\triangleq\frac{y}{x}$ | $u'x+u=f(u)$ |
$y'=f\left(ax+by+c\right)$ | $u\triangleq ax+by+c$ | $\frac{u'-a}{b}=f(u)$ |
Equilibrium
Characterization In order for an ODE to have equilibrium solutions, it must be (1) autonomous and (2) have a value $y^*$ that makes the derivative equal to 0, i.e:
Stability Equilibrium solutions can be classified into 3 categories:
- Unstable: solutions run away with any small change to the initial conditions.
- Stable: any small perturbation leads the solutions back to that solution.
- Semi-stable: a small perturbation is stable on one side and unstable on the other.
Linear first-order ODE technique
Standard form The standard form of a first-order linear ODE is expressed with $p(x), r(x)$ known functions of $x$, such that:
Remark: If $r=0$, then the ODE is homogenous, and if $r\neq0$, then the ODE is inhomogeneous.
General solution The general solution $y$ of the standard form can be decomposed into a homogenous part $y_h$ and a particular part $y_p$ and is expressed in terms of $p(x), r(x)$ such that:
Remark 1: Here, for any function $p$, the notation $\displaystyle\int pdx$ denotes the primitive of $p$ without additive constant.
Remark 2: The term $e^{-\int pdx}$ is called the basis of the ODE and $e^{\int pdx}$ is called the integrating factor.
Reduction to linear form The one-line table below sums up the change of variables that we apply in order to have a linear form:
Name, setting | Original form | Change of variables | New form |
Bernoulli, $n\in\mathbb{R}\backslash\{0,1\}$ | $y'+p(x)y=q(x)y^n$ | $u\triangleq y^{1-n}$ | $u'+(1-n)p(x)u=(1-n)q(x)$ |
Existence and uniqueness of an ODE
Here, we are given an ODE $y'=f(x,y)$ with initial conditions $y(x_0)=y_0$.
Existence theorem If $f(x,y)$ is continuous at all points in a rectangular region containing $(x_0,y_0)$, then $y'=f(x,y)$ has at least one solution $y(x)$ passing through $(x_0,y_0)$.
Remark: If the condition does not apply, then we cannot say anything about existence.
Uniqueness theorem If both $f(x,y)$ and $\frac{\partial f}{\partial y}(x,y)$ are continuous at all points in a rectangular region containing $(x_0,y_0)$, then $y'=f(x,y)$ has a unique solution $y(x)$ passing through $(x_0,y_0)$.
Remark: If the condition does not apply, then we cannot say anything about uniqueness.
Numerical methods for ODE - Initial value problems
In this section, we would like to find $y(t)$ for the interval $[0,t_f]$ that we divide into $N+1$ equally-spaced points $t_0< t_1 < ... < t_N = t_f$, such that:
Error In order to assess the accuracy of a numerical method, we define its local and global errors $\epsilon_{\textrm{local}}, \epsilon_{\textrm{global}}$ as follows:
Remark 1: If $\epsilon_{\textrm{local}}=O(h^k)$, then $\epsilon_{\textrm{global}}=O(h^{k-1})$.
Remark 2: When we talk about the 'error' of a method, we refer to its global error.
Taylor series The Taylor series giving the exact expression of $y_{n+1}$ in terms of $y_n$ and its derivatives is:
We can also have an expression of $y_n$ in terms of $y_{n+1}$ and its derivatives:
Stability The stability analysis of any ODE solver algorithm is performed on the model problem, defined by:
which gives $y_n=y_0\sigma^n$, for which $h$ verifies the condition $|\sigma(h)| < 1$.
Euler methods The Euler methods are numerical methods that aim at estimating the solution of an ODE:
Type | Update formula | Error | Stability condition |
Forward Euler | $y_{n+1}=y_n+hf(t_n,y_n)$ | $O(h)$ | $h<\frac{2}{|\lambda|}$ |
Backward Euler | $y_{n+1}=y_n+hf(t_{n+1},y_{n+1})$ | $O(h)$ | None |
Runge-Kutta methods The table below sums up the most commonly used Runge-Kutta methods:
Method | Update formula | Error | Stability condition | |
RK1 | Euler's | $y_{n+1}=y_n+hk_1$ where $k_1=f(t_n,y_n)$ |
$O(h)$ | $\displaystyle h<\frac{2}{|\lambda|}$ |
RK2 | Heun's | $y_{n+1}=y_n+h\left(\frac{1}{2}k_1+\frac{1}{2}k_2\right)$ where $k_1=f(t_n,y_n)$ and $k_2=f(t_n+h,y_n+hk_1)$ |
$O(h^2)$ | $\displaystyle h<\frac{2}{|\lambda|}$ |
RK3 | Kutta's | $y_{n+1}=y_n+h\left(\frac{1}{6}k_1+\frac{2}{3}k_2+\frac{1}{6}k_3\right)$ where $k_1=f(t_n,y_n)$ and $k_2=f(t_n+\frac{1}{2}h,y_n+\frac{1}{2}hk_1)$ and $k_3=f(t_n+h,y_n-hk_1+2hk_2)$ |
$O(h^3)$ | $\displaystyle h<\frac{2.5}{|\lambda|}$ |
RK4 | Classic | $y_{n+1}=y_n+h\left(\frac{1}{6}k_1+\frac{1}{3}k_2+\frac{1}{3}k_3+\frac{1}{6}k_4\right)$ where $k_1=f(t_n,y_n)$ and $k_2=f(t_n+\frac{1}{2}h,y_n+\frac{1}{2}hk_1)$ and $k_3=f(t_n+\frac{1}{2}h,y_n+\frac{1}{2}hk_2)$ and $k_4=f(t_n+h,y_n+hk_3)$ |
$O(h^4)$ | $\displaystyle h<\frac{2.8}{|\lambda|}$ |
System of Linear ODEs
Definition A system of $n$ first order linear ODEs
can be written in matrix form as:
where $A=\left(\begin{array}{ccc}a_{11}& \cdots& a_{1n}\\\vdots& \ddots & \vdots\\a_{n1}& \cdots& a_{nn}\end{array}\right)$ and $\vec{y}=\left(\begin{array}{c}y_1\\\vdots\\y_n\end{array}\right)$
Characteristic equation The characteristic equation of a linear system of $n$ equations represented by $A$ is given by:
For $n=2$, this equation can be written as:
Eigenvector, eigenvalue The roots $\lambda$ of the characteristic equation are the eigenvalues of $A$. The solutions $\vec{v}$ of the equation $A\vec{v}=\lambda I$ are called the eigenvectors associated with the eigenvalue $\lambda$.
System of homogeneous ODEs The resolution of the system of 2 homogeneous linear ODEs $\vec{y}'=A\vec{y}$ is detailed in the following table:
Case | Eigenvalues $\leftrightarrow$ Eigenvectors | Solution |
Real distinct eigenvalues | $\lambda_1\leftrightarrow\vec{\eta}_{\lambda_1}$ $\lambda_2\leftrightarrow\vec{\eta}_{\lambda_2}$ |
$\vec{y}=C_1\vec{\eta}_{\lambda_1}e^{\lambda_1t}+C_2\vec{\eta}_{\lambda_2}e^{\lambda_2t}$ |
Double root eigenvalues | $\lambda\leftrightarrow\vec{\eta}$ $\vec{\rho}\textrm{ s.t. }(A-\lambda I)\vec{\rho}=\vec{\eta}$ |
$\vec{y}=[(C_1+C_2t)\vec{\eta}+C_2\vec{\rho}]e^{\lambda t}$ |
Complex conjugate eigenvalues | $\alpha+i\beta\leftrightarrow\vec{\eta}_{R}+i\vec{\eta}_{I}$ $\alpha-i\beta\leftrightarrow\vec{\eta}_{R}-i\vec{\eta}_{I}$ |
$\vec{y}=C_1(\cos(\beta t)\vec{\eta}_R-\sin(\beta t)\vec{\eta}_I)e^{\alpha t}$ $\quad+ C_2(\cos(\beta t)\vec{\eta}_I+\sin(\beta t)\vec{\eta}_R)e^{\alpha t}$ |