SI&DA - System Identification - Mind Map

System Identification

Main Elements of System Identification

• Data Set $Z^N$
• Model Class $\mathcal{M}$
• Cost Function $J$
• Validate the model

Parameter Vector $\theta$

A Priori Information $\Theta$

Gray Box / Black Box Models

Identification Process Algorithm

Principle of Parsimony

---

Parametric System Identification

Monic Polynomial

Generic Model Class

$G(z):=\frac{B(z)}{F(z)}$ $H(z):=\frac{C(z)}{D(z)}$ Also another possible parameter to be estimated is the delay $r$.

ARX (Auto Regressive eXoguenous) Model

FIR (Finite Impulse Response) Model

ARMAX (Auto Regressive Moving Average) Model

OE (Output Error) Model

BJ (Box Jenkins) Model

Meaning of Each Acronym

Auto Regressive

The output $\bar{y}$ in the system model is of this form: $y(t)+a_{1}y(t-1)+\dots +a_{n_a}y(t-n_a)$

Exogenous Inputs:

The input $\bar{u}$ in the system model is of this form: $b_{1}u(t-1)+b_{2}u(t-2)+\dots +b_{n_b}u(t-n_b)$

First Input Response

The output $\bar{y}$ in the system model is of this form: $y(t)$

Moving Average

The noise $\bar{e}$ in the system model is of this form: $e(t)+c_{1}e(t-1)+\dots +c_{n_c}e(t-n_c)$

Output Error

The error $e(t)$ is not filtered.

Box-Jenkins

Name of the first people to use this model, is the most complete model of all the ones seen up till now.

---

Choosing the Best Model

$\epsilon (t,\theta )$ : Calculation

$$\begin{array}{llc}& \text{Starting from:}& y(t)=G(z)u(t)+v(t)\\ & \text{Arrive at:}& \epsilon (t,\theta )=H^{-1}(z)y(t)-H^{-1}(z)G(z)u(t)\end{array}$$

Regressor Vector

A vector containing all the past information $t\in \left[-\infty ,t-1\right]$ of $y(t),u(t),\epsilon (t)$; if any.

Error Function - ARX Model

We define the Regressor Vector as: So that we can write: Note how the error function is linear in $\theta$.

Error Function - ARMAX Model

$$\begin{array}{llc}& \text{Starting from:}& \hat{y}(t|t-1)=\left[1-\frac{A(z)}{C(z)}\right]y(t)+\frac{B(z)}{C(z)}u(t)\\ & \text{Arrive at:}& \epsilon (t)=\frac{A(z)}{C(z)}y(t)+\frac{B(z)}{C(z)}u(t)\end{array}$$

We also can find that:

$$\hat{y}(t|t-1)=\left[1-A(z)\right]y(t)+B(z)u(t)+\left[C(z)-1\right]\epsilon (t)$$

So: Note: $\epsilon (t,\theta )$ is non-linear in $\theta$, ==the pseudo-regressor $\phi$ contains past values of $\epsilon$, which depend on $\theta$.==

Error Function - OE Model

$$\begin{array}{llc}& \text{Starting from:}& y(t)=w(t)+e(t)\\ & \text{Arrive at:}& \hat{y}(t|t-1)=w(t)\end{array}$$

Since $w(t)=\hat{y}(t|t-1)$, the pseudo-regressor contains the terms $\hat{y}(t-1|t-2),\hat{y}(t-2|t-3),\dots ,\hat{y}(t-n_f|t-n_f-1)$, each of this prediction depends on $\theta$, so the error function $\epsilon (t,\theta )$ is non-linear in $\theta$.

Paradox of the Optimal Prediction

In the OE model, since $w(t)=\hat{y}(t|t-1)$, the pseudo-regressor contains the terms $\hat{y}(t-1|t-2),\hat{y}(t-2|t-3),\dots ,\hat{y}(t-n_f|t-n_f-1)$ so we theoretically need all the estimates from $t=-\infty$. Similar with ARMAX model, the pseudo-regressor contains $\epsilon$ values from $t-n_c$ , where $\epsilon (t-n_c,\theta )$ depends on $y$ starting from $y(t-n_c-n_a)$, which recursively depends on $\epsilon (t-n_c-n_a-n_c)$ and so on.

To solve this “paradox” we consider all output and prediction before $t_0$ equal to $0$.

Prediction Error

Linear: ARM Models

Non-Linear: ARMAX, OE, BJ Models

---

Cost Function

Prediction Error Filter:

Cost Function - ARX Model

$$\begin{array}{llc}& \text{Starting from:}& V_N\left(\theta ,Z^N\right)=\frac{1}{2N}{\sum }_{t=1}^N{\epsilon }^2(t,\theta )\\ & \text{Arrive at:}& {\hat{\theta }}^{\ast }=R^{-1}(N)\cdot \frac{1}{N}{\sum }_{t=1}^N\phi (t)y(t)\end{array}$$

Cost Function - ARMAX, OE, BJ Models

The ARMAX, OE, BJ Models are non-linear in $\theta$, so:

---

Preliminary Analysis of Data

Compare Different Model Classes / K-Step Ahead Prediction

Overfitting

Increase the Cost according to the Number of Parameters

Types of $U(d)$

• AIC : Akaike Information Criteria
• FPE : Final Prediction Error
• MDL : Minimum Description Length.

Change the Cost Function for Validation

---

Differentiate the Data Sets: Estimation and Validation

Usually the entire data set $Z^N$ it’s split in half.

Residual Analysis

Estimate the covariance function between the prediction error $\epsilon$ and the input $u$ and the variance of the prediction error $\epsilon$. Both of them have to be small $\forall \tau$ because we don’t want the prediction error to have any relationship from the input or past residuals.

Whiteness Test of the Residuals

Input Selection

Things to remember when choosing an input for the system identification:

Typical Noises:

• White Noise
• RBS : Random Binary Sequence
• PRBS : Pseudo Random Binary Sequence
• Sum of Sinusoids
• Chirp