DES - Steady-State Analysis

Steady-State Analysis:

In transient analysis we compute the probabilities:

$${\Pi }_i(t)\triangleq P(X(i)=i)$$

Next we will address the computation of the limits of these probabilities for $t\to \infty$ :

$${\Pi }_i\triangleq \underset{t\to +\infty }{\lim }{\Pi }_i(t)$$

This is the topic of steady-state analysis

Motivation

Drop the dependence of state probabilities on the time $t$, and work with constant probabilities (the limits) instead of time-varying ones.

---

Classification of the States

For the purpose of steady-state analysis, we first need to introduce the classification of the states of a CTHMC.

---

Recurrent Time of State $i$ ($T_{i,i}$):

Then, we introduce the following random variable $T_{i,i}$

$T_{i,i}$ is called recurrent time of state $i$. $T_{i,i}$ is a continuous random variable, described by:

$${\Phi }_i(t)\triangleq P(T_{i,i}\le t),t\ge 0$$

---

Limit of the CDF: $p_i$

Let:

$$p_i\triangleq \underset{t\to \infty }{\lim }{\Phi }_i(t)$$

Where $p_i$ is the probability that the system returns to the state $i$ in the future. $\Rightarrow$ $1-p_i$ is the probability that the system does not return to the state $i$ in the future.

It turns out that:

• If $p_i=1$, state $i$ is recurrent
• If $p_i<1$, state $i$ is transient

Now assume $p_i=1\Rightarrow {\Phi }_i(t)$ is the cdf of $T_{i,i}$.

Let:

$${\phi }_i(t)\triangleq \frac{d}{dt}{\Phi }_i(t)$$

Then: ${\phi }_i$ is the pdf of $T_{i,i}$

---

Expected Value of $T_{i,i}$:

Now if we compute expected value of $T_{i,i}$:

$$M_i\triangleq {\int }_0^{+\infty }t{\phi }_i(t)dt$$

• If $M_i<+\infty$, state $i$ is positive recurrent
• If $M_i=+\infty$, state $i$ is null recurrent

Where for positive recurrent state $i$, we expect ${\Pi }_i>0$ While for null recurrent state $i$, we expect ${\Pi }_i=0$

---

~Example:

• If $\lambda >\mu$ then the state number goes to infinity as time goes on.
• If $\lambda <\mu$ then the most visited state is the first, the second has been visited less the the first, and so on, $\Pi$ assume defined values.
• If $\lambda =\mu$ as the time increases at infinity all the states could have been visited the same number of times, $\Pi$ is a vector of all zeros.

---

Results:

• If $\mathcal{X}$ is finite, then at least one state is recurrent.
• If $i\in \mathcal{X}$ is positive recurrent, and $j\in \mathcal{X}$ is reachable from $i$, then also $j$ is positive recurrent.
• If $S\subseteq \mathcal{X}$ is closed and irreducible, then one of the following holds:
  • all states in $S$ are positive recurrent
  • all states in $S$ are null recurrent
  • all states in $S$ are transient

This means that it is sufficient to classify a single state from the system, then the others are classified accordingly.

• If $S\subseteq \mathcal{X}$ is closed and irreducible and finite, then all states in $S$ are positive recurrent

---