Proof box 2.1

[Gregwar]

Hello, first thank you for this work!

Sorry if I misunderstood something, but in proof box 2.1 of page 39, it is stated that:

Since every entry of $P_{\pi}$ is nonnegative and no greater than one, we have that $0 \leq P_{\pi}^k \leq 1$ for any $k$.

This would be true if $P_{\pi}^k$ were an elementwise exponent, but it appears to be a matrix exponent. For instance:

$$ P_{\pi} = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix} $$

(Which is of course not a proper stochastic matrix), satisfies the "every entry is nonnegative and no greater than one" condition, but:

$$ P_{\pi}^2 = \begin{bmatrix} 2 & 2 \\ 2 & 2 \end{bmatrix} $$

Clearly does not satisfy $0 \leq P_{\pi}^k \leq 1$.

---

Shouldn't the argument be based on $P_{\pi}$ being a stochastic matrix with a spectral radius of one instead ?

[MathFoundationRL]

Yes, this sentence can be stated in a more rigorous way.