I recently went through Kalman's paper "When is a Linear Control System Optimal" published in 1964. The paper makes me wonder whether the following statement is true: Any stabilizing control law is optimal for some LQR problems.
For the linear system $$ x_{t+1} = Ax_t + Bu_t, $$ and the average of a quadratic cost function \begin{equation} J(\pi) = \lim_{T\rightarrow\infty} \sum_{t=0}^{T-1}\left(x_t'Qx_t + u_t'Ru_t \right), \end{equation} where $Q$ is positive semi-definite and $R$ is positive definite, $(A,B)$ is controllable, and $(Q^{1/2},A)$ is observable, we know the policy $K$ such that $\pi(x) = K x$ is optimal for the cost $J(\pi)$ when $$ K = -(R+B'PB)^{-1}B'PA, \tag{a} $$ where $P$ is the unique positive definite solution of the algebraic Riccati equation $$ P = Q + A'PA- A'PB(R+B'PB)^{-1}B'PA \tag{b}. $$ We say $K$ is optimal for the LQR problem defined by $Q\succeq 0$ and $R\succ 0$ if conditions (a) and (b) are satisfied.
Or equivalently, we can say $K$ is optimal for $Q$ and $R$ if $$ \begin{aligned} Q + A'P(A+BK) - P =0,\\ RK+ B'P(A+BK)=0,\\ P\succeq0,\ \ Q\succeq 0, R\succ0, \end{aligned} \tag{c} $$ are satisfied.
The statement is that for any stabilizing $K$, there always exists some $Q\succeq 0$ and $R\succ 0$ such that conditions (a) and (b) or conditions (c) are satisfied (Any stabilizing control law is optimal for some LQR problems.)
I want to show that the statement is true if $B\in\mathbb{R}^{n\times m}$ with $m<n$ has rank $m$.
My effort:
I tried to show the statement is false. I select some stabilizing $K$. But for every $K$ I tried, I can always find $Q$ and $R$ that make $K$ optimal.
A similar statement can be found in Theorem 7 of Kalman's paper. But in the original paper, Kalman adopted a different form of the cost function (i.e., the control is scalar and an additional cost term $x_t'r u_t$ is considered).
