Converse linear quadratic optimal control

Question

It is well known that for a linear time invariant system

$$ \dot{x} = A x + B u \tag{1} $$

with $(A, B)$ controllable, there exists a static state feedback $u = -K x$ such that the cost function

$$ J = \int_0^{\infty} x^T Q x + u^T R u \, dt \tag{2} $$

is minimized, assuming $Q \geq 0$ (positive semi-definite) and $R > 0$ (positive definite). The gain $K$ is the solution of the algebraic Riccati equation:

$$ \begin{align} 0 &= A^T P + P A - P B R^{-1} B^T P + Q \\ K &= R^{-1} B^T P \\ P &= P^T \geq 0 \end{align} $$

known as linear quadratic regulator (LQR). However, I wonder whether the converse also holds?

That is, given a stabilizing $K_s$ (such that $A - B K_s$ is Hurwitz), do there exist matrices $Q \geq 0$ and $R > 0$ such that $u = -K_s x$ minimizes $(2)$ given $(1)$? Or put differently:

Question: Is every stabilizing linear state feedback optimal in some sense?

@A.Γ. I guess so. I assume that $|| . ||^2$ means Euclidean norm squared. Then that would be the same as $Q = K_s^T K_s \geq 0$, $R = I > 0$ (identity matrix) and $N = K_s^T$ for the cost function including the mixed term $J = \int_0^{\infty} x^T Q x + u^T R u + 2 x^T N u , dt$ which is also a standard LQR problem. However, how do you know that this will be the cost function which will give $K_s$ when solved? — SampleTime, Apr 22 '19 at 11:07
@A.Γ. Ok, just noticed that $J = 0$ for $u = -K_s x$ so it is obviously a minimizer. That is pretty cool, if you want to post it as an answer, I will accept it. However, this surprises me as it means that every stabilizing state feedback is optimal in the LQR sense for some $Q$, $R$, $N$. I didn't expect that to be true in general... — SampleTime, Apr 22 '19 at 11:48
@A.Γ. but this will also hold for an unstable state feedback. — Kwin van der Veen, Apr 22 '19 at 13:33
@KwinvanderVeen I do not understand what you mean by "this". An "unstable" feedback cannot be a solution to LQR in the class of stabilizing controllers, hence, cannot be optimal unless you optimize over all linear controllers (which is not usually the case). — A.Γ., Apr 22 '19 at 14:01
@A.Γ. $$J=\int_0^\infty|K,x+u|^2,dt$$ still has $u=-K,x$ as the minimizer even if $A-B,K$ is unstable. — Kwin van der Veen, Apr 22 '19 at 14:06

Arastas · Accepted Answer · 2019-04-23T11:34:45.903

5

See the paper: Kalman, R. E. (1964). When is a linear control system optimal?. Journal of Basic Engineering, 86(1), 51-60.

The answer is positive at least for a class of systems. As far as I remember, the answer is also positive for a general LTI system, but I cannot find a reference at the moment.

UPDATE: Every linear system with nondynamic feedback is optimal with respect to a quadratic performance index that includes a cross-product term between the state and control, see [R1].

If you do not allow for the cross-product term, then several sufficient and necessary conditions are known, see for example [R2] and the references there.

[R1] Kreindler, E., & Jameson, A. (1972). Optimality of linear control systems. IEEE Transactions on Automatic Control, 17(3), 349-351.

[R2] Priess, M. C., Conway, R., Choi, J., Popovich, J. M., & Radcliffe, C. (2015). Solutions to the inverse lqr problem with application to biological systems analysis. IEEE Transactions on control systems technology, 23(2), 770-777.

edited Apr 23 '19 at 11:34

answered Apr 22 '19 at 11:58

Arastas

2,443

At least for single input systems LQR can be shown to have at least 60° phase margin and $(\infty,0.5)$ gain margin. So not all (stable) state feedback will be optimal in some sense with a $P>0$. – Kwin van der Veen Apr 22 '19 at 13:32
@KwinvanderVeen That is exactly what I thought as well, but apparently you can always construct valid $Q, R, N$ such that the LQR problem has $K_s$ as a solution, so it should be true? – SampleTime Apr 22 '19 at 13:38
1

@SampleTime For that proposed solution you need to solve $A^\top P + P A - (P B + N) R^{-1} (B^\top P + N^\top) + Q = 0$, but $N,R^{-1} N^\top = Q$, so $P=0$ is always a solution. Normally numerical LQR solvers try to find a "sensible" solution with $P=P^\top>0$, so will probably return an error when trying to solve the CARE with those $Q$, $R$ and $N$. Also that proposed cost function also returns zero when the state feedback is unstable. – Kwin van der Veen Apr 22 '19 at 13:53
@KwinvanderVeen Thats an interesting observation, but I think $P$ is allowed to be zero, i.e. I have read in several documents that $P = P^T \geq 0$ is required for the solution of the Riccati equation (which would include $P = 0$ as valid stabilizing solution). At least in Matlab, the lqr command computes a stabilizing $K$ with $P = 0$ without complaints (tested only a few randomly generated cases though). – SampleTime Apr 22 '19 at 14:21
1

@SampleTime Check also this one: Priess, M. C., Conway, R., Choi, J., Popovich, J. M., & Radcliffe, C. (2015). Solutions to the inverse lqr problem with application to biological systems analysis. IEEE Transactions on control systems technology, 23(2), 770-777. – Arastas Apr 23 '19 at 11:08
1

Please see the updated answer. – Arastas Apr 23 '19 at 11:39
1

@SampleTime I just checked, the guaranteed gain and phase margins for LQR only hold when $N=0$. For the proposed LQR problem with $N = K^\top$ the lower bound for these margins are actually zero. – Kwin van der Veen Apr 23 '19 at 12:53
Thanks for the additional references, that answers the question. – SampleTime Apr 23 '19 at 17:43
@KwinvanderVeen Ok, that of course explains a lot. I was already wondering because many lecture notes point out the guaranteed gain and phase margins as a special feature of LQR. Would be quite pointless if every state feedback would give those guarantees. Where did you find that statement about $N$? – SampleTime Apr 23 '19 at 17:46
1

@SampleTime I derived that myself. Namely in the single input case (so $R$ scalar) with $Q$ invertible it can be shown that the open loop in the Nyquist plot stays outside the circle around the minus point with radius $1-N^\top Q^{-1} N /R$. – Kwin van der Veen Apr 24 '19 at 14:45
@KwinvanderVeen Interesting approach. I wonder if that also means that we only get the guaranteed margins if also $Q > 0$ (strictly positive definite)? – SampleTime Apr 26 '19 at 19:46
@SampleTime When $N=0$ and $Q$ is only semi-positive definite the margins do hold. For more details you could take a look at my notes (see last page for the margins derivation). – Kwin van der Veen Apr 27 '19 at 00:21
@KwinvanderVeen The derivation is very interesting, thank you for sharing. I actually meant $Q$ singular, not positive semi-definite (psd). I mixed that up because often a psd matrix is also singular. One common choice for $Q$ is for example $Q = C^T C$ which would for $C = \begin{bmatrix}1 & 0 \end{bmatrix}$ (also quite common) give a singular $Q \geq 0$ which would violate the assumptions for the margin derivation. – SampleTime Apr 28 '19 at 19:34

score 3 · Answer 2 · answered Apr 23 '19 at 12:23

The problem is called the Inverse Problem of Optimal Control (see page 147 - 148).

Given a system $$\dot{x}=Ax+Bu,\qquad x(t_0)=x_0$$ $$ z = \begin{bmatrix} Q^{1/2}& 0 \\ 0 & R^{1/2}\\ \end{bmatrix}\begin{bmatrix}x\\u \end{bmatrix},$$

with $(A,B)$ is stabilizable, $(Q,A)$ is detectable and $R>0$ (positive definite). The linear quadratic regulator problem is given by minimizing

$$\int_{0}^{\infty}z^Tz dt.$$

From Boyd et al 1994: The inverse problem of optimal control is the following. Given a matrix $K$, determine if there exists $Q ≥ 0$ and $R > 0$, such that $(Q, A)$ is detectable and $u = Kx$ is the optimal control for the corresponding LQR problem. Equivalently, we seek $R > 0$ and $Q ≥ 0$ such that there exists P nonnegative and P1 positive-definite satisfying

$$(A + BK)^T P + P(A + BK) + K^TRK + Q = 0, \quad B^T P + RK = 0$$

and $A^T P_1 + P_1A < Q$. This is an LMIP in $P$, $P_1$, $R$ and $Q$. (The condition involving $P_1$ is equivalent to $(Q, A)$ being detectable.)

Thanks for the additional reference as well. From there it becomes clear that $P = 0$ is allowed because stability is guaranteed by $P_1 > 0$. — SampleTime, Apr 23 '19 at 17:49

Converse linear quadratic optimal control

2 Answers2

Linked