10

I was watching a video on "system identification" in control theory, in which the creator says that we don't have solutions to nonlinear systems. And I have heard this many times in many contexts, related to control problems or nonlinear odes, etc. I think I am reacting to these kinds of blanket statements, and I would like to understand more precisely what is meant.

But I wanted to understand precisely what is meant that we can't solve nonlinear systems? Indeed, there are probably hundreds of questions on Math SE regarding numerical solutions to nonlinear systems. There are many algorithms for numerically solving different types of nonlinear systems of equations, including Newton's method, sequential quadratic programming, BFGS, Broyden's method, etc. All of these methods have their own limitations, such as positive definiteness, the existence of hessians, and so forth.

Now in a linear ode or linear system of equations, we can get the solution for the system pretty easily, even for large systems. So I can solve large system of equations using numerical linear algebra, etc. I can plot the vector field for a linear system of equations pretty easily, because I have a Jacobian for the system and I can plug in points to plot the corresponding vector field. For a nonlinear system I have to compute those trajectories directly.

So I am hoping that this question is not overly broad. But the issue is that I keep hearing this claim over and over, but I am not sure what the actual specific issue is. I understand that there are difficulties getting solutions for nonlinear equations, it is not like they are all "non-solvable."

krishnab
  • 2,653
  • 1
    Hi. Can you clarify what you mean by "plot" and "plot pretty easily"? Plotting makes sense to me when you have only two variables x,y, or at most three variables x,y,z, but starting at 4 variables I'm not sure what "plot" means since I can only see in 2 or 3 dimensions. – Stef Mar 26 '24 at 10:59
  • 1
    Note that when control engineers speak about "systems", they mean "dynamical systems" (i.e., systems of ordinary differential equations without explicit time dependence), not "systems of linear/nonlinear equations". – Federico Poloni Mar 26 '24 at 16:53
  • @Stef correct, I was just talking about plotting in 2 or 3 dimensions. What I was getting at, is that for linear systems I can find a formula" for computing the vector field of the system. I can just plug in points to the Jacobian in any number of dimensions, as long as the system is linear, and get the corresponding vectors. In a nonlinear system, I don't have that kind of formula for the vector field. If I have a nonlinear system, I can linearize around any point, compute the Jacobian and get the local vector field. But that is different than having a nice formula. That is all I meant. – krishnab Mar 26 '24 at 22:17
  • @FedericoPoloni yes true. I was being a bit loose with my language. So the systems are dynamical systems, in particular linear time invariant systems--in many/most cases--though control of nonlinear dynamical systems is an important area as well. So what I am talking about: linear/nonlinear odes, is different than just solving a purely nonlinear system of equations $f(x1, x2, ..) = 0$. – krishnab Mar 26 '24 at 22:32

2 Answers2

11

Your question is indeed overly broad. As asked, the answer is probably that the unsolvability refers to the absence of answers that are essentially formulas of some kind. The numerical methods you refer to don't count as "solutions" with this definition.

Ethan Bolker
  • 103,433
  • 3
    Thanks for the response. So just to clarify, when I hear someone say that there is no solution to nonlinear systems of equations, I should interpret that as there generally are no "analytical" solutions to systems of nonlinear equations; though of course there are some cases where nonlinear equations have analytical solutions. But the point you are making is the idea of a "formula," which in my mind refers to an analytical solution. Is that matching correct. – krishnab Mar 25 '24 at 20:34
  • 2
    Yes, that's pretty much how I would answer your very broad question. – Ethan Bolker Mar 25 '24 at 20:35
  • So I can understand that then--if I want to find the vector field corresponding to a nonlinear ode, then I have to numerically integrate for a bunch of initial values, and plot that numerical solution. If I have a linear ode, then I have a corresponding formula for the vector field and it is faster to compute. I have to make the usual caveats that numerical methods may not always converge. – krishnab Mar 25 '24 at 20:36
10

I think what you see there is essentially an oversimplified negation of the statement 'linear systems are solvable'. This is a true statement, they can be solved exactly if they are reasonably small and there are lots of numerical methods that are fast and have well-behaved errors. It is therefore desirable to formulate a problem as a linear system if that is possible.

Now a non-linear system is in general non solvable as easily. Some specific non-linear systems are solvable exactly, some also have very good numerical methods to solve them but it depends on the specific form the non-linear system has. So if you have a non-linear system you can't just take the standard solution. You have to look at the specific type and check whether in this case there are good methods to solve it. There may be but it is also possible that the only methods scale badly, work only most of the time or have badly behaving error terms.

quarague
  • 6,505
  • thanks for this. This explanation is more precise and makes sense to me. I have heard the hand wavy "can't solve nonlinear systems" so many times, but that is not an accurate statement. But I also wonder about the size of nonlinear systems. Certainly desirable to keep a nonlinear system small. And in say robotics, they manage to keep the dynamics models pretty small--compared to the number of degrees of freedom for a robot. But I am not sure about other disciplines. I would imagine in biological DNA models with master equations would be very high dim, but not sure. – krishnab Mar 26 '24 at 18:38
  • Numerical methods for nonlinear systems work by just solving a system of linear systems. The largest nonlinear system you can solve is probably slightly smaller than the largest linear system you can solve. In applications, it is not uncommon to solve nonlinear systems on the order of billions of equations. – whpowell96 Mar 29 '24 at 17:01