Intuition behind duality in linear programming

Question

I'm looking for an intuitive explanation of the duality principle in Linear Programming.

About having a solution or not:

Farkas' Lemma: $A x=b ; x \geq 0$ has a solution <=> $A^T y \geq 0 ; b^T y<0$ doesn't have a solution

About optimality: $ \max _x c^T x=\min _y b^T y $ $$ \left\{\begin{array}{l} A x \leq b \\ x \geq 0 \end{array}\right. $$ <=> $$ \left\{\begin{array}{l} A^T y \geq c \\ y \geq 0 \end{array}\right. $$

But I don't really see what they have to do with each other? I don't understand the intuition behind the duality. Could someone explain it to me?

What I also don't understand, is how the solution of the dual problem (when applying the method with the tableau) corresponds to the coefficients of the variables (and not the values, which is the case for the original problem)?

P.S. I have always worked with maximising and with "<=", so it would be nice if you could take that into account in your answer (e.g., when you provide examples in your answer).

Answer 1

I think this blog post does quite a good job of showing how the dual LP comes from the primal, and I'll try to provide a short version here:

Say we have an LP like this:

$$\begin{eqnarray} \max & 5x_1 - 6x_2 \\ s.t. &\ 2x_1 - x_2 = 1 & (1) \\ & x_1 + 3x_2 \leq 9 & (2) \\ & x_1 \geq 0 \end{eqnarray}$$

and we want to estimate an upper bound on the solution. Well, we can take $9 \times (1) + 1 \times (2)$ and get

$$\begin{eqnarray} 9(2x_1 - x_2) + 1(x_1 + 3x_2) & \leq & 9 \times 1 + 1 \times 9 \\ 19 x_1 - 6 x_2 & \leq & 18\end{eqnarray}$$

But because $x_1 \geq 0$, $5 x_1 - 6 x_2 \leq 19 x_1 - 6 x_2 \leq 18$, so 18 is an upper bound on the solution to the LP. More broadly, we can consider various combinations of $y_1 \times (1) + y_2 \times (2)$, and if we can choose values right, we can still claim to have found an upper bound to the LP. Essentially, we're looking for values of $y_1$ and $y_2$ that let us do the following:

$$5 x_1 - 6 x_2 \leq y_1 (2 x_1 - x_2) + y_2 (x_1 + 3 x_2) \leq 1 y_1 + 9 y_2$$

Skipping some details that the blog post covers, the constraints on $y_1$ and $y_2$ are $2 y_1 + y_2 \geq 5, -y_1 + 3 y_2 = -6, y_2 \geq 0$. So for any choice of $y_1$ and $y_2$ in the region defined by those constraints, the value of $y_1 + 9 y_2$ is an upper bound to the solution of the primal. But that means that the minimal such value is still an upper bound, and hopefully is the lowest upper bound. So in other words, we hope to find the maximum of the primal LP by solving:

$$\begin{eqnarray} \min & y_1 + 9 y_2 \\ s.t. & \ 2 y_1 + y_2 \geq 5 \\ & - y_1 + 3 y_2 = -6 \\ & y_2 \geq 0 \end{eqnarray}$$

which we notice is the dual LP to this problem.

One weird way to think about it is that the variables of the dual problem represent how much weight we put on caring about each constraint of the primal, and the dual as a whole represents working out how much each constraint of the primal contributes in determining its optimal solution.