9

I'm struggling with the following problem:

Let $A, B \in \mathbb R^{n \times d}$. Denote by $\mathcal{P}$ the set of all possible permutations of the rows of $A$. Find a permutation $\pi \in \mathcal{P}$ that minimizes $$\sum_{i=1}^n\sum_{j=1}^d \left( A(\pi)_{ij} – B_{ij} \right)^2$$

Is this problem related — or can it be converted — to another well-known optimization problem for which an efficient algorithm is available? If not, is exhaustive search the only possible approach?

Rodrigo de Azevedo
  • 23,223
Eva
  • 111
  • 1
    You say that classical optimization algorithms cannot be used to solve the problem. But does it mean that exhaustive search is the only possible approach? – Eva May 14 '19 at 13:14
  • 1
    Do you include mixed-integer optimization in exhaustive search? (the problem is readily written as a MIQP) – Johan Löfberg May 14 '19 at 13:50
  • I'm not sure if I understood your question correctly. By exhaustive search, I mean brute force, that is, going through all possible permutations. – Eva May 14 '19 at 13:59
  • 2
    Hint: drop the sums and write the cost function in terms of the squared Frobenius norm. You will need to multiply $A$ by a permutation matrix. Then use the definition of the Frobenius norm to expand the cost function. You will find interesting things. Lastly, check some properties of permutation matrices. – Rodrigo de Azevedo May 14 '19 at 19:58
  • 1
    You have a special case of the Procrustes problem. To solve the resulting maximization problem, use the Birkhoff polytope. Here is an example. – Rodrigo de Azevedo May 15 '19 at 19:15
  • @Rodrigo de Azevedo Very interesting remarks. As you may see the recent edit to my answer (for which I am indebted to your remark above) I make a connection with the hungarian algorithm. I would be happy that you transform your different remarks into an answer : I would immediately upvote it. – Jean Marie May 16 '19 at 15:51

1 Answers1

7

Using the excellent indications given by @Rodrigo de Azevedo under the following form where $\|\|_F$ denotes the Frobenius norm (see remark at the bottom), your issue is equivalent to :

$$\text{minimize} \ \|AP-B\|_F^2=\|AP\|_F^2-2\langle AP,B\rangle+\|B\|_F^2$$

$$\text{minimize} \ \|AP-B\|_F^2=\|A\|_F^2-2 \ \text{trace}(P^TA^TB)+\|B\|_F^2$$

Have a look at https://en.wikipedia.org/wiki/Frobenius_inner_product

As $\|A\|_F^2+\|B\|_F^2$ is constant, it remains to :

$$\text{maximize} \ \text{trace}(PC) \ \text{with} \ C:=A^TB$$

on the group of permutation matrices $P$.

Edit :

Operation trace($PC$) can be understood on the particular $3 \times 3$ case :

$$\text{trace} \begin{pmatrix}1&0&0\\0&0&1\\0&1&0\end{pmatrix}\begin{pmatrix}c_{11}&c_{12}&c_{13}\\c_{21}&c_{22}&c_{23}\\c_{31}&c_{32}&c_{33}\end{pmatrix}=c_{11}+c_{23}+c_{32},$$

A straightforward generalization to the $n \times n$ case explains that the objective is to find, among all sums :

$$s_{\sigma}=c_{1\sigma(1)}+c_{2\sigma(2)}+\cdots+c_{n\sigma(n)} \ \text{,} \ \sigma \in \frak{S}_n$$

the one which is maximal.

Fortunately, this can be done by the efficient Hungarian algorithm applied on matrix $C$, or more exactly because its direct form deals with minimization, an adapted version of it for a maximization context.

Why do we say efficient ? Because this algorithm has complexity $O(n^4)$ instead of the $O(n!)$ complexity of the brute force approach.

Remarks :

  1. In fact, operation $A \to AP$, where $P$ is a permutation matrix, provides a permutation of columns, which is clearly equivalent to a permutation of lines provided by $A \to PA$.

  2. Connected : https://math.stackexchange.com/q/175893.

Jean Marie
  • 88,997