Remove repeated quadratic constraints to minimize function

Question

Description

I want to minimize $f(\mathbf{x})$ in the region $S$ defined by $n$ matrices $M_{i}$ positive-defined.

$$S = \left\{\mathbf{x} \in \mathbb{R}^{p} : \ \mathbf{x}^{T} M_i \mathbf{x} \le 1 \ \ \forall i=1,\cdots, n \right\}$$

Geometrically speaking, $\mathbf{x}^T M_i \mathbf{x} \le 1$ defines the interior of an $p$-dimentional ellipse, and $S$ is the intersection of these $n$ ellipses.

But I want to remove the redundant constraints. That is, I want to keep the minimum matrices such $S$ keeps the same.

Question

What would be a strategy to remove the redundant matrices?

That is, find the smallest subset of matrices $K$ of size $1 \le m \le n$ from the matrices $M$ such $S$ keeps the same

$$S = \left\{\mathbf{x} \in \mathbb{R}^{p} : \ \mathbf{x}^{T} K_{j} \mathbf{x} \le 1 \ \ \forall j=1,\cdots, m \right\}$$

---

Motivation

A mechanical calcul gives stress field $\bar{\bar{\sigma}}$ for every element. The criteria to decide if something will pass is by checking if Von Mises is bellow a certain value:

$$\text{VonMises}(\bar{\bar{\sigma}}) \le Y$$

I can combinate unitary cases by multiplying scalars to unitary stress fields.

$$\dfrac{1}{Y}\cdot \text{VonMises}\left(\sum_{j=1}^{p} x_j \cdot \bar{\bar{\sigma}}_{j}\right) \le 1$$

For 2D case with scalars $(x_1, x_2)$,expanding the equation, I get into an 2D-ellipsoid equation:

$$\left(\dfrac{\bar{\bar{s_1}}:\bar{\bar{s_1}}}{Y}\right) \cdot x_1^2 + \left(\dfrac{\bar{\bar{s_1}}:\bar{\bar{s_2}}}{Y}\right) \cdot 2x_1x_2 + \left(\dfrac{\bar{\bar{s_2}}:\bar{\bar{s_2}}}{Y}\right) \cdot x_2^2 \le 1$$ $$\underbrace{\begin{bmatrix}x_1 & x_2\end{bmatrix}}_{\mathbf{x}^T} \cdot \underbrace{\left(\dfrac{1}{Y}\begin{bmatrix}\bar{\bar{s_1}}:\bar{\bar{s_1}} & \bar{\bar{s_1}}:\bar{\bar{s_2}} \\ \bar{\bar{s_1}}:\bar{\bar{s_2}} & \bar{\bar{s_2}}:\bar{\bar{s_2}} \end{bmatrix}\right)}_{M} \underbrace{\begin{bmatrix}x_1 \\ x_2\end{bmatrix}}_{\mathbf{x}} \le 1$$

Then, having $n$ elements would lead to $n$ constrains, which is expansive to optimize $f(\mathbf{x})$.

I have $10 000$ matrices $M$ with $p=6$ and minimizing $f(\mathbf{x})$ is taking hours since it has too many constraints. I reduced this number to 5000 by checking if one ellipse is inside another ellipse by using this answer, but it's not enough.

Answer 1

I had a geometric inspiration for this solution, and I will write it in 2 steps.

1. Find $p$ ellipsoids in perpendicular directions

   1.1. Find the highest eigenvalue from all matrix, associated with eigenvector $\mathbf{v}_1$

   1.2. Reduce the dimension of all matrices, excluding the direction $\mathbf{v_1}$

   1.3. Back to step 1.1, do it for $p$ times

2. Project all the minor axis of each ellipsoid in the perpendicular directions.

Step 1

The matrix $M_i$ generates an ellipsoid $L_{i}$

$$L_i = \left\{\mathbf{x} \in \mathbb{R}^{p} : \ \mathbf{x}^{T} M_{i} \mathbf{x} \le 1 \right\}$$

Also, an ellipsoid can be described by $p$ perpendicular axis $\mathbf{a}_{ij}$

$$L_i = \left\{\sum_{j=1}^{p} q_j \cdot \mathbf{a}_{ij} \in \mathbb{R}^{p} : \ \sum_{j=1}^{p}q_j^2 \le 1 \right\}$$

The translation between $M_i$ and $\mathbf{a}_{ij}$ is by its inverse

$$\left[M_i^{-1}\right]_j = \mathbf{a}_{ij}$$

As I'm interested in the intersection region, from all the ellipsoids $L_i$, the one with the minor axis $\mathbf{a}$ from all will be included. That means, the matrix $M_{i}$ with the highest eigenvalue will be included.

Name $k_1$ the index of the matrix such has the highest eigenvalue $\mu_1$ and the eigenvector $\mathbf{u}_1$.

Since we know the maximum value that $S$ can achieve in the direction $\mathbf{u}_1$, we search the maximum in $p-1$ directions. For example, if $p=3$, we collapse all the 3D-ellipsoids in the plane perpendicular to $\mathbf{u}_1$ and then we start searching the minor axis in 2D-ellipsoids.

A matrix $M_i^{(0)} = M_i$ can be decomposed in eigenvalues $\lambda_{ij}^{(0)}$ and eigenvectors $\mathbf{v}_{ij}^{(0)}$

$$M_{i}^{(0)} = \sum_{j=1}^{p} \lambda_{ij}^{(0)} \cdot \mathbf{v}_{ij}^{(0)} \otimes \mathbf{v}_{ij}^{(0)}$$

Removing projection of the eigenvector leads to

$$\mathbf{v}_{ij}^{(1)} = \mathbf{v}_{ij}^{(0)}-\langle \mathbf{v}_{ij}^{(0)}, \ \mathbf{u}_1 \rangle \cdot \mathbf{u}_1$$

$$M_{i}^{(1)} = \sum_{j=1}^{p} \lambda_{ij}^{(0)} \cdot \mathbf{v}_{ij}^{(1)} \otimes \mathbf{v}_{ij}^{(1)}$$

Doing the same for $p$ times, $p$ values of $k$, $\mu$ and $\mathbf{u}$ are obtained, having $p$ boundary ellipsoids in each direction.

Step 2

The nearest point to the origin of the ellipsoid $L_i$ is its minor axis, which I call $\mathbf{b}_i$.

• If $\mathbf{b}_i$ is not inside all the ellipses $M_{k_1}$, $\cdots$, $M_{k_p}$, then we exclude $M_i$ cause

$$\underbrace{L_{k_1} \cap \cdots \cap L_{k_p}}_{\bar{S}} \subset L_{i}$$

• Else, then possibly $M_i$ creates a boundary of $S$.

Then, for every matrix $M_i$, check if the minor point is inside $\bar{S}$. If it's inside, then $\bar{S}$ should be redefined as $\bar{S}\cap L_i$, that means, the group of intersection matrix should include $M_i$.

It's possible to include an strategy of which matrices will be tested first. The easiest one is by ordering the matrix by the lenght of minor axis. Here I show the projected vector $\mathbf{c}$ that can be ordored by $\|\mathbf{c}\|$.

Since there are $p$ directions of the smallest axis from all matrix, we project the smaller axis $\mathbf{b}_i$ of ellipse in these components:

$$\mathbf{c}_i = \left(\langle \mathbf{b}_i, \ \sqrt{\mu_1}\mathbf{u}_1 \rangle, \ \cdots, \ \langle \mathbf{b}_i, \ \sqrt{\mu_p}\mathbf{u}_p \rangle\right)$$

This vector $\mathbf{c}$ lies on a unitary $p$ space $W$. Which means, the translated ellipse $L_{k_1}$ in $W$, would touch $(\pm1, \ 0, \ \cdots, \ 0)$ and the intersection would be inside the cube $\left[-1, \ 1\right]^{p}$

• If $\max_{j}\left(\mathbf{c}_i\right)_{j} > 1$, then all points of boundary of $L_i$ are outside $\bar{S}$, cause it's outside the cube

• If $\sum_{j=1}^{p} \left|\left(\mathbf{c}_i\right)_{j}\right| < 1$, then $\mathbf{b}_i$ is inside the $p$ ellipsoids

• In between, check if $\mathbf{b}_{i}^{T} M_{j} \mathbf{b}_i \le 1 \forall j$

The order would be

Answer 2

You could try the following:

Suppose there exist matrices $M_i$ and $M_j$ ($i \neq j$) such that $M_i - M_j$ is PSD. Then $$x^\top (M_i - M_j) x \geq 0 \Longrightarrow x^\top M_j x \leq x^\top M_i x \quad \forall x \in \mathbb{R}^p.$$ Then $M_j$ can be removed from the set description, since its corresponding inequality is already implied, via $$x^\top M_i x \leq 1 \Longrightarrow x^\top M_j x \leq x^\top M_i x \leq 1.$$ You can make this more general by considering convex combinations of matrices $M_i$. For example, suppose $\sum_{k = 1, k \neq i}^n \lambda_i M_i - M_i$ is PSD for some nonnegative $\lambda$ summing to $1$. Then $$x^\top M_k x \leq 1 \quad \forall k \neq i\Longrightarrow x^\top M_i x \leq x^\top \left( \sum_{k = 1, \, k \neq i}^n \lambda_i M_i \right)x \leq 1,$$ but it seems hard to find such convex combinations, or pairs of matrices even.