Python memory error: compute inverse of large sized matrix

Question

I have a big matrix of size 200000 x 200000. I need to compute its inverse. But it gives out of memory error on using numpy.linalg.inv. Is there any way to compute the inverse of a large sized matrix.

Answer 1

Well it does not matter the language you will use, I am surprised you can even store that matrice in memory... I had a similar problem while researching on kernel methods.

• First, to put in perspective how huge that matrix is, I will assume each element is 32 bit word: $ 200,000 \times 200,000 \times 32 = 1.28 \times 10^{12} $ bits or 149 GB

The bitlenght is variable but even for a boolean matrix that is about 4,65 GB and you said your data is a bit more complex than that.

Solution 1

If you trying to solve a linear system there are many iterative solutions that might help you computing an 200,000 x 1 array aproximation of your system answer without having to store that absurdly large matrix in memory. But you should be aware that this might take a bit long because you might have to load information from the disk.

Solution 2

Another possible solution, if you really need that inverse is if that matrix can be expressed as sumation of outer products of vectors. If so, you can do this:

$$ M = \sum v_i.u_i^T $$

Get the first set element of the sum and invert it so you have $M^{-1}_{bad aproximation}$ matrix by using pseudoinverse, instead of doing the outerproduct you can invert $u_1$ and $v_1$ and use Moore-Penrose Inverse property:

$$ M^{-1}_{bad aproximation} = (v_1.u_1)^\dagger = v_1^\dagger . u_1^\dagger $$

Combine this with Sherman-Morrison formula to add the other $v_i$ and $u_i$ vectors.

Sherman-Morrison formula gives: $$ (A + uv^T)^{-1} = A^{-1} - \dfrac{A^{-1}uv^TA^{-1}}{1 + v^TA^{-1}u} $$

Solution 3

You might try using Schur Complement which allows you to do the following:

Given a matrix $M$, it can be expressed as $$ M = \begin{bmatrix} A & B \\ C & D \\ \end{bmatrix} $$ where $A$ and $B$ must be square you can solve the inverse of $M$ blockwise as:

$$ M^{-1} = \begin{bmatrix} (M/D)^{-1} & (M/D)^{-1}BD \\ -D^{-1}C(M/D)^{-1} & (M/A)^{-1} \\ \end{bmatrix} $$ where $(M/D)$ denotes the schur complement of block D of the Matrix M and is defines as

$$ (M/D) = A - BD^{-1}C $$

also $$ (M/A) = D - BA^{-1}C $$

If $A$ or $D$ are singular, the matrice $M$ is singular and a pseudo inverse might be used.

This can give a reduction from inverting a $200,000 \times 200,000$ matrice to inverting 4 $50,000 \times 50,000$ matrices and some matrice multiplication, also this can be implement in parallel applications and there is way to do this recursively given certain conditions (enabling a reduction greater than 4 in size)

The $50,000 \times 50,000$ matrices store is only 9.3 GB each and will be easier to invert

Hope it helps. Also you might want to search answers in Math Stack or in Stack Overflow and even ask there.

Edit: Added another solution with Schur Complement