In spectral graph partition theory, the eigenvector $v_2$ (also called Fiedler vector) corresponding to the second smallest eigenvalue $\lambda_2$ (also known as Fiedler eigenvalue, which actually also defines the algebraic connectivity of a graph) of the Laplacian matrix $L$ of a graph $G$, in general, is used to partition the graph.
What is the underlying philosophy behind this? Any reference to any related proofs on this?
I found this exposition of the Smallest Eigenvalues of a Graph Laplacian by Shriphani Palakodety to be readable and informative.
The article begins with a discussion of eigenvectors for the smallest eigenvalue, which in the case of the graph Laplacian happens to be zero. The number of eigenvectors for this eigenvalue gives the connected components of the graph (and the nonzero entries of each eigenvector point to the nodes of each connected component).
Then the discussion turns to the second smallest eigenvalue and what it has to do with clustering of nodes and therefore partitioning of a graph. The corresponding bibliography reference is to a "landmark paper" by Miroslav Fiedler (1973) on Algebraic Connectivity of Graphs.
A deeper survey is in this 2007 paper by Nair Maria Maia de Abreu, "Old and new results on algebraic connectivity of graphs".
The key ingredient of this result is Perron-Frobenius theorem.
Let $L=D-A$, where $D$ is the degree matrix and $A$ is the adjacency matrix (See this wiki page). Recall that $L$ is a singular positive semidefinite symmetric matrix.
Now, notice that if $D$ is singular then the graph is already disconnected and the nullity of $L$ is bigger or equal to 2. Let us assume that $D$ is invertible.
Let $L_{1}=Id-D^{-\frac{1}{2}}AD^{-\frac{1}{2}}$, which is also positive semidefinite. Notice that the nullities of $L_{1}$ and $L$ are equal.
Now the nullity of $L_{1}=Id-D^{-\frac{1}{2}}AD^{-\frac{1}{2}}$ is bigger or equal to 2 if and only if the biggest eigenvalue of $D^{-\frac{1}{2}}AD^{-\frac{1}{2}}$, which is $1$ (why?), has multiplicity bigger or equal to $2$.
Since $D^{-\frac{1}{2}}AD^{-\frac{1}{2}}$ is symmetric with positive entries and the multiplicity of the spectral radius (the biggest eigenvalue) is not 1 then $D^{-\frac{1}{2}}AD^{-\frac{1}{2}}$ is not irreducible. Thus $A$ is also not irreducible (Check this wiki page).
Since $A$ is not irreducible, there is a permutation matrix $P$ (which represents a relabeling of the vertices of the graph) such that $PAP^{-1}=\begin{pmatrix} A_1 & B\\ 0 & A_2\end{pmatrix}$, where $A_1,A_2$ are square matrices.
Since $P$ is a permutation then $P^{-1}=P^t$. Hence $PAP^t$ is symmetric, since $A$ is symmetric. Thus, $B=0$. Notice that $A_1,A_2$ are adjacency matrices too. Thus, the graph is not connected.
Some of this answer repeats a few details in the answers before this response and adds to them.
There is a very readable discussion about graph partitions using eigenvalues of the Laplacian matrix in the paper by Newman 2006, “Finding Community Structure In Networks Using The Eigenvectors Of Matrices” which can be found as a pre-print at this url: https://arxiv.org/pdf/physics/0605087.pdf
NOTE: the paper emphasizes that for context such as social graphs, the leading eigenvector of the modularity matrix provides a better partitioning than the leading eigenvector of the Laplacian.
The following is mostly verbatim excerpts from the paper:
[With the goal of partitioning an unlabeled unweighted graph into non-overlapping groups using the eigenvalues of the Laplacian (which is positive and symmetric)] one sees that the smallest eigenvalue = 0, and is always the smallest eigenvalue of the Laplacian. The corresponding eigenvector is v1 = (1, 1, 1, . . .)/ √ n, and the minimum cut size R, is in the term corresponding to the lowest eigenvalue λ1 = 0, hence R=0. Having a minimum cut size of 0 is equivalent to placing all the vertices in group 1 and none of them in group 2, which is a valid division of the network, but it is not a useful one for most context. That’s the trivial solution and there are various methods to forbid it to solve whatever graph partition problem one is trying to achieve. (see page 3 and 4 of the paper).
One goal might be to achieve a partition such that the number of nodes are equal between 2 groups. This solution works best as an approximation when there is a large spectral gap (= difference between largest absolute values of eigenvalues).
Knowing the definition of R in equation 12, and that the goal is to have n1 = n2, R would be minimized by choosing s proportional to the second smallest eigenvector v2 of the Laplacian, also called the Fiedler vector.
This choice places all of the weight in Eq. (12) in the term involving the second-smallest eigenvalue λ2, also known as the algebraic connectivity. The other terms would automatically be zero, since the eigenvectors are orthogonal. Unfortunately, there is an additional constraint on s imposed by the condition, Eq. (3), that its elements take the values ±1, which means in most cases that s cannot be chosen parallel to v2. This makes the optimization problem much more difficult. Often, however, quite good
approximate solutions can be obtained by choosing s to
be as close to parallel with v2 as possible.
The best solution is achieved by assigning vertices to one of the groups in order of the elements in the Fiedler vector, from most positive to most negative, until the groups have the required sizes.
For groups of different sizes there are two distinct ways of doing this, one in which the smaller group corresponds to the most positive elements of the vector and one in which the larger group does.
We can choose between them by calculating the cut size R for both cases and keeping the one that gives the better result.
This then is the spectral partitioning method in its simplest form. It is not guaranteed to minimize R, but, particularly in cases where λ2 is well separated from the eigenvalues just larger than it, it often does very well.
