If $A$ is nonzero symmetric positive semi-definite and noninvertible, it may be written as $SDS^{-1}$ for $S$ orthogonal and $D$ a diagonal matrix with diagonal entries $d_1,\cdots,d_k,0,\cdots,0$ for $d_i >0$ and $k>0$. Use $x_1,\cdots,x_n$ for coordinates before diagonalization and $w_1,\cdots,w_n$ for coordinates after diagonalization.
Write $A_Q = \begin{pmatrix} A & \frac{1}{2}b \\ \frac{1}{2} b^T & c \end{pmatrix}$ and write $S' = \begin{pmatrix} S & 0 \\ 0 & 1\end{pmatrix}$. Then $S'A_QS'^{-1} = \begin{pmatrix} D & \frac{1}{2} b' \\ \frac{1}{2} b'^T & c \end{pmatrix}$ where $b' = Sb$.
If by analogy with the case of $y=ax^2+bx+c$ with $a>0$, we take the most symmetric region of the graph (previously: minimum of this equation, which is bad because of things like the graph of z=x^2+y) to be the vertex, we find something interesting: it is no longer necessary for our vertex to be a point. Instead, it's the intersection of a codimension $k$ vector space shifted by $b'$ with our graph - that makes it a codimension $k+1$ subset. This vertex is in fact everything on our graph that satisfies the $k$ equations $w_i = \frac{-b'_i}{2d_i}$ for $d_i\neq 0$. The "axis of symmetry" is just the linear space determined by the equations above. A fun extra fact is that the symmetries of this higher-dimensional parabola are exactly the group $\prod_{\lambda \neq 0} O(\dim A_\lambda)$ where the product is taken over $\lambda$ in the eigenvalues of $A$ and each factor acts on the eigenspace $A_\lambda$.
I don't as of yet see a good way to phrase the axis of symmetry in terms of just $A,b$ owing to the fact that $A$ is explicitly non-invertible. Perhaps there is something to do with taking a maximal-rank minor, inverting it, and applying that to $b$ (or maybe a pseudo-inverse?), but I don't see how to make that work yet.
Edit 7/8 2304 GMT: replaced "minimum" in paragraph 3 with "most symmetric region" because of examples with no minimum.
