Let me reformulate your question in a form that is more easily answerable. The number of irreps is the cardinality of an orthonormal basis of $L^2(G)$, where $G$ is the group under consideration. So, let us reason in terms of $L^2$ spaces.
Definition. Let $\Omega$ be a set and let $d\mu$ be a measure on it. We let $L^2(\Omega)$ denote the vector space of all functions $f\colon \Omega\to \mathbb C$ with the property that
$$\tag{1}
\int_{\Omega}|f(x)|^2\, d\mu <\infty.$$
In the following we consider $\Omega=\{1, 2, \ldots, n\}$, with the "counting measure", which means that (1) reads
$$
\int_{\{1, 2, \ldots, n\}} |f(k)|^2\, d\mu = \sum_{k=1}^n |f(k)|^2, $$
and there is no need to require finiteness, as that's just a finite sum. Since a vector in $\mathbb C^n$ can be interpreted as a function from $\{1, 2, \ldots, n\}$ to $\mathbb C$, we see that $L^2(\{1, 2, \ldots, n\})$ is just a fancier way of writing $\mathbb C^n$.
We also consider $L^2(0, 1)$, in which case the measure is the standard Lebesgue measure on $(0,1 )$.
You have noticed that $\mathbb C^n=L^2(\{1, 2, \ldots, n\})$ admits an orthonormal basis made of $\lvert \{1, 2, \ldots, n\}\rvert$ elements, and this led you to conjecture that $L^2(0, 1)$ should have an orthonormal basis made of $\lvert (0, 1) \rvert$ elements. However, this is false, as orthonormal bases of $L^2(0, 1)$ are countable; one example of such a basis is the trigonometric system $\{e^{2\pi i xk}\ :\ k\in\mathbb Z\}$.
The reason for this mismatch is that the two kind of orthonormal bases are deeply different. In the case of $\mathbb C^n$, we are talking of an algebraic basis, while in the case of $L^2(0,1)$ there are many more elements to take into account. First, elements of $L^2(0,1)$ are not just functions of $(0,1)$ into $\mathbb C$, like in the finite-dimensional case; they must also satisfy quite stringent requirements (being measurable and square-integrable). This rules out a lot of functions, dramatically reducing the cardinality of a possible basis. Moreover, the basis itself is not an algebraic one; it involves finite sums, so it requires the introduction of a topology.
A final note. You ask whether the irreps are always the eigenfunctions of some operator. I think that the answer is true, but I don't know for sure. The keyword to search for is "Casimir operator". For $SO(n)$, this operator is the Laplace-Beltrami on the sphere, which indeed has a discrete spectrum. In the comments, you suggest that this is a general phenomenon. It may well be, but I don't know if it is a general fact that a Casimir operator (whatever that is) is the Laplace-Beltrami for some compact Riemannian manifold.
What I would bet on is: the same compactness mechanism that is responsible for the discreteness of the spectrum of a compact Riemannian manifold is responsible for the discreteness of irreps in Peter-Weyl's theorem (follow the link for more information on such compactness in the Riemannian manifold case). To be sure, one should go through the proof of Peter-Weyl and check this.
Final note: you say "discreteness of spectrum for bounded geometries"; I have interpreted this, in mathematical terms, as "discreteness of the spectrum of a compact Riemannian manifold".
