The reflection of a rotation "undoes" that rotation, so your interpretation of the definition is correct.
For this dihedral group, $\mathbb{Z}_6$ is the rotational subgroup, and $\mathbb{Z}_2$ is the reflection subgroup of $D_6$. This means that a rotation composed with itself 6 times yields the identity and, similarly, a reflection composed with itself yields the identity. Intuitively speaking, one might guess that a reflection does the action of three (6/2=3) rotations. However, this is incorrect. The reason that $\mu \neq \rho^3$ is that reflections alter the orientation for rotations; rotations mean something different before a reflection versus after it. The result is that $D_6$ is not a cyclic group. That means it cannot be generated by a single element. So, there are two types of actions on $D_6$. Therefore, to determine the stabilizer of a vertex, we must consider whether any sort of rotation or reflection keeps that vertex in the same location. Obviously, the identity action is in the stabilizer since doing nothing preserves the location of a vertex. No rotation (except the identity) stabilizes a vertex. However, a reflection through that vertex does stabilize it. This means that the only nontrivial stabilizer of 2 is the reflection about the line through 2 and the center of the hexagon.
Computing the stabilizer of $\{2,4,6\}$ is entirely different from finding the stabilizer of 2. The question that we need to answer is: What actions (reflections, rotations, or compositions thereof) preserve the location of 2, 4, and 6? Obviously, the trivial action (identity) works. Also, $\rho^2$ by which $2 \mapsto 4$, $4 \mapsto 6$, and $6 \mapsto 2$. Moreover, $\rho^4$ works since $2 \mapsto 6$, $4 \mapsto 2$, and $6 \mapsto 4$. Now that we identified all of the rotational elements in the stabilizer subgroup, we must find the reflectional elements in it. Observe that the reflection through 0 and the midpoint of 2 and 4, the reflection through 2 and the midpoint of 0 and 4, and the reflection through 4 and the midpoint of 0 and 2 all preserve the set $\{2,4,6\}$.
Therefore, we conclude that the stabilizer subgroup of $D_6$ contains seven elements: the identity, three rotations, and three reflections (as defined above).