Cheerful Parsnip gave a sketch of a proof in his comment. The essential ingredient is the following
Lemma. If $M$ is a connected manifold of dimension $\ge 2$ and $p \in M$, then $M' = M \setminus \{p\}$ is also a connected manifold.
Proof. $M'$ is a manifold since it is an open subset of $M$. For manifolds connectedness and path connectedness coincide. Thus it suffices to show that for all $x, y \in M'$ with $x \ne y$ there exists a path in $M'$ connecting $x$ and $y$. Choose a path $u : I \to M$ such that $u(0) = x, u(1) = y$. Since $M$ is a manifold, $p$ has an open neighborhood $U$ which admits a homeomorphism $h : U \to U'$ to an open subset $U' \subset \mathbb R^n$. W.l.o.g. we may assume that $h(p) = 0$ and $x',y' \notin U$. Th set $U'$ contains a closed ball $D^n_r$ with center $0$ and radius $r > 0$. Its boundary is $S^{n-1}_r$ is path-connected.
If $u(I) \cap h^{-1}(D^n_r) = \emptyset$, we are done. Otherwise $S = u^{-1}(h^{-1}(D^n_r))$ is a non-empty closed subset of $I$ not containing $0,1$. It has an minimum $m$ and a maximum $M$ such that $0 < m \le M < 1$. Clearly $u(m), u(M) \in h^{-1}(S^{n-1}_r)$ (otherwise one of $u(m), u(M)$ would be contained in the interior of $h^{-1}(D^n)$ which contradicts the definition of $m, M$). Since $h^{-1}(S^{n-1}_r)$ is path connected, we can replace $u \mid_{[m,M]}$ by a path in $h^{-1}(S^{n-1}_r) \subset M'$. This gives a path in $M'$ connecting $x$ and $x$.
Corollary. If $M$ is a connected manifold of dimension $\ge 2$ and $F$ is a finite subset of $M$, then $ M \setminus F$ is also a connected manifold.
Let us prove that $F_m(X)$ is a connected manifold.
- $F_m(X)$ is open in $X^m$ (in particular, it is a manifold).
Let $x = (x_1,\ldots, x_m) \in F_m(X)$. There exist open neighborhoods $U_i$ of $x_i$ in $X$ such that $U_i \cap U_j = \emptyset$ for $i \ne j$. See for example Let $x_1,\dots,x_n$ be distinct elements of a Hausdorff space, prove there are pairwise disjoint open sets $U_1,\dots,U_n$ s.t. $x_i \in U_i$. Then $x \in U_1 \times \ldots \times U_m \subset F_m(X)$.
- $F_m(X)$ is path-connected.
Let $x = (x_1,\ldots, x_m)$ and $y = (y_1,\ldots, y_m)$ be points of $F_m(X)$. We have to find a path in $F_m(X)$ connecting $x$ and $y$.
Let us first consider the case that $\{x_1,\ldots,x_m\} \cap \{y_1,\ldots, y_m \} = \emptyset$.
Then all $z^k = (y_1,\ldots, y_k,x_{k+1},\ldots,x_m) \in F_m(X)$. Choose a path $u_1$ in $X$ from $x_1$ to $y_1$ such that $u_1(I) \subset X \setminus \{x_2,\ldots,x_m\}$. This can be done because $x_1,y_1 \notin \{x_2,\ldots,x_m\}$. Then $\bar u_1 : I \to X^m, \bar u_1(t) = (u_1(t),x_2,\ldots,x_m)$ is a path from $x$ to $z^1$. By construction $\bar u_1(t) \in F_m(X)$. Hence $\bar u_1$ is a path in $F_m(X)$. Next choose a path $u_2$ in $X$ from $x_2$ to $y_2$ such that $u_2(I) \subset X \setminus \{y_1,x_3,\ldots,x_m\}$. This can be done because $x_2,y_2 \notin \{y_1, x_3,\ldots,x_m\}$. Then $\bar u_2 : I \to X^m, \bar u_2(t) = (y_1, u_2(t),x_3,\ldots,x_m)$ is a path from $z^1$ to $z^2$. By construction $\bar u_2(t) \in F_m(X)$. Hence $\bar u_2$ is a path in $F_m(X)$. Proceeding inductively, we get paths in $F_m(X)$ from $z^k$ to $z^{k+1}$. This gives a path in $F_m(X)$ from $x$ to $z^m = y$.
We now come to the general case. It suffices to find a point $w = (w_1,\ldots, w_m) \in F_m(X)$ such that $\{w_1,\ldots,w_m\} \cap \{x_1,\ldots,x_m,y_1,\ldots, y_m \} = \emptyset$. In fact, the above special case gives paths in $F_m(X)$ from $w$ and $x$ and from $w$ to $y$.
As in 1. take open neighborhoods $U_i$ of $x_i$ in $X$ such that $U_i \cap U_j = \emptyset$ for $i \ne j$. Pick $w_i \in U_i \setminus \{x_1,\ldots,x_m,y_1,\ldots, y_m \}$. Then $w = (w_1,\ldots, w_m) \in F_m(X)$ and all $w_i \notin \{x_1,\ldots,x_m,y_1,\ldots, y_m \}$.