Let me give two rough sketches.
The quick and dirty way is algebraically. Namely, Poincaré duality yields $H_0(M\setminus N;\mathbb{Z}/2\mathbb{Z})\cong H_c^n(M\setminus N;\mathbb{Z}/2\mathbb{Z})$ and the latter can be calculated by again applying Poincaré duality to the other terms in the exact localization sequence
\begin{equation*}
0=H_c^{n-1}(M;\mathbb{Z}/2\mathbb{Z})\rightarrow H_c^{n-1}(N;\mathbb{Z}/2\mathbb{Z})\rightarrow H_c^n(M\setminus N;\mathbb{Z}/2\mathbb{Z})\rightarrow H_c^n(M;\mathbb{Z}/2\mathbb{Z})\rightarrow H_c^n(N;\mathbb{Z}/2\mathbb{Z})=0.
\end{equation*}
(Furthermore, $M$ is orientable as it is simply connected and hence $M\setminus N$ is orientable too. An analysis of the localization sequence with $\mathbb{Z}$-coefficients then implies that $N$, too, is orientable.)
In case our manifolds are smooth, I offer up a geometric argument that is less efficient, but somewhat less high-powered. Following your intuition, let $N\subseteq U\subseteq M$ be a tubular neighborhood. Argue using the simple connectedness of $M$ that the inclusion $U\setminus N\rightarrow M\setminus N$ induces a bijection between their components, e.g. by analyziging a fitting Mayer-Vietoris sequence. Now, $U\setminus N$ is the total space of an $\mathbb{R}^{\times}$-bundle over the connected space $N$, so it is connected if and only if that bundle has no monodromy if and only if the normal bundle of $N$ in $M$ is trivial if and only if $N$ is orientable (these equivalences are all general theory of normal/line bundles). The latter is the case for any closed (equivalent to properly embedded) codimension $1$ submanifold. This is "well-known", but proofs aren't as common as they should be. You can adapt this argument by Georges Elencwajg or this argument by Samelson.