Hint: The best way to see this is to look it locally. If $f$ is a diffeomorphism, $f^{-1}$ is a diffeomorphism and the inverse of $df$ is $df^{-1}$ defined by $df^{-1}(x,v)=(f^{-1}(x),df^{-1}(v))$.
If $f$ is an immersion submersion and$x\in X$ there exists a chart $U$ containing $x$ such that the restriction of $f$ to $U$ is an immersion/submersion whose image is contained in a chart containing $f(y)$. It is enough to show the result when $f:U\subset \mathbb{R}^n\rightarrow \mathbb{R}^m$. In this case, immersion submersion are characterized by the constant rank theorem and local diffeomorphism
see
Local Submersion Theorem - Differential Topology of Guillemin and Pollack
and
Local Immersion Theorem in $\mathbb{R}^n$ proof
which reduce the problem to a linear map.