I am looking into various one way functions and I stumbled upon a Rabin function, which is squaring modulo an RSA modulus $N=pq$, where $p,q$ are prime: $R_N(x) = x^2 \mod N$.
Would it lose the one-way property if $N$ is prime and not a product of two primes?
UPD: Also, is Rabin function still one-way if factorization of $N=pq$ is known?
Would [$f_N(x)=x^2\bmod N$] lose the one-way property if $N$ is prime and not a product of two primes?
Yes, thanks to the Tonelli-Shanks algorithm (special cases here).
[Is] Rabin function still one-way if factorization of $N=pq$ is known?
No, because the main ("only") information advantage the private key holder has in the Rabin cryptosystem over the public key holder is the factorization of $N$. If the one-wayness was preserved even under known factorization the system wouldn't allow decryption in its normal form.
The way to see how factorizing allows square roots in this scenario is to remember that the chinese remainder theorem establishes a ring isomorphism between $\mathbb Z_N$ and $\mathbb Z_p \times \mathbb Z_q$ ... and as established above we know how to compute square roots in $\mathbb Z_p$.
