If you use a deterministic encryption algorithm (so that you can actually verify passwords without the private key) it basically works like a backdoored hash. An attacker will be able to use a brute force or dictionary attack normally.
One obvious problem with any reversible encryption is that it reveals (at least something about) the password length. (E.g. if you use raw RSA you have to decide how to handle passwords longer than the modulus.) With a real password hash you can allow practically unlimited length without leaking anything about it if your database is compromised. Among other things, this may also allow an attacker to spend their resources on cracking the shortest passwords without wasting them on those that are too strong to crack.
You also miss out on the opportunity to use a slow and/or memory hard password hash. While asymmetric encryption is usually slower than a simple cryptographic hash would be, it is not slower by many orders of magnitude like a password hash allows.
Finally, there is a single point of failure (your private key) that allows leaking all the passwords in the database. If your key is, despite you precautions, compromised an attack can decrypt all the hashes. This means the scheme is at least theoretically weaker than password-hashing that has no back door and so requires every password to be attacker individually.
I would also recommend using established password hashing functions, though rather than either of the ones SEJPM mentions, I would recommend using whatever your programming language or available libraries already support. Whether that is bcrypt or scrypt.
If you really need the plaintext password to be recoverable, you could store both a strong password hash and a normal, non-deterministic public-key encryption of the password: $H(s, p)||E_{pk}(p)$. Password verification would be done using the slow password hash and possible recovery of the plaintext password would be done by decrypting. The limitations with regard to password length and a single point of failure that I mentioned above would still apply.