I was thinking of a potential new way to increase someone's account security.
The idea was allowing the user to choose one of the hash formulas available in the software (example: "MD5", "SHA512" and "Whirlpool"), decreasing the chances of any intruder to find out the hashing system used in case passwords are — for some reason — leaked increasing a potential required time of brute-force or dictionary attacks.
Would that be a good idea?
This doesn't sound like a good idea. As CodeInChaos already pointed out, choose one good password hashing scheme like scrypt or bcrypt and go with that
The reason why what you proposed isn't so great is that you'll give the user the choice between a few good password hashing options and the work of an attacker is at best multiplied by the number of options you proposed.
By doing that it's also unlikely that a bad password hashing will make its way into the list and the users picking that one will be at the attacker's mercy. Lastly I don't think it's a good design to leave security relevant choices up to the user.
How should an average user know better than the designer of the system what's good for him ?
To supplement the other answers:
...decreasing the chances of any intruder to find out the hashing system used...
If you don't pad your hashes to make them of equal length, the attacker can figure out which scheme is used. That would unnecessarily increase your complexity factor (and might even confuse your own code if not done right).
Furthermore, you are dividing your users in three categories:
This class of users will get irritated, confused and potentially feel stupid. Interesting article on this but the point is, they might end up hating you. Or they might escalate to know-it-all's.
...who believe a simple hash function is all it takes to make a website hacker-proof. See the Dunning-Kruger effect and decide if that's your target audience.
The people sites like this attract (except for me, I'm more of a know-it-all). They will avoid your website like the plague, and if they absolutely must register to see something, they'll use Bloody Vikings! and PASSword1 as their password, and never set foot in your website again. They might even send you a strongly-worded email about your policy (if you're lucky), or even use you as a counter-example in their next presentation.
Bonus: You are also violating Kerckhoffs's principle by relying on the inability of the attacker to guess the applicable scheme. You want to make his life difficult - that's why we use salts and good password policies for. Enforcing the latter will get you much better results.
That said, as Alexandre et al says, use scrypt or bcrypt instead. This and this question on Security.SE might help you decide on the work factor.
If you have only "advanced" users they will choose the "better" function available. If you have some "no-advanced" users is, probably, better that you choose at their place.
So even giving a large choice, every one should stick to the better one.
So I'd choose just one strong and secure function and stick to it.
Other people already suggested you to use scrypt, and give a look also to bcrypt.
