How do I know if my algorithm is right (without coding)?

Question

For some background : I think I am quite good at advanced (graduate/research level) mathematics and physics but I have no CS background. Now I am taking a graduate level class on algorithms.

I don't think I have ever written a wrong proof in mathematics - as in either I can prove something or I can't. I have never faced a situation whereby I wrote a wrong proof - its just obvious if I am writing a wrong proof.

BUT with algorithms it seems that its just trivial to write a wrong algorithm. I am simply not getting as to how does one know if what one has written is right or wrong. There seems to be something "mysterious" going on here - like while I write a recursive/dynamic programming solution etc. - I am thinking that I wrote something right but then somebody comes along with an example where my answer doesn't work!

Its quite contrary to my years of experience of writing proofs in mathematics!

This is getting freaking scary! Its better to not be able to do something than to be writing a wrong answer and not knowing it! (...often enough I land up writing a greedy solution trying to optimize something locally but its not the globally correct answer!..how does one see it!?...coming up with a counter-example is also almost always very hard!..)

Can someone help?

---

I am currently doing a lot of practice with this but I wonder if there is something that I am missing totally!

Answer 1

You prove that the algorithm is correct. This requires (1) writing out a precise specification of what you mean by correctness, (2) writing a mathematical proof that your algorithm meets this specification. Algorithms courses and textbooks often include some material on how to prove algorithms correct, as well as many examples of this, so that would be a good starting point to study, if you want to learn more about proofs of correctness for algorithms. See also formal methods / formal verification, and possibly the following question on this site: How to fool the "try some test cases" heuristic: Algorithms that appear correct, but are actually incorrect

Answer 2

That's a great question, and the answer has nothing to do with coding. The answer is simple: in order to prove that an algorithm is correct, you will have to... hmmm... prove it!

An algorithm, in many senses is an abstract mathematical creature, which you can mathematically verify under certain assumption and conditions. Maybe the confusing part is that Algorithm can be CODED into a program, but this is only an instantiation of the algorithm, not the algorithm itself. It might be easier to keep the algorithm as a "pseudo code" to be able to define its supposed action, without the disguising details of the specific programing language you use. That way it is easier to believe that an algorithm is indeed a mathematical object (which is run on a mathematical abstraction of a computing device, e.g. on a Turing machine).

Given an algorithm $\textsf{ALG}$ that is supposed to do $X$, you can consider the (mathematical!) theorem "$\textsf{ALG}$ does X" and then try to prove it. To be concrete: assume you write a sorting algorithm $\textsf{SORT}$, then to prove it correct you want to prove the theorem "on input any list of $n$ numbers, $\textsf{SORT}$ outputs a lexicographical ordering of the input"

The details of proving such a theorem vary according to your algorithm and the problem you wish to solve. It's too broad to discuss in this thread.

However, to get some intuition, I would recommend you to look at graph-theory and algorithms for graph-theory. There are many nice algorithms for various graph-related tasks (e.g., performing a BFS search, doing a DFS search, finding a spanning tree, etc) and you will be able to find detailed proofs that those algorithms actually work they way that should (e.g., that BFS covers all the nodes and in the right order; that the spanning tree is indeed a spanning tree, etc.). Since the field of graph-theory is very close to mathematics, these examples might help you to "bridge" you prior knowledge and apply it onto computer-science questions.