Combination with replacement: why is the formula NOT $n^k/n!$?

Question

I found a number of questions on Math Stackexchange that ask why this value is $\binom{n+k-1}{k}$, with answers that explain this or link to someplace that explains this. e.g. Combination with repetitions. Formula for Combinations With Replacement

My question is slightly different.

My reasoning goes: there are $k$ slots to fill with $n$ items, with replacement. If order mattered, we would have $n^k$ ways of doing this.

Now, couldn't we just account for the $k!$ overcounts due to not having order matter? Our answer would then be $n^k/k!$

Where am I going wrong?

Answer 1

Unfortunately, $n$ and $k$ are used differently by different sources, so it is important to first be clear as to which things are identical, and which are distinct.

The formula you quote, implies that $k$ and $n$ respectively refer to identical and distinct things,
which means putting $k$ identical items into $n$ distinct slots.

If both were distinct, there would be $n^k$ ways, e.g. for $n=3, k = 2, 3^2=9$ ways.

Why we can't divide by $k!$ will become crystal clear if we enumerate for this example.

$2-0-0 :$ Place in slots in $\binom31 = 3$ ways

$1-1-0 :$ Place in slots in $\binom32 = 3$ ways

The total of $6$ should be (and is) what the combination with repetitions formula gives, $\binom{2+3-1}{2}$

But if the items were distinct, total # of ways would become $3 + 3\cdot2! = 9 = 3^2$

The point is that the permutations taking the objects to be distinct would vary from pattern to pattern, and so we can't simply divide by $k!$

Answer 2

I was watching some lectures by Joe Blitzstein (https://www.youtube.com/watch?v=FJd_1H3rZGg) that suggested working with the simplest non-trivial example. So I've worked out the answer to my question by working through a couple of examples.

Let my n=3 set be {A, B, C}. I have k=2 slots.

My permutations without replacement are AB, AC, BA, BC, CA, CB. $3 * 2 = 6$ of these. Three ways to pick the first symbol, two ways to pick the second.

Permutations with replacement are AB, AC, BA, BC, CA, CB, AA, BB, CC. $3^2 = 9$ of these. Three ways to pick the first symbol, three ways to pick the second.

My combinations without replacement are: AB, AC, BC. There are $\binom{3}{2} = 3$ of them. Three ways to pick the first symbol, two ways to pick the second, and since there are 2 slots, 2! sets of the same answers to compensate for.

My combinations with replacement are AA, BB, CC, AB, BC, CA. There are 6 of these.

Three ways to pick the first symbol. Three ways to pick the second. 9 permutations. Now, only the ones which don't repeat the same symbol are overcounted. So we only compensate for AB, BC, AC which have been counted twice. AA, BB, CC have not been overcounted by our permutation in the first place.

Thus we do not compensate for overcounts simply by dividing by k!, and our answer is NOT $n^k/k!$

So we do something along the lines of (No. combinations without replacement) + (No. of combinations with replacement excluding combinations without replacement).

But let us take the argument further.

We know the value of the first term.

For the second term, we have: three ways to pick the first symbol, one way to pick the second (the second symbol is conditioned on the first, we have effectively reduced our sample space to outcomes in which the first symbol has been picked)

So for this example, we get 3+3 = 6.

---

In the interest of looking at a simple non-trivial example, let us now look at four symbols G, A, T, C and 3 places to put them.

Permutations without replacement? 4 x 3 x 2 = 24.

Permutations with replacement? $4 ^ 3$ = 64

Combinations without replacement? 4 x 3 x 2 / 3! = (4!) / (1!3!) = $\binom{4}{3}$ = 4

We can list these: GAT, GCT, GAC, ATC.

Combinations with replacement: Let's start with our permutation, which doesn't overcount things like GGG or GCG. It does overcount GGC, GAT, etc.

The reason we can't do a simple divide-by-k! is that different outcomes have been overcounted differently by our permutation. It doesn't overcount strings with 3 repeated symbols the same way as strings with 2 repeated symbols and so on.

We will need to divide each of these cases by the respective overcount factor.

Where there was no replacement, we didn't have this issue, so all we had to do was divide by k!

But with replacement, you need different factors to correct for overcounting different numbers of repeated symbols.