Is there a meaningful difference between O(1) and O(log n)?

Question

A computer can only process numbers smaller than say $2^{64}$ in a single operation, so even an $O(1)$ algorithm only takes constant time if $n<2^{64}$. If I somehow had an array of $2^{1000}$ elements to process, even an $O(1)$ operation such as an index lookup will start to take longer as the index has to be calculated in multiple operations. I think it will take at least $O(\log n)$.

Similarly even if an algorithm has $O(\log n)$ complexity, $\log n$ cannot possibly grow larger than about a hundred, so could be ignored as no larger than a small constant.

So, is it really meaningful to treat $O(1)$ and $O(\log n)$ as different?

The same applies of any difference of $\log n$, like between $O(n)$, $O(n\log n)$ and $O(n/\log n)$.

Answer 1

There are actually all sorts of cases where $\log n$ gets way bigger than 100. For example, if you're working with variable-length integers - say, for cryptography - $\log n$ represents the number of bits in the integer, which can get up to 4096 or higher in practice. Those bits will be spread out over many machine words, so the $\log n$ term refers to how many machine words the number occupies and therefore definitely has an impact on the runtime.

Similarly, $O(1)$ does not mean "constant time only if $n < 2^{64}$." Yes, the computer has a fixed word size, but that doesn't mean that an $O(1)$-time algorithm given a gigantic input can't run in a fixed amount of time. Taking large integers as an example, consider the problem of checking whether an integer is even or odd. That just requires you to look at the last digit, which takes time $O(1)$ even if the number is spread out over thousands of machine words.

The last detail is that typically, we assume that we work with a machine model where the word size scales with the log of the size of the input. This allowance a to safely assume that you can do array accesses in time $O(1)$ because array indices are assumed to fit into a constant number of machine words.

Answer 2

There are several answers to your question. The first is: on such a finite machine, asymptotic notation makes no sense at all, and all algorithms have finitely bounded running time. That's a rather boring model, though.

so could be ignored as no larger than a small constant.

Why is 100 small? If my computer booted 100 times longer than it does I would be quite upset. Moden algorithm engineering fights for percents, and in scientific computing probably for fractions of that!

Also, don't forget that your logarithmic term may have another constant factor assigned to it. What if the running time is $\sim 10^{-10} \cdot n + 10^6 \cdot \log n$?

The same applies of any difference ...

There are people who dismiss polylogarithmic factors, probably for similar reasons than you do. They use the soft Landau notation to hide them away conveniently.

Answer 3

For not very large n, log n can be around 20. A factor log n matters about as much as it matters to have a computer built in 2016 and not one built in 1999.

On the other hand, O (n log n) can be faster than O (n) for practical n if the constant factor in the O (n) algorithm is say 50 times larger than for the O (n log n) algorithm.

A factor log n for say n ≥ 100 cannot be ignored if everything else is equal. Sometimes everything else is equal, sometimes it isn't.

Answer 4

Saying $O(\log{}n)$ can't get above about 100 doesn't make sense when talking about about Big-O notation. The "$\log$" in $O(\log{}n)$ doesn't refer to any specific base. Big-O is about how the computational cost grows as $n$ gets larger, within a constant factor. Changing the base just changes that constant factor, so I can make the argument that $O(\log{}n)$ can be as arbitrarily large (or small) as you want to make it. But the growth stays the same! That's what's important when discussing Big-O. If you double $n$, the cost still increases in an $O(\log{}n)$ algorithm, unlike an $O(1)$ algorithm, but much less than the cost increase in an $O(n)$ algorithm.

The cool thing about $O(\log{}n)$ is that increasing the problem size makes it "easier" to increase the problem size even further. Increasing the problem size from $10^2$ to $10^3$ will increase the computational cost by $1/2$, but going from $10^3$ to $10^4$ will only increase the cost by $1/3$. So for very large $n$ the additional cost keeps decreasing, but it never reaches $O(1)$.

So no, $O(1)$ and $O(\log{}n)$ are not the same. In practice, which is faster will likely be problem dependent, but if it gets large enough $O(1)$ will win.

Answer 5

There can be. The obvious case is when $n$ is really huge. But there is another case which is very common. It depends on the constants involved. Let me give an example from data structure: a red-black tree has operations which are $O(\lg n)$. This seems good enough if the constants are small. However think about the situation where we have a huge amount of data in each node and we cannot keep the whole tree in the memory and have to use hard disks to store it. The access time to hard disk is huge compare to memory. Every access counts. If we have to access $\lg n$ times to the disk to get the value of the block we want it would be a huge drop in performance. That is the reason for data structure like B-trees. Not only $\Theta(\lg n)$ and $\Theta(1)$ differ a lot in practice even $2$ access and $3$ access can have a significant effect.

Answer 6

Consider a function:

int foo(int n)
{
    //mysterious code details
}

It can be implemented in a few different ways, and you've managed to come up with an easy algorithm in O(log(n)) but you think there might be a tricky one in O(1). You choose the easy path because there's not much of a difference, right?

Well, you're not looking at the big picture. Sean, the guy in the cubicle next to you, likes your function foo and decides he's going to use it in his code:

void main()
{
    print(bar(9999999999999));
}

int bar(int n)
{
    int sum = 0;
    for (int i = 0; i < n; i++)
    {
        for (int j = 0; j < n; j++)
        {
            for (int k = 0; k < n; k++)
            {
                for (int l = 0; l < n; l++)
                {
                    sum += foo(i) * foo(j) * foo(k) * foo(l);
                }
            }
        }
    }
    return sum;
}

Do you think it matters now?

Answer 7

The assumption "A computer can only process numbers smaller than say $2^{64}$" does not hold in the frame of asymptotic behavior computation. $n$ is considered unbounded and the algorithm runs on a theoretical computer with unlimited resources.

It is not the peculiar values of $f(n)$ that matter, but how the function grows with $n$.

If you take the finiteness of the representation into account, all algorithms are running in time $O(1)$, even the super-duper-exponential ones.

And in practice, an $O(1)$ algorithm can run much slower than a $O(\log(n))$ one.

May sound weird, but that's the way it is.

---

Final remark:

On modern machines, the access times to memory really depends on the number of elements. This effect has an effect much larger than $\log n$, so that relying too much on the idealized computer model is misleading.