Sorting Lower Bound

(CLRS 8.1)

We have seen Θ(n log n) and Θ(n^{2}) sorting algorithms:( bubblesort), insertion sort, (selection sort), mergesort, quicksort. all have worst-case running time Ω(n lg n).

Can we do better than n lg n? If we did not find a faster algorithm, it does not mean that there isn’t one..

Can we say that it is impossible to sort faster than Ω(n lg n) in the worst case? If we could, then this would be what it’s called a lower bound.

A lower bound for a problem is the worst-case running time of the best possible algorithm for that problem.

To prove a lower bound of Ω(n lg n) for sorting, we would have to prove that no algorithm, however smart, could possibly be faster, in the worst-case, then n lg n.

So, how to prove a lower bound, without going through all possible algorithms?!?! To approach this type of problem we have to go back and remind ourselves whhat is an algorithm. The

power of an algorithm depends on the primitive instructions we allow, which in turn depend on the machine model.

↓ How fast we can solve a problem depends on the model of computation. The more powerful the machine

model, the smaller the lower bound.

(Ultimately, if our model allowed sort as a primitive operation, then we could sort in one instruction, hence constant time!!)

Coming back to sorting...

The Comparison Model

• All sorting algorithms that we have seen so far use only comparisons to gain information about the input.

• We will prove that such algorithms have to do Ω(n log n) comparisons

• To prove bound, we need formal model

Comparison (or Decision tree) Model

– Binary tree where each internal node is labeled a_{i} ≤ a_{j} (a_{i} is the i’th input element)

– Execution corresponds to root-leaf path

∗ at each internal node a comparison a_{i} ≤ a_{j} is performed and branching made

– Leaf contains result of computation

• Note: Decision tree model corresponds to algorithms where only comparisons can be used to gain knowledge about input • Example: Decision tree for sorting 3 elements.

– a leaf contains permutation giving sorted order.

• Worst case number of comparisons performed corresponds to maximal height of tree ; therefore lower bound on height ⇒ lower bound on sorting.

Sorting Lower Bound in the Comparison Model

Theorem: Any decision tree sorting n elements has height Ω(n log n)

Proof:

– Assume elements are the (distinct) numbers 1 through n

– There must be n! leaves (one for each of the n! permutations of n elements)

– Tree of height h has at most 2^{h} leaves

2^{h} ≥ n! ⇒ h ≥ log(n!)
= log(n(n − 1)(n − 2) ⋯ (2))
= log n + log(n − 1) + log(n − 2) + ⋯ + log 2
= ∑_{i=2}^{n} log i
= ∑_{i=2}^{n/2−1} log i + ∑_{i=n/2}^{n} log i
≥ 0 + ∑_{i=n/2}^{n} log \frac{n}{2}
= \frac{n}{2} · log \frac{n}{2}
= Ω(n log n)