\documentclass[10pt]{article}
\usepackage{fullpage}
\usepackage{graphics,epsfig}


\begin{document}

\title{{\bf Sorting Lower Bound}\\
\normalsize{(CLRS 8.1)}}

\date{}

\maketitle

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%




We have seen several sorting algorithms. All have worst-case running
time at least $\Omega(n \lg n)$. The natural question to ask is: {\bf
  Can we do better than $\Theta(n \lg n)$ in the worst-case?} In other
words, is there a sorting algorithm whose worst-case running time is
asymptotically better than $\Theta(n \lg n)$?  

This is a hard question. If we did not find a faster algorithm, it
does not mean that there isn't one. Can we say that it is impossible
to sort faster than $\Omega(n \lg n)$ in the worst case? If we could,
then this would be what's called a \emph{lower bound}.

\vspace{\baselineskip}

\fbox{
  \parbox{15cm} {
    A \emph{lower bound} for a problem is the worst-case running time of
    the best possible algorithm for that problem.
  }
}

\vspace{\baselineskip}

To prove a lower bound of $\Omega(n \lg n)$ for sorting, we would have
to prove that no algorithm, however smart, could possibly be faster,
in the worst-case, then $\Theta(n \lg n)$.

BUT, how to prove a lower bound, without going through all possible
algorithms?!?!

To approach this type of problem we have to go back to what is an
algorithm. The power of an algorithm depends on the primitive
instructions we allow, which in turn depend on the machine model.

$\downarrow$

How fast we can solve a problem depends on the model of
computation. The more powerful the machine model, the smaller the
lower bound.

(Ultimately, if our model allowed \emph{sort} as a primitive
operation, then we could sort in one instruction, hence constant
time!!)
%
Now back to sorting:


\section*{The Comparison Model}

\begin{itemize}
\item All sorting algorithms we have seen so far use only comparisons
  to gain information about the input.

\item We will prove that such algorithms have to perform $\Omega(n\log
  n)$ comparisons.
\item To prove bound, we need {\em formal model}:

\fbox{
  \parbox{15cm}{
    {\bf Comparison  (or Decision tree) Model} 
    \begin{itemize}
    \item Binary tree where each internal node is labeled $a_i \leq
      a_j$ ($a_i$ is the $i$'th input element)
    \item Execution corresponds to root-leaf path     
      \begin{itemize}
      \item at each internal node a comparison $a_i \leq a_j$ is
        performed and branching
      \end{itemize}
    \item Leaf contains result of computation
    \end{itemize}
  }
}

\item Decision tree model corresponds to algorithms where only
  comparisons can be used to gain knowledge about input.
  
\item Any algorithm has a corresponding decision tree (just ignore
  everything else but the comparisons made by the algorithm); and any
  decision tree corresponds to an algorithm.


\item Example: Decision tree for sorting 3 elements. 
  
  \vspace{.5\baselineskip}
  
  {\centering \includegraphics[width=6cm]{decision_tree.pdf}}


\item The algorithm must be able to sort any possible input (of n
  elements, for any n).  Put differently, for each input the algorithm
  must be able to compute the permutation of the input that gives
  sorted order. 

\item An execution of the algorithm corresponds to a root-to-leaf path
  in the tree; it corresponds to identifying the permulation that
  sorts the input.

\item Each leaf of the tree represents a permutation (that corresponds
  to sorted order for that path).

\item Worst case number of comparisons performed corresponds to
  maximal root-leaf path (=height of tree).

\item Therefore lower bound on height $\Rightarrow $ lower bound on
  sorting.

\item If we could prove that any decision tree on $n$ inputs must have
  height $\Omega(n \lg n)$ $\Rightarrow $ any sorting algorithm (that
  uses only comparisons) must make at least $\Omega(n \lg n)$
  comparisons in the worst case.

\end{itemize} 


\section*{Sorting Lower Bound in the Comparison Model}

\fbox{
\parbox{12cm}{
{\bf Theorem:} Any decision tree sorting $n$ elements has height
$\Omega(n\log n)$.
}}

\noindent 
Proof: 
\begin{itemize}
\item Assume elements are the (distinct) numbers 1 through $n$
\item There must be $n!$ leaves (one for each of the $n!$
  permutations of $n$ elements)
\item Tree of height $h$ has at most $2^h$ leaves
\item Thus the height must be so that $2^h \ge n!$ and we get 

  \begin{eqnarray*}
    h \geq & \log (n!) \\
%    & = & \log (n(n-1)(n-2)\cdots (2)) \\
 %  & = & \log n + \log (n-1) + \log (n-2) +\cdots + \log 2 \\
    & = & \sum_{i=2}^n \log i \\
    & = & \sum_{i=2}^{n/2 - 1} \log i + \sum_{i=n/2}^n \log i \\
    & \geq & 0 +  \sum_{i=n/2}^n \log \frac{n}{2} \\
    & = & \frac{n}{2} \cdot \log \frac{n}{2} \\
    & = & \Omega (n\log n) \\
  \end{eqnarray*}
\end{itemize}

\fbox{
  \parbox{12cm}{ {\bf Corollary:} Any sorting algorithm that uses only
    comparisons takes $\Omega(n\log n)$ in the worst case.  }}
        
\end{document}
