\documentclass[10pt]{article}
\usepackage{graphics,epsfig}

\usepackage{fullpage}

\begin{document}

\title{{\bf Linear Time Selection}\\
\normalsize{(CLRS 9)}}


\date{}

\maketitle

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


% \section{Quick-Sort Review}

% \begin{itemize}
% \item The last two lectures we have considered Quick-Sort:
%         \begin{itemize}
%         \item Divide $A[1...n]$ (using {\sc Partition}) into subarrays
%         $A'=A[1..q-1]$ and $A"=A[q+1...n]$ such that all elements in $A"$
%         are larger than $A[q]$ and all elements in $A'$ are smaller than
%         $A[q]$.
%         \item Recursively sort $A'$ and $A"$.
%         \end{itemize}
% \item We discussed how split point $q$ produced by {\sc Partition}
% only depends on last element in $A$
% \item We discussed how randomization can be used to get good expected
% partition point.
% \item Analysis:
%         \begin{itemize}
%         \item Best case ($q=n/2$): $T(n)=2T(n/2)+\Theta(n) \Rightarrow
%         T(n)=\Theta(n\log n)$.
%         \item Worst case ($q=1$): $T(n)=T(1)+T(n-1)+\Theta(n) \Rightarrow
%         T(n)=\Theta(n^2)$.
%         \item Expected case for randomized algorithm: $\Theta(n\log n)$
%         \end{itemize}
% \end{itemize}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


The selection problem is the following: Given an array A of $n$
elements and a value $i$ ($1 \le i \le n$), find the $i$th smallest
element in an array. This element is called the element of $rank =
i$. 

For simplicity, we assume the elements are distinct.

\vspace{\baselineskip}

\fbox{\parbox{10.5cm}{{\sc Select}($A, i$):  returns  the $i$'th smallest element in A}}

\vspace{\baselineskip}


Note that for $i=1$ we want the {\emph minimum} element, and for $i=n$
we want the \emph{maximum} element. 

The element of rank $i = n/2$ is called the \emph{median}. To be
precise, a set of odd size has one median, the element with rank
$\lceil \frac n 2 \rceil$. A set of even size has two medians, the elements
with rank $\frac n 2$ and $\frac n 2 + 1$.

 
\section*{Selection via  sorting} 

Of course we can find the $i$th smallest element by first sorting $A$,
and then returning the $i$th element in the sorted array.  This
approach will run in $O(n \lg n)$ time.

The question is: can we do better?  Selecting one element seems easier
than sorting so we should be able to do it faster. To add to that
intuition, let's look at some special cases of {\sc Select}($i$)


\begin{itemize}
\item Minimum or maximum can easily be found in $n-1$ comparisons
  \begin{itemize}
  \item Scan through elements maintaining
    minimum/maximum
  \end{itemize}
\item Second largest/smallest element can be found in
  $(n-1)+(n-2)=2n-3$ comparisons
  \begin{itemize}
  \item Find and remove minimum/maximum
  \item Find minimum/maximum
  \end{itemize}
\item Median:
  \begin{itemize}
  \item Using the above idea repeatedly we can find the
    median in time $\sum_{i=1}^{n/2}
    (n-i)=n^2/2-\sum_{i=1}^{n/2}i=n^2/2-(n/2\cdot
    (n/2+1))/2=\Theta(n^2)$
  \item So....this approach does not scale.
  \end{itemize}
\end{itemize} 


Can we design $O(n)$ time algorithm for general $i$?
  




\newpage
\section*{Selection in $O(n)$ expected time via partitioning}

\begin{itemize}

\item The crucial observation is that once we computed a partition (like in Quicksort), that helps us SELECT. 

\item Basically, we recursively look for the element in one of the
  sides of the partition:
  
  \fbox{
    \parbox{7.3cm}{
      {\sc Select}($A,p,r,i$) \\
      \\
      base case: IF $p=r$ then return  $A[p]$ \\

      $q$={\sc Partition}($A,p,r$) \\
      
      \centerline{\includegraphics[height=2cm]{select.pdf}}
      
      $k=q-p+1$ //k is the rank of pivot A[q]\\ 
      IF $i ==  k$ then 
      \begin{itemize}
        \item[] //we found  it!
      \item[] return A[q]
      \end{itemize}
      IF $i < k$ then 
      \begin{itemize}
      \item[] //recurse on the left of pivot 
      \item[] return {\sc Select}($A,p,q,i$)
      \end{itemize}
      ELSE 
      \begin{itemize}
        \item[] //recurse on the right of pivot 
      \item[] return  {\sc Select}($A,q+1,r,i-k$)
      \end{itemize}
    }
  }
  
  \vspace{\baselineskip}
  
  To find the $i$th element in $A$, call  {\sc Select}($A,0,n-1,i$)
  
\item CORECTNESS: This is pretty clever, but why does this work?
  We'll see that for efficiency it is crucial that the recursion
  happens \emph{only on one side} of the partition (not on
  both). This is correct, because once we partitioned, the pivot is in
  the right place , all the elements before it are $\le$ pivot, and
  all the elements to the right of it are $\ge$ pivot. So if we know
  that the rank of the element we are looking for is smaller than the
  rank of the pivot (i.e. $i < k$), then we can ``throw away'' the right
  side of the partition and recurse only on the left side.



\item ANALYSIS: How fast is this, in the worst case?  The trick is that we only recurse on one side.

  \begin{itemize} 
  \item If the partition was perfect ($q=n/2$) we have: 
  \begin{eqnarray*}
    T(n) & = & T(n/2)+n \\
    & = & n+n/2+n/4+n/8+\cdots +1 \\
    & = & \sum_{i=0}^{\log n} \frac{n}{2^i} 
    =  n\cdot \sum_{i=0}^{\log n} (\frac{1}{2})^i 
    \leq  n\cdot \sum_{i=0}^{\infty} (\frac{1}{2})^i 
    =  \Theta(n)
  \end{eqnarray*}
  
\item If the partition was bad:   $T(n)=T(n-1)+n=\Theta(n^2)$
  
\end{itemize} 


\item Thus, the algorithm runs in $O(n^2)$ time in the worst case. We could
argue that on average we get $O(n)$ time, assuming that we ran the
algorithm on a set of inputs such that all input permutations were
equally likely. This assumption is often not true though...


\item But, we could use the randmonized version of partition, {\sc
    Randomized-Partition}, that gives a balanced partition on the
  avarege.  The resulting selection algorithm is referred to as {\sc
    Randomized-Select} and has expected running time $O(n)$ (on any
  set of inputs).

\item Even though a worst-case $O(n)$ selection algorithm exists (see
  below), in practice {\sc Randomized-select} is preferred.

\end{itemize} 




\section*{Selection in $O(n)$ worst-case}

\begin{itemize} 
\item It turns out that we can modify the algorithm and get
  $T(n)=\Theta(n)$ in the worst case

\item The idea is to find a pivot element $x$ such that we always
  eliminate a fraction of the elements:
    
  \vspace{\baselineskip}
  
  \fbox{
    \parbox{11.5cm}{
      
      {\sc Select}($i$)
      
      \begin{itemize}
      \item Divide $n$ elements into groups of 5
      \item Select median of each group ($\Rightarrow \lceil
        \frac{n}{5}\rceil$ selected elements)
      \item Use {\sc Select} recursively to find median $x$ of selected
        elements
      \item Partition {\em all} elements using $x$ as pivot 
        
        \vspace{.5\baselineskip}
        
        \centerline{\includegraphics[height=1.5cm]{detselect.pdf}}
        
      \item Use {\sc Select} recursively to find $i$'th element
        \begin{itemize}
        \item If $i ==  k$ then return $x$
        \item If $i\leq k$ then use {\sc Select}($i$) on the   $k$
          elements before the pivot 
        \item If $i>k$ then use {\sc Select}($i-k$) on the $n-k$
          elements after the pivot 
        \end{itemize}
      \end{itemize}
    }}      
  
  \vspace{\baselineskip}
  
\item ANALYSIS: How long does this take, in the worst case? If $n'$ is
  the maximal number of elements we recurse on in the last step of the
  algorithm, the running time is given by $T(n)=\Theta(n)+T(\lceil
  \frac{n}{5}\rceil)+\Theta(n)+T(n')$
  
  
\item Estimation of $n'$:
  \begin{itemize}
  \item Consider the following figure of the groups of 5 elements
  \item An arrow between element $e_1$ and $e_2$ indicates that
    $e_1>e_2$
  \item The $\lceil \frac{n}{5} \rceil$ selected elements are
    drawn solid ($q$ is median of these)
  \item Elements $>q$ are shown inside the box
 
  
  \vspace{\baselineskip}
  
  \centerline{\includegraphics[height=3.5cm]{fraction.pdf}}
  
  \vspace{\baselineskip}

\item Number of elements $>q$ is at least
  $3(\frac{1}{2}\lceil\frac{n}{5}\rceil -2)\geq \frac{3n}{10}-6$
  \begin{itemize}
  \item We get 3 elements from each of
    $\frac{1}{2}\lceil\frac{n}{5}\rceil$ columns except
    possibly the one containing $q$ and the last one.
  \end{itemize}
\item Similarly, the number of elements $<q$ is at least  than
  $\frac{3n}{10}-6$
  
  $\Downarrow$
  
  We recurse on at most $n'=n-(\frac{3n}{10}-6)=\frac{7}{10}n+6$
  elements
\end{itemize}


\item So {\sc Select}($i$) runs in time $T(n)=\Theta(n)+T(\lceil
\frac{n}{5}\rceil)+T(\frac{7}{10}n+6)$
\item It can be shown that the solution to $T(n)=n+T(\lceil \frac{n}{5}\rceil)+T(\frac{7}{10}n+6)$ is  $T(n) = O(n)$.

%         \begin{itemize}
%         \item Guess $T(n)\leq cn$
%         \item Induction:
%         \begin{eqnarray*}
%         T(n) & =    & n+T(\lceil \frac{n}{5}\rceil)+T(\frac{7}{10}n+6) \\
%              & \leq & n+c\cdot \lceil \frac{n}{5}\rceil+c\cdot
%                 (\frac{7}{10}n+6) \\
%              & \leq & n+c\frac{n}{5}+c+\frac{7}{10}cn+6c \\
%              & =    & \frac{9}{10}cn+n+7c \\
%              & \leq & cn
%         \end{eqnarray*}
%         If $7c+n\leq \frac{1}{10}cn$ which can be satisfied (e.g. true for
%         $c=20$ if $n>140$)




\item Comments:  
  \begin{itemize} 
  \item It is important that we chose every 5'th element, not all
    other choices will work.
    
  \item This algorithm gives $\sim 16n$ comparisons. Best know $\sim
    2.95n$. Best lower bound $>2n$.
  \end{itemize} 

\end{itemize} 




\section*{Selection and Quicksort}  

\begin{itemize} 
\item Recall that the running time of Quicksort depends on how good the partition is 
  \begin{itemize}
  \item Best case ($q=n/2$): $T(n)=2T(n/2)+\Theta(n) \Rightarrow
    T(n)=\Theta(n\log n)$.
  \item Worst case ($q=1$): $T(n)=T(1)+T(n-1)+\Theta(n) \Rightarrow
    T(n)=\Theta(n^2)$.
  \end{itemize}
  
\item How balanced the partition is depends on well the chosen pivot
  splits the input

\item We could call SELECT to find the median in $O(n)$ time, and use
  the median as pivot in {\sc Partition}.
  \begin{itemize}
  \item We could just exchange $e$ with last element in $A$ in
    beginning of {\sc Partition} and thus make sure that $A$ is  always
    partition in the middle
  \end{itemize}

\item Thus we could make quick-sort run in $\Theta(n\log n)$ time
  worst case.
   

\item In practice, although a worst-case time $\Theta(n \lg n)$
  algorithm is possible, it is not used because the constant factors
  in {\sc SELECT} are too high. {\sc Randomized-Quicksot} remains the
  fastest sort in practice.
\end{itemize} 



\end{document}
