Summations

Reading: CLRS A

Why study summations?

1. We saw that a summation came up in the analysis of Insertion-Sort. In general, the running time of a while loop can be expressed as the sum of the running time of each iteration.

2. May come up in solving recurrences.

1 Basic Summations

1.1 Arithmetic series

∑_{k=1}^{n} k = 1+2+3+⋯+n = \frac{n(n+1)}{2} = Θ(n^{2})

How can we prove this? By induction: • Base case: n = 1 ⇒ ∑_{k=1}^{1} 1 = \frac{1(1+1)}{2} = 1

• Assume it holds for n: ∑_{k=1}^{n} k = \frac{n(n+1)}{2} Show it holds for n + 1: ∑_{k=1}^{n+1} k = \frac{(n+1)(n+2)}{2} = \frac{1}{2}n^{2} + \frac{3}{2}n + 1 Proof:

∑_{k=1}^{n+1} k = ∑_{k=1}^{n} k + (n + 1)
= \frac{n(n + 1)}{2} + (n + 1)
= \frac{1}{2}n^{2} + \frac{1}{2}n + n + 1
= \frac{1}{2}n^{2} + \frac{3}{2}n + 1

In general we can prove that ∑_{k=1}^{n} k^{d} = Θ(n^{d+1}) You will prove it in Homework 2.

1.2 Geometric series

∑_{k=0}^{n} x^{k} = 1+x+x^{2} +⋯ x^{n} = \frac{x^{n+1}−1}{x−1} = Θ(x^{n})

Proof by induction:

• Basis: \begin{aligned}n = 1 ⇒ & ∑_{k=0}^{1} x^{k} = 1 + x \\ & \frac{x^{n+1}−1}{x−1} = \frac{x^{2}−1}{x−1} = \frac{(x+1)(x−1)}{(x−1)} = x + 1 \end{aligned}

• Induction:

Assume holds for n: ∑_{k=0}^{n} x^{k} = \frac{x^{n+1}−1}{x−1} Show it holds for n + 1: ∑_{k=0}^{n+1} x^{k} = \frac{x^{n+2}−1}{x−1} Proof:

∑_{k=0}^{n+1} x^{k} = ∑_{k=0}^{n} x^{k} + x^{n+1}
= \frac{x^{n+1} − 1}{x − 1} + x^{n+1}
= \frac{x^{n+1} − 1 + x^{n+1}(x − 1)}{x − 1}
= \frac{x^{n+1} − 1 + x^{n+2} − x^{n+1}}{x − 1}
= \frac{x^{n+2} − 1}{x − 1}

Note: x < 1 ⇒ ∑_{k=1}^{n} k < ∑_{k=1}^{∞} = \frac{1}{1−x} = Θ(1) Example: 1 + \frac{1}{2} + … \frac{1}{2^{n}} = ∑_{k=1}^{n} \frac{1}{2^{k}} = Θ(1)

1.3 Harmonic Series

∑_{k=1}^{n} \frac{1}{k} = 1 + \frac{1}{2} + \frac{1}{3} + ⋯ + \frac{1}{n} = Θ(log n)

We will not do a proof for this one.

2 Bounding Summations

Most of the summations cannot be computed precisely. In these cases we can try to find an asymptotic upper and lower bound for the summation. In the ideal case, we get a Θ() bound. There are a couple of ways to bound summations.

2.1 Induction

Guess the summation bound and try to prove it by induction.

Example: Consider for example that we want to prove that ∑_{k=0}^{n} 3^{k} = O(3^{n}), that is, that ∑_{k=0}^{n} 3^{k} ≤ c · 3^{n} for some c. Proof by induction:

• Basis: n = 1 ⇒ ∑_{k=0}^{1} 3^{x} = 1 + 3 = 4 we want 4 ≤ c · 3 Ok if c > 4/3

• Assume holds for n: ∑_{k=0}^{n} 3^{k} ≤ c · 3^{n} Show holds for n + 1: ∑_{k=0}^{n+1} 3^{k} ≤ c · 3^{n+1} Proof:

∑_{k=0}^{n+1} 3^{k} = ∑_{k=0}^{n} 3^{k} + 3^{n+1}
≤ c · 3^{n} + 3^{n+1}
= c · 3^{n+1}(1/3 + 1/c)
≤ c · 3^{n+1}

Ok if 1/3 + 1/c < 1 which holds if c > 3/2

2.2 Bounding the terms

Use the largest (smallest) value of a term to bound others:

a_{1} + a_{2} + … a_{n} ≤ a_{max} + a_{max} + … + a_{max} = n · a_{max}
a_{1} + a_{2} + … a_{n} ≥ a_{min} + a_{min} + … + a_{min} = n · a_{min}

Example: ∑_{k=1}^{n} k ≤ ∑_{k=1}^{n} n = n · ∑_{k=1}^{n} 1 = n^{2} ⟹ ∑_{k=1}^{n} k = O(n^{2}). ∑_{k=1}^{n} k ≥ ∑_{k=1}^{n} 1 = n ⟹ ∑_{k=1}^{n} k = Ω(n).

2.3 Splitting the summation

Split the summation in two and bound each part (by bounding the terms).

Example: ∑_{k=1}^{n} k = ∑_{k=1}^{n/2−1} k + ∑_{k= \frac{n}{2}}^{n} k ≥ ∑_{k=1}^{n/2−1} 0 + ∑_{k= \frac{n}{2}}^{n} k ≥ (\frac{n}{2})^{2} = Ω(n^{2}).

3 Summation conclusion

We have a sum. We want a (tight) bound for it. What can we do?

• express it in terms of basic summations (for which we know the bound) Example: ∑_{k=1}^{lg n} i =?

• guess a bound and prove it by induction

• obtain upper and lower bounds by bounding terms and/or splitting