5.1 Algorithm Efficiency: Linear, Quadratic, Exponential, and Logarithmic Growth

What this competency asks

Under limits of computing, ETS asks you to identify and compare algorithms that are linear, quadratic, exponential, or logarithmic. One ETS sample question describes an algorithm that generates every possible ordering of a salesperson's cities and evaluates each one. The running time is factorial in the number of cities. Your job is to look at an algorithm and name its growth class.

Why we count steps instead of seconds

Measured running time depends on the processor, the language, and other programs running at the same time. To compare algorithms, computer scientists count how many basic operations (comparisons, assignments, arithmetic) are performed as a function of the input size n, and ask how that count grows.

Big-O notation gives an upper bound on growth. Formally, f(n) is O(g(n)) if there are constants c > 0 and n₀ such that f(n) ≤ c · g(n) for all n ≥ n₀. In practice, two rules do most of the work:

1. Drop constant factors. 500n and 2n are both O(n).
2. Drop lower-order terms. For f(n) = 3n² + 50n + 1000, the n² term dominates as n grows. At n = 10,000 it is 300,000,000 of the 300,501,000 total operations, about 99.8%. So f(n) is O(n²).

Related notations: Ω (big-Omega) is a lower bound and Θ (big-Theta) is a tight bound (both upper and lower). These are bounds on functions. Separately, we analyze an algorithm's best case, average case, and worst case. For example, linear search is Θ(1) in its best case and Θ(n) in its worst case.

The growth classes

What happens when n doubles

A related shortcut: for an exponential algorithm, adding just one item doubles the work.

Concrete growth

At a million items, an O(n log n) sort does about 20 million basic steps, while an O(n²) sort does about a trillion, roughly 50,000 times as many.

Reading growth from code

Rule 1: sequential blocks add, and the largest term wins

for ( int i ← 0; i < n; i ← i + 1 )
    print i
end for
for ( int i ← 0; i < n; i ← i + 1 )
    for ( int j ← 0; j < n; j ← j + 1 )
        print i + j
    end for
end for

The first loop is O(n) and the nested loops are O(n²). The total is O(n + n²) = O(n²).

Rule 2: nested loops multiply

An outer loop of n passes containing an inner loop of m passes runs the body n × m times. When m = n, that is O(n²).

Rule 3: dependent (triangular) loops are still quadratic

int count ← 0
for ( int i ← 1; i ≤ n; i ← i + 1 )
    for ( int j ← i; j ≤ n; j ← j + 1 )
        count ← count + 1
    end for
end for

The inner loop runs n, then n − 1, …, down to 1 time: n(n + 1)/2 total. Dropping the constant ½ and the lower-order term leaves O(n²).

Rule 4: a multiplied or divided loop variable is logarithmic

int p ← 1
while ( p < n )
    p ← p * 2
end while

p takes the values 1, 2, 4, 8, …, so the loop runs about log₂ n times: O(log n). The same holds for a loop that halves a value until it reaches 1.

Rule 5: branching recursion is often exponential

A recursive procedure that makes two recursive calls on inputs only slightly smaller (for example, n − 1 and n − 2) builds a call tree that roughly doubles at each level, which is exponential (Section 6.3).

Space as well as time

The same notation describes memory use. Finding the maximum of a list uses O(1) extra space: one variable. Merge sort uses O(n) extra space for its temporary arrays. Recursion uses stack space proportional to its depth. ETS lists space limitations alongside time limitations (Section 5.2), so read whether a question asks about time or memory.

Common data-structure costs