9.1 Queuing Theory: Little's Law, M/M/1, and Multi-Server Systems

Queuing theory is the mathematical study of waiting lines, congestion, and service delays in stochastic systems. In industrial and systems engineering, queuing models govern the design and optimization of manufacturing cells, automated material handling networks, maintenance repair depots, telecommunication switches, hospital emergency departments, and logistics supply chains. The NCEES FE Reference Handbook collects essential queuing formulas under its Industrial and Systems Engineering section. To excel on the FE exam, candidates must master Kendall's notation, recognize system stability limits, apply Little's Law across arbitrary system boundaries, calculate operating characteristics for single-server ($M/M/1$) and multi-server ($M/M/s$) systems, and execute economic trade-off analyses between service capacity and waiting costs.

---

1. Elements of a Queuing System

Every queuing system consists of six fundamental structural components:

  Calling                                                              Departing
Population ──> [ Arrival Process (λ) ] ──> [ Queue / Buffer (K) ] ──> [ Service Facility (s servers, μ) ] ──>
                                                Queue Discipline (FCFS)

1. Arrival Process: Characterizes how entities enter the system over time. The standard assumption in stochastic queuing models is a Poisson arrival process with constant mean arrival rate $\lambda$ (customers/units per unit time). Under a Poisson arrival process, the number of arrivals in an interval of length $t$ follows a Poisson distribution with parameter $\lambda t$, and the time between consecutive arrivals (interarrival times) is independent and identically distributed (i.i.d.) according to an Exponential distribution with mean $1/\lambda$.
2. Service Mechanism: Characterizes the duration required to serve an entity. Service times are typically modeled as random variables. In Markovian queues, service duration follows an Exponential distribution with mean service time $1/\mu$, where $\mu$ is the mean service rate per busy server (units served per unit time).
3. Number of Servers ($s$ or $c$): The number of identical, parallel service channels available to process arriving entities. If $s = 1$, arriving entities form a single line for a single server. If $s > 1$, entities typically wait in a single common queue and advance to the first available server.
4. Queue Discipline: The priority rule dictating the order in which waiting entities are selected for service:
   • FCFS / FIFO: First-Come, First-Served (First-In, First-Out). The default baseline for most physical queuing systems.
   • LCFS / LIFO: Last-Come, First-Served (e.g., stack-based inventory retrieval or push-down data stacks).
   • SIRO: Service in Random Order.
   • Priority Disciplines: Entities possess priority attributes. In preemptive priority, a higher-priority arrival interrupts a lower-priority customer currently in service; in non-preemptive priority, the higher-priority customer moves to the head of the queue but waits for the current service to finish.
   • Processor Sharing: All waiting entities share available server capacity equally (common in computer operating systems and web server thread allocation).
5. System Capacity ($K$): The maximum number of entities allowed in the system concurrently (including both entities waiting in queue and entities actively in service). An infinite capacity system ($K = \infty$) can accommodate an arbitrary queue length. In a finite capacity system ($K < \infty$), when the system contains $K$ entities, any subsequent arrival is turned away (blocked or lost), such as in telecommunications call centers.
6. Calling (Source) Population ($N$): The pool of potential customers. An infinite calling population assumes the arrival rate $\lambda$ is unaffected by the number of customers currently in the system. A finite calling population ($N < \infty$) assumes each customer currently in the system reduces the arrival rate of remaining potential customers, typical in internal machine maintenance pools (e.g., a technician maintaining a shop floor of $N = 10$ stamping presses).

---

2. Kendall's Notation

David G. Kendall formalized a standardized shorthand notation to classify queuing systems using the format:

$A \, / \, B \, / \, s \, / \, K \, / \, N \, / \, Z$

Where:

• $A$: Arrival interarrival time distribution
• $B$: Service time distribution
• $s$: Number of parallel servers ($s = 1, 2, 3, \dots$)
• $K$: System capacity (maximum units in queue plus service; omitted if $\infty$)
• $N$: Calling population size (omitted if $\infty$)
• $Z$: Queue discipline (omitted if FCFS/FIFO)

Standard Distribution Codes

• $M$: Markovian (exponential interarrival or service times, possessing the memoryless property; Poisson arrival process)
• $D$: Deterministic / constant time (zero variance, such as an automated transfer indexing line)
• $E_k$: Erlang distribution with shape parameter $k$ (modeling phases of independent exponential operations)
• $G$ or $GI$: General independent distribution (arbitrary mean and variance)

Exam Shorthand Convention: When only three descriptors are provided ($A/B/s$), the system implicitly assumes infinite capacity ($K = \infty$), infinite calling population ($N = \infty$), and FCFS queue discipline ($Z = \text{FCFS}$). Thus, $M/M/1$ denotes $M/M/1/\infty/\infty/\text{FCFS}$.

---

3. Traffic Intensity and Stability Conditions

Single-Server Utilization

For a single-server system ($s = 1$), the server utilization factor $\rho$ represents the fraction of time the server is busy:

$\rho = \frac{\lambda}{\mu}$

• Steady-State Stability Criterion: A queuing system reaches a stationary, steady-state operating equilibrium if and only if: $\rho = \frac{\lambda}{\mu} < 1 \quad \iff \quad \lambda < \mu$
• If $\lambda > \mu$, arrivals outpace service capacity. The queue grows infinitely large over time without bound, and no steady-state distribution exists.
• If $\lambda = \mu$, stochastic variability guarantees that the queue will eventually wander toward infinity; hence $\rho = 1$ is unstable for infinite capacity queues.

Multi-Server Utilization

For a multi-server system with $s$ identical parallel servers each operating at mean rate $\mu$, the total combined service capacity is $s\mu$. The system utilization factor $\rho$ (average utilization per server) is defined as:

$\rho = \frac{\lambda}{s \mu}$

The term $r = \lambda / \mu$ is the offered load (or traffic intensity in Erlangs), representing the expected number of servers required to handle the traffic. The multi-server stability condition requires:

$\rho = \frac{\lambda}{s \mu} < 1 \quad \iff \quad \lambda < s \mu$

---

4. Little's Law and Fundamental Conservation Equations

John D.C. Little formulated what is arguably the most powerful and widely applied relationship in industrial engineering and operations research: Little's Law.

Mathematical Formulation

In any steady-state queuing system where entities do not vanish:

$L = \lambda W$ $L_q = \lambda W_q$

Where:

• $L$: Average number of entities in the system (both waiting in line and actively receiving service)
• $L_q$: Average number of entities waiting in the queue
• $W$: Average total time an entity spends in the system (waiting time plus service time)
• $W_q$: Average time an entity waits in the queue prior to the start of service
• $\lambda$: Long-run average arrival rate to the system

                               │◄─────────────────────── W (System Time) ──────────────────────►│
                               │◄──────── Wq (Queue Time) ────────►│◄──── 1/μ (Service Time) ──►│
                               ┌───────────────────────────────────┬────────────────────────────┐
  Arrivals (λ) ───────────────>│           Queue (Lq)              │        Service (L - Lq)    │───────────> Departures
                               └───────────────────────────────────┴────────────────────────────┘
                               │◄─────────────────────── L (System Units) ─────────────────────►│

Time and Entity Decomposition

The total time an entity spends in the system is the sum of its waiting time in queue and its active service time:

$W = W_q + \frac{1}{\mu}$

Multiplying this entire relationship by the arrival rate $\lambda$ yields the entity decomposition formula:

$\lambda W = \lambda W_q + \frac{\lambda}{\mu} \quad \implies \quad L = L_q + \frac{\lambda}{\mu}$

For a single-server system, $\lambda / \mu = \rho$, so $L = L_q + \rho$. The quantity $\lambda / \mu$ represents the average number of entities currently being served.

Universal Scope of Little's Law

Little's Law is exceptionally robust because it requires no assumptions regarding:

• The arrival distribution (Poisson, deterministic, or arbitrary general arrivals)
• The service distribution (exponential, normal, uniform, or empirical)
• The number of servers ($s = 1, 2, \dots, \infty$)
• The queue discipline (FCFS, LCFS, priority, or processor sharing)

As long as the system is stable and in steady state, Little's Law holds across the entire system, across the queue alone, or across any designated sub-process.

---

5. The $M/M/1$ Queuing Model

The $M/M/1$ queue is the foundational analytical model in queuing theory. It assumes Poisson arrivals (rate $\lambda$), exponential service times (rate $\mu$), 1 server, infinite capacity ($K = \infty$), infinite calling population, and FCFS queue discipline.

State Probabilities

Let $P_n$ denote the steady-state probability that there are exactly $n$ customers in the system ($n = 0, 1, 2, \dots$):

$P_0 = 1 - \rho = 1 - \frac{\lambda}{\mu}$ $P_n = (1 - \rho) \rho^n = \left( 1 - \frac{\lambda}{\mu} \right) \left( \frac{\lambda}{\mu} \right)^n$

• Idle Probability ($P_0$): The probability that the system is completely empty (server is idle).
• Server Busy Probability: $1 - P_0 = \rho = \lambda / \mu$.
• Probability of $n$ or More Customers in System: Summing the geometric series from $n$ to $\infty$: $P(N \ge n) = \sum_{k=n}^{\infty} P_k = \rho^n = \left( \frac{\lambda}{\mu} \right)^n$

Operating Characteristics Formulas

From the state probabilities, the standard performance metrics provided in the NCEES Reference Handbook are derived:

Hyperbolic Growth as $\rho \to 1$: Notice that as server utilization $\rho$ approaches $1.0$, the denominator $(\mu - \lambda) \to 0$. As a result, $L, L_q, W,$ and $W_q$ do not grow linearly—they explode hyperbolically toward infinity. Operating a system at $95%$ utilization produces an average queue nineteen times larger than operating at $50%$ utilization ($L = 0.95 / 0.05 = 19$ vs. $L = 0.50 / 0.50 = 1$).

---

6. Multi-Server Systems: The $M/M/s$ Queuing Model

In an $M/M/s$ queue, entities arrive according to a Poisson process with rate $\lambda$ and are served by $s$ parallel identical servers, each with an exponential service rate $\mu$. If an arriving entity finds any server idle, service begins immediately. If all $s$ servers are busy, arriving entities wait in a single shared queue.

State Probabilities and Idle Probability ($P_0$)

The state-dependent service rate $\mu_n$ reflects that when $n < s$ customers are present, only $n$ servers are active:

$\mu_n = \begin{cases} n\mu, & 0 \le n < s \\ s\mu, & n \ge s \end{cases}$

The probability that the system is completely idle ($P_0$) is:

$P_0 = \left[ \sum_{n=0}^{s-1} \frac{(\lambda / \mu)^n}{n!} + \frac{(\lambda / \mu)^s}{s!} \left( \frac{1}{1 - \frac{\lambda}{s\mu}} \right) \right]^{-1} = \left[ \sum_{n=0}^{s-1} \frac{r^n}{n!} + \frac{r^s}{s!} \left( \frac{1}{1 - \rho} \right) \right]^{-1}$

Where $r = \lambda / \mu$ is the offered load and $\rho = \lambda / (s\mu) < 1$ is the server utilization.

Probability That Arriving Entities Must Wait (Erlang C Formula)

An arrival must wait in the queue if all $s$ servers are occupied ($N \ge s$). This probability is given by the Erlang C delay formula:

$P(N \ge s) = \frac{r^s}{s! (1 - \rho)} P_0 = \frac{(\lambda / \mu)^s}{s! \left( 1 - \frac{\lambda}{s\mu} \right)} P_0$

Multi-Server Operating Characteristics

From $P_0$, the expected queue length is:

$L_q = \frac{r^s \rho}{s! (1 - \rho)^2} P_0 = \frac{(\lambda / \mu)^s \left( \frac{\lambda}{s\mu} \right)}{s! \left( 1 - \frac{\lambda}{s\mu} \right)^2} P_0$

Once $L_q$ is determined, Little's Law immediately yields the remaining metrics without requiring further integration:

$W_q = \frac{L_q}{\lambda}$ $W = W_q + \frac{1}{\mu}$ $L = L_q + \frac{\lambda}{\mu} = \lambda W$

Pooled Capacity Principle: A single queue feeding $s$ parallel servers ($M/M/s$) always outperforms $s$ independent single-server queues ($s \times M/M/1$) each handling an arrival rate of $\lambda / s$. Server pooling eliminates the scenario where one server sits idle while customers wait in line at an adjacent busy server.

---

7. Economic Analysis of Queuing Systems

Industrial engineers frequently evaluate queuing models to balance the economic trade-off between the cost of providing service capacity and the cost of delays.

  Cost ($/hr)
      ▲
      │                 Total Cost Curve TC(s) = s*Cs + L*Cw
      │                     \               /
      │                      \   Optimal   /
      │                       \_   s*    _/
      │                         \_______/             Service Cost: s * Cs
      │                           │                 / 
      │  Waiting Cost: L * Cw     │               /   
      │     \                     │             /     
      │       \                   │           /       
      │         \_                │         /         
      │           \______________ │ _______/          
      └───────────────────────────┴────────────────────────► Number of Servers (s)

Total Expected Cost Formulation

The total expected cost per unit time ($TC$) as a function of the number of servers $s$ is:

$\text{TC}(s) = s \cdot C_s + L \cdot C_w$

Or, if waiting costs apply strictly to entities waiting in the queue rather than those in active service:

$\text{TC}_q(s) = s \cdot C_s + L_q \cdot C_w$

Where:

• $C_s$: Cost of operating each server per unit time (including capital equipment leasing, operator wages, space, and energy)
• $C_w$: Cost of entity waiting time per unit time (including in-process inventory holding cost, expediting charges, customer ill-will, or machine downtime penalties)

Because $s$ is an integer ($s = 1, 2, 3, \dots$), finding the optimal number of servers $s^*$ requires marginal analysis:

1. Compute $\text{TC}(s)$ for increasing integer values of $s$ starting from the minimum stable value ($s > \lambda / \mu$).
2. The optimal configuration $s^*$ satisfies the discrete optimality condition: $\text{TC}(s^* - 1) \ge \text{TC}(s^*) \le \text{TC}(s^* + 1)$

---

8. Step-by-Step Worked Engineering Examples

Worked Example 9.1.1: $M/M/1$ Tool Regrinding Station Analysis

Problem: An automated CNC tooling bay routes dull end-mills to a dedicated regrinding technician. Cutting tools arrive according to a Poisson process at an average rate of $\lambda = 6$ tools per hour. The technician re-sharpens tools with exponentially distributed service times, averaging $8$ minutes per tool.

1. Determine the technician's utilization $\rho$ and the probability that the station is completely idle $P_0$.
2. Compute the average number of tools waiting in the queue ($L_q$) and the average number of tools in the entire station ($L$).
3. Calculate the average waiting time a tool spends in queue ($W_q$) and total turnaround time in the bay ($W$).
4. What is the probability that there are more than 3 tools in the bay?

Solution:

1. Convert Parameters to Consistent Hourly Rates:
   • Arrival rate: $\lambda = 6$ tools/hour
   • Mean service time: $E(S) = 8$ minutes $= 8 / 60 = 2 / 15$ hours
   • Service rate: $\mu = 1 / E(S) = 60 / 8 = 7.5$ tools/hour
   • Utilization factor: $\rho = \frac{\lambda}{\mu} = \frac{6.0}{7.5} = 0.80 \quad (80.0\% \text{ busy})$
   • Since $\rho = 0.80 < 1$, the system is stable in steady state.
   • Idle probability: $P_0 = 1 - \rho = 1 - 0.80 = 0.20 \quad (20.0\% \text{ idle})$
2. Calculate Average Tool Inventory ($L$ and $L_q$):
   • Average number in system: $L = \frac{\lambda}{\mu - \lambda} = \frac{6.0}{7.5 - 6.0} = \frac{6.0}{1.5} = 4.0 \text{ tools}$
   • Average number in queue: $L_q = \frac{\lambda^2}{\mu(\mu - \lambda)} = \frac{6.0^2}{7.5(1.5)} = \frac{36.0}{11.25} = 3.2 \text{ tools}$
   • Check via decomposition: $L = L_q + \rho = 3.2 + 0.80 = 4.0$ tools (Consistent).
3. Calculate Average Turnaround Times ($W_q$ and $W$):
   • Average waiting time in queue: $W_q = \frac{L_q}{\lambda} = \frac{3.2}{6.0} = 0.5333 \text{ hours} = 32.0 \text{ minutes}$
   • Average total time in system: $W = \frac{L}{\lambda} = \frac{4.0}{6.0} = 0.6667 \text{ hours} = 40.0 \text{ minutes}$
   • Check: $W = W_q + 1/\mu = 32.0 + 8.0 = 40.0$ minutes (Consistent).
4. Probability of More Than 3 Tools in System ($N > 3$): $P(N > 3) = P(N \ge 4) = \rho^4 = (0.80)^4 = 0.4096 \approx 41.0\%$

Worked Example 9.1.2: Economic Optimization of Single- vs. Dual-Server Configuration

Problem: A circuit board assembly plant generates test failures requiring diagnostic rework at an average rate of $\lambda = 8$ boards per hour (Poisson arrivals). Management is evaluating two alternatives:

• Alternative A: Keep the current single technician ($s = 1$) who processes $\mu = 10$ boards per hour (exponential service times).
• Alternative B: Add a second identical technician to create an $M/M/2$ pooled rework station ($s = 2$, each with $\mu = 10$ boards per hour).

Technician operating cost is $C_s = $40.00$ per hour per technician. Work-in-process (WIP) waiting cost is $C_w = $60.00$ per hour per board in the system. Which alternative minimizes total hourly cost?

Solution:

1. Evaluate Alternative A ($M/M/1$):
   • $\lambda = 8$ boards/hr, $\mu = 10$ boards/hr, $\rho = 8 / 10 = 0.80$.
   • Average boards in system: $L_A = \frac{\lambda}{\mu - \lambda} = \frac{8}{10 - 8} = \frac{8}{2} = 4.0 \text{ boards}$
   • Total hourly cost for Alternative A: $\text{TC}_A = s \cdot C_s + L_A \cdot C_w = 1(\$40.00) + 4.0(\$60.00) = \$40.00 + \$240.00 = \$280.00 / \text{hour}$
2. Evaluate Alternative B ($M/M/2$):
   • Offered load: $r = \lambda / \mu = 8 / 10 = 0.80$.
   • System utilization: $\rho = \lambda / (s\mu) = 8 / (2 \times 10) = 0.40$.
   • Calculate $P_0$ for $s = 2$: $P_0 = \left[ \sum_{n=0}^{1} \frac{r^n}{n!} + \frac{r^2}{2!} \left( \frac{1}{1 - \rho} \right) \right]^{-1} = \left[ \frac{0.8^0}{0!} + \frac{0.8^1}{1!} + \frac{0.8^2}{2(1 - 0.40)} \right]^{-1}$ $P_0 = \left[ 1 + 0.80 + \frac{0.64}{2(0.60)} \right]^{-1} = \left[ 1.80 + \frac{0.64}{1.20} \right]^{-1} = \left[ 1.80 + 0.5333 \right]^{-1} = \frac{1}{2.3333} = \frac{3}{7} \approx 0.4286$
   • Calculate $L_q$ for Alternative B: $L_q = \frac{r^2 \rho}{2!(1 - \rho)^2} P_0 = \frac{(0.64)(0.40)}{2(0.60)^2} \left( \frac{3}{7} \right) = \frac{0.256}{0.72} \left( \frac{3}{7} \right) = (0.3556)(0.4286) \approx 0.1524 \text{ boards}$
   • Calculate total boards in system $L_B$: $L_B = L_q + \frac{\lambda}{\mu} = 0.1524 + 0.80 = 0.9524 \text{ boards}$
   • Total hourly cost for Alternative B: $\text{TC}_B = 2(\$40.00) + 0.9524(\$60.00) = \$80.00 + \$57.14 = \$137.14 / \text{hour}$
3. Economic Comparison: $\text{Hourly Cost Savings} = \text{TC}_A - \text{TC}_B = \$280.00 - \$137.14 = \$142.86 / \text{hour}$
   • Adding the second technician reduces total expected cost by over $51%$, as the massive drop in WIP waiting cost (saving $$182.86$/hr) far outweighs the additional $$40.00$/hr labor cost.

---

9. NCEES Reference Handbook Tips & Realistic Exam Traps

• Rate vs. Mean Time Trap: The most common error on the FE exam is entering mean service time directly as $\mu$. If a question states that service takes an average of $12$ minutes, $\mu \neq 12$. You must convert time to rate: $\mu = 60 / 12 = 5$ units per hour. Always verify that $\lambda$ and $\mu$ share identical time dimensions (e.g., both per hour or both per minute).
• Queue vs. System Distinction: Carefully read whether the exam asks for:
  • Time in queue ($W_q$) vs. total time in system ($W$). Note that $W = W_q + 1/\mu$.
  • Entities in queue ($L_q$) vs. entities in system ($L$). Note that $L = L_q + \lambda/\mu$.
• Stability Violation Trap: If a question provides $\lambda = 15$ arrivals/hr and a single server with $\mu = 12$ services/hr, $\rho = 15/12 = 1.25 > 1$. Attempting to plug these numbers into the $M/M/1$ formula $L = \lambda / (\mu - \lambda)$ produces $15 / (12 - 15) = -5$ customers, which is physically impossible. When $\rho \ge 1$, the queue is unstable and has no steady-state solution.
• Units in Little's Law: When computing $L = \lambda W$, ensure the time units in $W$ match the rate units in $\lambda$. If $\lambda = 10$ parts/hour and $W = 15$ minutes, convert $W$ to $0.25$ hours before multiplying ($L = 10 \times 0.25 = 2.5$ parts).