2.3 Linear Algebra: Matrix Operations, Systems of Equations, and Vectors

Linear algebra is the foundational mathematical language of industrial systems engineering. It forms the computational mechanics beneath the Simplex method in Linear Programming (LP), discrete-time Markov chains in queuing and reliability, Input-Output Leontief models in supply chain logistics, and 3D vector statics in material handling equipment. Mastery of matrix properties, determinants, inverses, eigenvalues, and vectors is a high-yield competency on the NCEES FE exam.

---

1. Matrix Operations and Fundamental Algebraic Properties

Matrix Conformability and Multiplication

A matrix of dimensions $m \times n$ possesses $m$ rows and $n$ columns. Two matrices $A$ and $B$ can be added or subtracted if and only if their dimensions are identical ($m \times n$).

Matrix multiplication $C = A B$ is conformable if and only if the number of columns in $A$ equals the number of rows in $B$:

$$(m \times k) \times (k \times n) \implies (m \times n)$$

Each entry $c_{ij}$ in the product matrix is the dot product of row $i$ of matrix $A$ and column $j$ of matrix $B$:

$$c_{ij} = \sum_{r=1}^k a_{ir} b_{rj}$$

Core Properties of Matrix Algebra

• Non-Commutative: In general, $A B \neq B A$. Even if both $A B$ and $B A$ exist and share identical square dimensions, their product matrices are typically unequal.
• Associative: $A(B C) = (A B)C$ and $k(A B) = (k A)B = A(k B)$.
• Distributive: $A(B + C) = A B + A C$ and $(A + B)C = A C + B C$.
• Identity Matrix ($I$): A square matrix with ones on the main diagonal and zeros elsewhere, satisfying $A I = I A = A$.

Matrix Transpose and Inverse Properties

The transpose $A^T$ is formed by swapping the row and column indices of $A$ (i.e., $(A^T)_{ij} = a_{ji}$):

1. $(A^T)^T = A$
2. $(A + B)^T = A^T + B^T$
3. Reversal of Product Transpose: $(A B)^T = B^T A^T$ (order reverses!)
4. A square matrix is symmetric if $A^T = A$, and skew-symmetric if $A^T = -A$.

A square matrix $A$ has an inverse $A^{-1}$ if and only if its determinant is non-zero ($\det(A) \neq 0$, non-singular):

$$A A^{-1} = A^{-1} A = I$$

• Reversal of Product Inverse: $(A B)^{-1} = B^{-1} A^{-1}$ (order reverses!)
• Inverse Transpose: $(A^T)^{-1} = (A^{-1})^T$
• Orthogonal Matrix: A matrix where $A^T = A^{-1}$, which means $A^T A = I$.

---

2. Determinants, Inverses, and Scaling Rules

Calculating Determinants

• $2 \times 2$ Matrix: $$\det \begin{bmatrix} a & b \\ c & d \end{bmatrix} = ad - bc$$
• $2 \times 2$ Matrix Inverse: $$\begin{bmatrix} a & b \\ c & d \end{bmatrix}^{-1} = \frac{1}{ad - bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$$
• $3 \times 3$ Matrix (Cofactor Expansion along Row 1): $$\det \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix} = a_{11}(a_{22} a_{33} - a_{23} a_{32}) - a_{12}(a_{21} a_{33} - a_{23} a_{31}) + a_{13}(a_{21} a_{32} - a_{22} a_{31})$$

Exam Trap: Remember the alternating signs of the cofactors: $+ - +$ across the first row. The middle term MUST carry a minus sign: $-a_{12} M_{12}$.

Critical Determinant Theorems Tested on the FE

---

3. Systems of Linear Equations: Existence, Uniqueness, and Methods

A system of $m$ linear equations in $n$ unknowns is written in matrix form as:

$$A \mathbf{x} = \mathbf{b}$$

Where $A$ is the $m \times n$ coefficient matrix, $\mathbf{x}$ is the $n \times 1$ vector of decision variables, and $\mathbf{b}$ is the $m \times 1$ right-hand-side requirements vector.

The Rouché-Capelli Theorem (Rank Conditions)

To determine the solvability of $A \mathbf{x} = \mathbf{b}$, analyze the rank of the coefficient matrix $A$ and the augmented matrix $[A \mid \mathbf{b}]$:

Linear System: A x = b
 ├── rank(A) < rank([A | b]) ──────> Inconsistent (No solution)
 └── rank(A) = rank([A | b]) = r
      ├── r = n (Full rank) ───────> Consistent: Unique solution
      └── r < n (Underdetermined) ──> Consistent: Infinite solutions (n - r free variables)

Solution Techniques

1. Matrix Inversion: If $A$ is square ($n \times n$) and $\det(A) \neq 0$: $$\mathbf{x} = A^{-1} \mathbf{b}$$
2. Gaussian Elimination: Applies elementary row operations to reduce $[A \mid \mathbf{b}]$ to upper triangular row echelon form, followed by back substitution. The standard algorithm used in computer solvers.
3. Cramer's Rule: For a square system where $\det(A) \neq 0$, the $i$-th unknown $x_i$ is computed directly as: $$x_i = \frac{\det(A_i)}{\det(A)}$$ Where $A_i$ is the matrix formed by replacing the $i$-th column of $A$ with the right-hand-side vector $\mathbf{b}$.

---

4. Eigenvalues, Eigenvectors, and System Dynamics

The Characteristic Equation

For an $n \times n$ matrix $A$, a scalar $\lambda$ is an eigenvalue and a non-zero vector $\mathbf{v}$ is its corresponding eigenvector if:

$$A \mathbf{v} = \lambda \mathbf{v} \quad \iff \quad (A - \lambda I)\mathbf{v} = \mathbf{0}$$

Because $\mathbf{v} \neq \mathbf{0}$, the system has non-trivial solutions if and only if the coefficient matrix is singular:

$$\det(A - \lambda I) = 0$$

This yields the $n$-th degree characteristic polynomial. Its roots are the eigenvalues $\lambda_1, \lambda_2, \dots, \lambda_n$.

Fundamental Eigenvalue Invariants

• Trace Relationship: The sum of the eigenvalues equals the trace (sum of main diagonal elements) of matrix $A$: $$\sum_{i=1}^n \lambda_i = \operatorname{tr}(A) = \sum_{i=1}^n a_{ii}$$
• Determinant Relationship: The product of the eigenvalues equals the determinant of $A$: $$\prod_{i=1}^n \lambda_i = \det(A)$$

These two invariants provide an instantaneous check on the FE exam: if the eigenvalues you calculate do not sum to the trace of $A$ or multiply to $\det(A)$, an arithmetic error occurred.

---

5. Vector Operations: Dot Product, Cross Product, and Projections

Let vectors $\mathbf{u} = u_x \mathbf{i} + u_y \mathbf{j} + u_z \mathbf{k}$ and $\mathbf{v} = v_x \mathbf{i} + v_y \mathbf{j} + v_z \mathbf{k}$ with angle $\theta$ between them:

Dot Product (Scalar Product)

$$\mathbf{u} \cdot \mathbf{v} = u_x v_x + u_y v_y + u_z v_z = \|\mathbf{u}\| \|\mathbf{v}\| \cos \theta$$

• Orthogonality Test: $\mathbf{u} \cdot \mathbf{v} = 0 \iff \mathbf{u} \perp \mathbf{v}$ (non-zero vectors are perpendicular).
• Vector Magnitude: $\|\mathbf{u}\| = \sqrt{\mathbf{u} \cdot \mathbf{u}} = \sqrt{u_x^2 + u_y^2 + u_z^2}$.
• Angle Between Vectors: $\cos \theta = \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\| \|\mathbf{v}\|}$.
• Scalar Component of $\mathbf{u}$ on $\mathbf{v}$: $\operatorname{comp}_{\mathbf{v}}(\mathbf{u}) = \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{v}\|}$.
• Vector Projection of $\mathbf{u}$ on $\mathbf{v}$: $$\operatorname{proj}_{\mathbf{v}}(\mathbf{u}) = \left( \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{v}\|^2} \right) \mathbf{v}$$

Cross Product (Vector Product in 3D)

The cross product $\mathbf{u} \times \mathbf{v}$ produces a vector perpendicular to both $\mathbf{u}$ and $\mathbf{v}$ according to the right-hand rule:

$$\mathbf{u} \times \mathbf{v} = \det \begin{bmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ u_x & u_y & u_z \\ v_x & v_y & v_z \end{bmatrix} = (u_y v_z - u_z v_y)\mathbf{i} - (u_x v_z - u_z v_x)\mathbf{j} + (u_x v_y - u_y v_x)\mathbf{k}$$

• Anti-Commutative: $\mathbf{u} \times \mathbf{v} = -(\mathbf{v} \times \mathbf{u})$.
• Magnitude: $\|\mathbf{u} \times \mathbf{v}\| = \|\mathbf{u}\| \|\mathbf{v}\| \sin \theta$, which equals the area of the parallelogram formed by $\mathbf{u}$ and $\mathbf{v}$.
• Parallelism Test: $\mathbf{u} \times \mathbf{v} = \mathbf{0} \iff \mathbf{u}$ and $\mathbf{v}$ are collinear (parallel).

---

6. Industrial Engineering Applications: LP Simplex and Markov Chains

Linear Programming (LP) Simplex Matrix Mechanics

In an LP with standard constraints $A \mathbf{x} = \mathbf{b}, \mathbf{x} \ge \mathbf{0}$, the matrix $A$ is partitioned into basic columns $B$ ($m \times m$) and non-basic columns $N$ ($m \times (n - m)$):

$$[B \mid N] \begin{bmatrix} \mathbf{x}_B \\ \mathbf{x}_N \end{bmatrix} = \mathbf{b} \quad \implies \quad B \mathbf{x}_B + N \mathbf{x}_N = \mathbf{b}$$

• Basic Feasible Solution (BFS): Set non-basic variables to zero ($\mathbf{x}_N = \mathbf{0}$), giving: $$\mathbf{x}_B = B^{-1} \mathbf{b}$$
• Simplex Multipliers (Dual Variables / Shadow Prices): With objective vector partitioned as $\mathbf{c} = [\mathbf{c}_B \mid \mathbf{c}_N]$, the shadow price vector $\boldsymbol{\pi}$ is: $$\boldsymbol{\pi} = \mathbf{c}_B B^{-1}$$
• Reduced Costs (Relative Profits): For entering variable selection: $$\bar{c}_j = c_j - \boldsymbol{\pi} \mathbf{a}_j = c_j - \mathbf{c}_B B^{-1} \mathbf{a}_j$$

Discrete-Time Markov Chains (DTMC)

A discrete Markov chain models system state transitions across time steps. The square transition probability matrix $P = [p_{ij}]$ satisfies $p_{ij} \ge 0$ and row sum $\sum_{j} p_{ij} = 1$.

• $n$-Step Transitions: The transition probabilities after $n$ operational steps are given by the matrix power $P^n$.
• Steady-State (Stationary) Probability Distribution ($\boldsymbol{\pi}$): For an irreducible and ergodic chain, the long-run probability vector $\boldsymbol{\pi} = [\pi_1, \pi_2, \dots, \pi_k]$ satisfies: $$\boldsymbol{\pi} P = \boldsymbol{\pi} \quad \text{and} \quad \sum_{i=1}^k \pi_i = 1$$ Notice that this is a linear system: $\boldsymbol{\pi}(P - I) = \mathbf{0}$. It represents finding the normalized left eigenvector of matrix $P$ corresponding to eigenvalue $\lambda = 1$.

---

7. Step-by-Step Worked Engineering Examples

Worked Example 2.3.1: Steady-State Availability of a Machine Workcell

Problem: An automated CNC mill operates in two operational states: State 1 (Operational) and State 2 (Under Maintenance). Hourly transition probabilities are defined by matrix $P$:

$$P = \begin{bmatrix} 0.8 & 0.2 \\ 0.4 & 0.6 \end{bmatrix}$$

Compute the long-run stationary probability $\pi_1$ that the machine is operational.

Solution:

1. Formulate the steady-state equation $\boldsymbol{\pi} P = \boldsymbol{\pi}$ with normalization $\pi_1 + \pi_2 = 1$: $$[\pi_1 \quad \pi_2] \begin{bmatrix} 0.8 & 0.2 \\ 0.4 & 0.6 \end{bmatrix} = [\pi_1 \quad \pi_2]$$
2. Expand the column equations: $$\text{Column 1: } 0.8 \pi_1 + 0.4 \pi_2 = \pi_1 \quad \implies \quad 0.4 \pi_2 = 0.2 \pi_1 \quad \implies \quad \pi_1 = 2 \pi_2$$ $$\text{Column 2: } 0.2 \pi_1 + 0.6 \pi_2 = \pi_2 \quad \implies \quad 0.2 \pi_1 = 0.4 \pi_2 \quad (\text{redundant})$$
3. Substitute $\pi_1 = 2 \pi_2$ into the normalization condition $\pi_1 + \pi_2 = 1$: $$2 \pi_2 + \pi_2 = 1 \quad \implies \quad 3 \pi_2 = 1 \quad \implies \quad \pi_2 = \frac{1}{3} \approx 0.3333$$
4. Calculate $\pi_1$: $$\pi_1 = 2 \left(\frac{1}{3}\right) = \frac{2}{3} \approx 0.6667$$
5. Engineering Conclusion: In the long run, the milling cell is operational $66.7\%$ of the time.

Worked Example 2.3.2: 3D Force Torque Vector via Cross Product

Problem: A maintenance gantry crane cable exerts a tension force vector $\mathbf{F} = 4\mathbf{i} - 2\mathbf{j} + 6\mathbf{k}\text{ kN}$ at point $P$. The position vector from the pivot origin to point $P$ is $\mathbf{r} = 2\mathbf{i} + 3\mathbf{j} - 1\mathbf{k}\text{ m}$. Compute the resulting torque (moment) vector $\mathbf{M} = \mathbf{r} \times \mathbf{F}$.

Solution:

1. Set up the determinant form: $$\mathbf{M} = \mathbf{r} \times \mathbf{F} = \det \begin{bmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ 2 & 3 & -1 \\ 4 & -2 & 6 \end{bmatrix}$$
2. Expand by cofactors along the top row: $$\mathbf{i}\text{-component}: (3)(6) - (-1)(-2) = 18 - 2 = 16\mathbf{i}$$ $$\mathbf{j}\text{-component}: -[(2)(6) - (-1)(4)] = -[12 - (-4)] = -[16] = -16\mathbf{j}$$ $$\mathbf{k}\text{-component}: (2)(-2) - (3)(4) = -4 - 12 = -16\mathbf{k}$$
3. Combine terms: $$\mathbf{M} = 16\mathbf{i} - 16\mathbf{j} - 16\mathbf{k}\text{ kN}\cdot\text{m}$$

---

8. NCEES Reference Handbook Tips & Realistic Exam Traps

• The Scalar Matrix Scaling Determinant Trap: If a question states that an $n \times n$ matrix $A$ has $\det(A) = 4$ and asks for $\det(3A)$, the answer is NOT $3 \times 4 = 12$. For an $n \times n$ matrix, every row is multiplied by $3$, which factors out $3^n$: $\det(3A) = 3^n \det(A)$. For a $3 \times 3$ matrix, $\det(3A) = 3^3 (4) = 27 \times 4 = 108$.
• Order Reversal in Inverses and Transposes: Remember that $(AB)^{-1} = B^{-1} A^{-1}$ and $(AB)^T = B^T A^T$. Questions frequently present $A^{-1} B^{-1}$ or $A^T B^T$ as attractive distractors.
• Cramer's Rule Column Placement: In Cramer's rule $x_i = \frac{\det(A_i)}{\det(A)}$, replace the $i$-th column of $A$ with $\mathbf{b}$, not the $i$-th row. Interchanging rows instead of columns will lead to completely incorrect solutions.
• Cross Product Non-Commutativity: Vector cross products are anti-commutative: $\mathbf{r} \times \mathbf{F} = -(\mathbf{F} \times \mathbf{r})$. Computing moment as $\mathbf{F} \times \mathbf{r}$ will produce the opposite signs across every vector component.