3.1 Storage Topologies Compared: DAS, NAS, and SAN Architectures

Storage Topologies Compared: DAS, NAS, and SAN Architectures

Enterprise storage architectures are defined by how storage media connects to compute resources and how data is abstracted for consuming applications. Deciding between Direct-Attached Storage (DAS), Network-Attached Storage (NAS), and Storage Area Networks (SAN) represents one of the most critical decisions in enterprise infrastructure design. Each topology embodies distinct trade-offs across latency, scalability, protocol complexity, fault isolation, and cost.

---

1. Executive Summary & Storage Topology Classification

Storage topologies are categorized based on two fundamental architectural axes:

1. Data Access Level: Does the host operating system interact with raw disk sectors via block commands (Block-Level), or does an intermediary storage controller manage the file system hierarchy and expose structured files via network sharing protocols (File-Level)?
2. Interconnect Medium & Topology: Is the storage coupled directly to the host server bus (Point-to-Point DAS), delivered over a shared local area IP network (LAN-attached NAS), or transported across a dedicated, high-speed storage fabric (Switched Fabric SAN)?

+-------------------------------------------------------------------------+
|                        STORAGE TOPOLOGY TAXONOMY                        |
+-----------------------------------+-------------------------------------+
| BLOCK-LEVEL ACCESS                | FILE-LEVEL ACCESS                   |
| (Host manages file system)        | (Storage appliance manages FS)      |
+-----------------------------------+-------------------------------------+
| DAS: Point-to-Point SAS/NVMe/SATA | NAS: Shared IP/Ethernet Network     |
| SAN: Switched FC / iSCSI / NVMe-oF| Protocols: NFS v3/v4, SMB 2.1/3.1.1 |
+-----------------------------------+-------------------------------------+

---

2. Direct-Attached Storage (DAS) Architecture

Direct-Attached Storage (DAS) is digital storage directly attached to the host computer accessing it, without an intervening storage network. DAS represents the most traditional storage implementation model.

Internal vs. External DAS Implementations

• Internal DAS: Physical drives (NVMe U.2/U.3, 12G/24G SAS, or SATA SSDs/HDDs) reside within the server chassis (e.g., HPE ProLiant DL380 drive cages). Drives connect directly across a passive midplane to an internal storage controller—such as an HPE Smart Array or MegaRAID PCIe Host Bus Adapter (HBA) / SmartRAID controller.
• External DAS: Physical drives reside inside an external drive enclosure (commonly referred to as a JBOD - Just a Bunch of Disks, or JBOF - Just a Bunch of Flash), such as the HPE D3000-series disk enclosures or the high-density HPE D6020 and D8000 enclosures. The external enclosure houses SAS expander modules and redundant power supplies, connecting to the host server via external Mini-SAS HD (SFF-8644) or SAS-4 (SFF-8674) cables plugged into an external SAS HBA or Smart Array controller.

Architectural Characteristics & Protocols

• Interconnect Protocols: Direct PCIe/NVMe, Serial Attached SCSI (SAS-3 at 12 Gb/s per lane, SAS-4 at 24 Gb/s per lane, aggregating 4 lanes per Mini-SAS HD x4 cable for 48 Gb/s or 96 Gb/s bandwidth), and SATA-3 (6 Gb/s).
• Point-to-Point Architecture: Communication occurs strictly between the host controller and the attached disks. There are no fabric routing tables, name servers, or network switch latencies.
• Protocol Overhead: Extremely low. SCSI commands (Command Descriptor Blocks - CDBs) or NVMe command submission/completion queues pass directly across the PCIe or SAS bus.

Architectural Trade-offs & Limitations

• Isolated Storage Silos: Capacity attached to Server A cannot be dynamically presented to Server B. If Server A has 40 TB of unallocated flash while Server B runs out of capacity, an administrator cannot shift capacity without physically migrating drives or performing complex network migrations.
• Limited Multi-Host Sharing: DAS fundamentally lacks native multi-host concurrent write capabilities. While certain dual-domain SAS JBOD enclosures provide dual-port SAS drive connections enabling two servers to connect to the same enclosure for two-node High Availability (HA) clustering (e.g., Microsoft Storage Spaces Direct or Linux Corosync/Pacemaker), scaling beyond two to four hosts is physically constrained.
• Blast Radius & Maintenance: Upgrading or servicing host server hardware often requires taking down the attached DAS, directly impacting application availability.

---

3. Network-Attached Storage (NAS) Architecture

Network-Attached Storage (NAS) is a dedicated file-level data storage appliance connected to a computer network providing data access to a heterogeneous group of clients. Unlike block-level storage, a NAS system owns, formats, and manages the file system.

Appliance Architecture & File Abstraction

A NAS solution consists of dedicated storage hardware running an optimized operating system (e.g., Linux/FreeBSD kernel with ZFS, XFS, or proprietary clustered file systems). The NAS appliance abstracts physical disks into storage pools, creates file systems locally on the appliance, and exposes directories as Shares (Windows) or Exports (UNIX/Linux) over standard Local Area Networks (LANs) via Ethernet (1GbE, 10GbE, 25GbE, 100GbE).

[Client Application] 
       |  File I/O (POSIX read(), write(), open())
       v
[Host Operating System Network Redirector]
       |  NFS / SMB Packets over TCP/IP
       v
[Ethernet Switch Fabric]
       |  Standard IP Packets
       v
[NAS Appliance NIC / Network Stack]
       |  NFS / SMB Daemon
       v
[Appliance Local File System (e.g., ZFS/XFS/WAFL)]
       |  Local Block Read/Write
       v
[Physical Storage Media (SSDs/HDDs)]

Core Network File Protocols

1. Server Message Block (SMB / CIFS)

• Primarily utilized in Microsoft Windows environments, though fully supported on Linux via Samba.
• Dialect Evolution:
  • SMB 2.1: Introduced client leasing to reduce round-trip chatter and large MTU support.
  • SMB 3.0: Introduced critical enterprise capabilities including SMB Direct (SMB over Remote Direct Memory Access [RDMA] using RoCE or iWARP for microsecond-level latency and near-zero CPU utilization), SMB Multichannel (aggregating multiple network connections to combine bandwidth and provide fault tolerance), and transparent failover for continuous availability.
  • SMB 3.1.1: Added pre-authentication integrity verification and AES-128-GCM / AES-256-GCM encryption.

2. Network File System (NFS)

• The standard file sharing protocol for UNIX, Linux, and VMware ESXi datastores.
• NFS v3 vs. NFS v4 Technical Comparison:
  • NFS v3: Fundamentally stateless. The server does not maintain client state across operations. File locking cannot be handled natively by NFS v3; it requires auxiliary helper protocols and daemons: NLM (Network Lock Manager), NSM (Network Status Monitor), and rpc.mountd. These auxiliary services use dynamically assigned RPC ports, making NFS v3 notoriously difficult to secure through firewalls. If a network disruption occurs, stale locks frequently require manual intervention.
  • NFS v4 / v4.1 / v4.2: Fundamentally stateful. The server maintains open file and lock states. File locking and mounting are integrated directly into the core protocol, operating strictly over a single well-known port (TCP 2049), which simplifies firewall traversal. NFS v4 adds compound RPC procedures (grouping multiple operations into a single network round trip), built-in Kerberos security (RPCSEC_GSS), and pNFS (Parallel NFS) in v4.1, which decouples the metadata server from storage data servers to deliver direct, parallel client-to-storage I/O.

Concurrency, File Locking & Multi-Protocol Access

When multiple network clients access the same file repository, the NAS appliance must enforce strict concurrency control:

• File Locking: Supports mandatory locking (denying conflicting access) and advisory locking (notifying processes of lock presence). Windows SMB uses opportunistic locks (oplocks) and file leases, allowing clients to cache read/write data locally until another client requests conflicting access, at which point the server revokes the lease.
• Multi-Protocol Sharing Challenges: Presenting the same directory concurrently via SMB and NFS introduces significant architectural complexity:
  • Identity Mapping: Windows uses variable-length Security Identifiers (SIDs); UNIX/Linux uses numeric 32-bit User IDs (UIDs) and Group IDs (GIDs). Enterprise NAS appliances must integrate with Active Directory (AD) and Lightweight Directory Access Protocol (LDAP) to resolve user identities.
  • Permission Semantics: Reconciling POSIX permission mode bits (rwx for owner/group/other) with granular Windows NTFS Access Control Lists (ACLs).
  • Filename Handling: Enforcing case-insensitive lookups for Windows clients while maintaining case-sensitive namespace preservation for Linux clients.

---

4. Storage Area Network (SAN) Architecture

A Storage Area Network (SAN) is a specialized, high-performance, dedicated network engineered specifically to interconnect application servers (Initiators) with shared enterprise storage controllers (Targets). Unlike NAS, a SAN transports block-level data exclusively.

The Block Storage Abstraction

In a SAN topology, the storage array aggregates physical drives into RAID sets or wide-striped storage pools. An administrator carves out virtual block devices called Logical Unit Numbers (LUNs). When a LUN is mapped and presented to a server, the host operating system discovers it as an unformatted, unpartitioned raw disk (identical to a newly installed local SAS or NVMe drive). The host OS creates a partition table (e.g., GPT) and formats the volume with its own native file system (such as NTFS, XFS, ext4, or VMware VMFS).

SAN Fabric Technologies

+-----------------------+-----------------------+-----------------------+
| FEATURE               | FIBRE CHANNEL (FC)    | iSCSI (IP SAN)        |
+-----------------------+-----------------------+-----------------------+
| Transport Layer       | Dedicated FC Layer 2  | TCP/IP over Ethernet  |
| Typical Speeds        | 16Gb, 32Gb, 64Gb/s    | 10Gb, 25Gb, 100Gb/s   |
| Hardware Interface    | Fibre Channel HBA     | Standard NIC or iHBA  |
| Switch Infrastructure | Dedicated FC Switches | Standard / DCB Switches|
| Frame Delivery        | Lossless (BB_Credits) | TCP flow control / PFC|
| Protocol Overhead     | Extremely Low         | Medium (TCP stack)    |
| Cabling Medium        | Multi-mode/Single LC  | Twisted Pair / DAC /  |
|                       | Optical Fiber         | Optical Fiber (SFP+)  |
+-----------------------+-----------------------+-----------------------+

1. Fibre Channel (FC)

Fibre Channel is a high-speed networking technology purpose-built for storage networks. It uses a dedicated framing protocol and a five-layer stack (FC-0 physical through FC-4 protocol mapping).

• Lossless Fabric Design: Standard Ethernet drops packets when buffers overflow, relying on TCP retransmissions. Fibre Channel implements a deterministic, hardware-based flow control mechanism called Buffer-to-Buffer Credits (BB_Credits). Before transmitting a frame, a transmitter port verifies that the receiving port has an available receive buffer. Every transmitted frame decrements the credit count; every receiver buffer release sends an R_RDY primitive signal back, replenishing credit. If credits reach zero, transmission pauses, ensuring frames are never dropped due to congestion.
• Hardware Components: Requires specialized Host Bus Adapters (HBAs) (e.g., HPE SN1610E 32Gb FC HBA) installed in servers and dedicated Fibre Channel switches (e.g., HPE B-series Brocade switches or HPE C-series Cisco MDS switches).

2. Internet Small Computer Systems Interface (iSCSI)

iSCSI transports SCSI-3 command descriptor blocks encapsulated directly within standard TCP/IP packets (default listening port TCP 3260).

• Economics & Infrastructure: Enables organizations to build block SANs utilizing commodity Ethernet switches and standard server Network Interface Cards (NICs), leveraging software initiators built into Linux, Windows, and VMware ESXi.
• Performance Tuning: Requires dedicated VLANs to isolate storage traffic from user LAN traffic, enabling Jumbo Frames (MTU 9000) to reduce packetization overhead and CPU interrupts, and deploying NICs with TCP Offload Engines (TOE) or iSCSI Hardware HBAs in high-throughput environments.

3. NVMe over Fabrics (NVMe-oF)

NVMe-oF represents the modern evolution of block SAN fabrics, replacing the legacy SCSI command set with lightweight, highly parallel NVMe commands across network fabrics.

• Supported Transports: FC-NVMe (encapsulating NVMe across existing 32Gb/64Gb FC switches alongside standard FCP), NVMe/TCP (running NVMe over standard 25GbE/100GbE IP networks), and NVMe over RDMA (RoCEv2 or InfiniBand).
• Parallelism: Removes the queuing limits of legacy interfaces (AHCI/SATA, for example, offers one queue of 32 commands), supporting up to 64,000 queues with 64,000 commands per queue.

Initiator, Target, and Addressing Concepts

• Initiator: The client endpoint initiating a storage transaction (the server HBA or software iSCSI driver).
• Target: The storage endpoint receiving and executing storage commands (the storage array controller port).
• Addressing Schemes:
  • Fibre Channel: Every HBA port and array port has a burned-in, globally unique 64-bit hexadecimal identifier called a World Wide Port Name (WWPN) (e.g., 50:01:43:80:12:34:56:78), analogous to an Ethernet MAC address. Devices also share a World Wide Node Name (WWNN) identifying the chassis. During fabric login (FLOGI), the switch assigns a dynamic 24-bit FC ID used for frame routing across the fabric.
  • iSCSI: Uses human-readable iSCSI Qualified Names (IQNs) (e.g., iqn.2026-10.com.example:storage:alletra-01) or Extended Unique Identifiers (EUI-64 names).

SAN Security: Zoning vs. LUN Masking

A critical requirement on the HPE0-J81 exam is mastering the architectural difference between Fabric Zoning and Storage Array LUN Masking:

+-------------------------------------------------------------------------+
|                       SAN SECURITY ARCHITECTURE                         |
+------------------------------------+------------------------------------+
| LAYER 1: FABRIC ZONING             | LAYER 2: LUN MASKING               |
| (Configured on SAN Switch Fabric)  | (Configured on Storage Controller) |
+------------------------------------+------------------------------------+
| Defines which switch ports or      | Defines which specific LUNs are    |
| WWPNs can see each other.          | presented to authorized host WWPNs/|
| Prevents unauthorized cross-talk;  | IQNs. Restricts volume mounting to |
| isolates RSCN notifications.       | designated initiators.             |
+------------------------------------+------------------------------------+

• Fabric Zoning (Switch Layer): Implemented on Fibre Channel switches. Divides the SAN fabric into logical security zones. Ports outside a zone cannot communicate with or discover devices inside that zone. Zoning prevents hosts from probing unauthorized storage ports and restricts Registered State Change Notifications (RSCNs) from causing fabric-wide link renegotiation when a single server boots.
  • WWPN zoning (often called soft zoning): zones are defined by device WWPNs. This is the usual best practice because a zone keeps working if a cable moves to another switch port.
  • Port zoning (often called hard zoning): zones are defined by switch domain and port number (e.g., Switch 1, Port 4). Moving a cable breaks connectivity. Strictly, "hard" and "soft" describe how a switch enforces zoning, and modern switches enforce WWPN zones in hardware.
• LUN Masking (Storage Array Layer): Implemented on the storage array controllers. Even if a host initiator resides in the same fabric zone as an array target port, LUN masking ensures the array only exposes designated LUNs to the specific initiator WWPN or IQN. This prevents catastrophic data corruption that would occur if multiple non-clustered servers attempted to write to the same unmanaged file system simultaneously.

---

5. Unified Storage & Hybrid Topologies

Historically, enterprises were forced to maintain two completely separate storage infrastructures: a Fibre Channel SAN for transactional databases and virtual machines, and a dedicated NAS cluster for unstructured user files. Unified Storage (or multiprotocol storage arrays) resolves this division.

Unified Array Architecture

A unified storage array presents simultaneous block-level endpoints (FC, iSCSI, NVMe-oF) and file-level endpoints (NFS, SMB) from a single shared pool of physical storage media and a unified management console.

• Native Dual-Persona Controllers: The array operating system natively exposes both block targets and file services directly from its redundant controller nodes, eliminating external gateways.
• NAS Gateway Topologies: Certain enterprise architectures deploy dedicated NAS gateway heads (stateless compute nodes running file server software) connected via Fibre Channel to backend block enterprise storage arrays (e.g., an HPE StoreEasy or NAS gateway sitting in front of an enterprise block array).

---

6. Multi-Attribute Comparison: DAS vs. NAS vs. SAN

---

7. Real-World Enterprise Workload Selection Scenarios

Understanding how workload requirements dictate storage topology selection is essential for technical storage design and certification exams:

+-------------------------------------------------------------------------+
|                   ENTERPRISE WORKLOAD MAPPING DECISION                  |
+------------------------------------+------------------------------------+
| WORKLOAD PROFILE                   | OPTIMAL TOPOLOGY & RATIONALE       |
+------------------------------------+------------------------------------+
| Mission-Critical Relational DB     | SAN (Fibre Channel or NVMe-oF)     |
| (Oracle RAC, Microsoft SQL Server) | Sub-millisecond deterministic I/O, |
|                                    | dedicated fabric, zero packet drop |
+------------------------------------+------------------------------------+
| Unstructured File Collaboration    | Scale-Out NAS (SMB 3.1.1 / NFS v4) |
| (User homes, CAD, media archives)  | Multi-user concurrency, native     |
|                                    | file locking, ACL integration      |
+------------------------------------+------------------------------------+
| Enterprise Virtualization Cluster  | Block SAN (FC/iSCSI) or Scale-Out  |
| (VMware vSphere ESXi, Hyper-V)     | NFS Datastores                     |
|                                    | VMFS clustered block locking or    |
|                                    | NFS v4.1 high-density vSphere disks|
+------------------------------------+------------------------------------+
| Dedicated Analytics Scratch /      | High-Density SAS / NVMe DAS        |
| Local Video Ingestion Buffer       | Maximum throughput, lowest cost,   |
|                                    | no network fabric overhead         |
+------------------------------------+------------------------------------+

Detailed Analysis of Workload Scenarios

1. High-Transaction Online Transaction Processing (OLTP): A banking core processing thousands of write transactions per second requires guaranteed low latency and predictable response times. A Fibre Channel SAN or NVMe-oF array is mandatory. Fibre Channel's Buffer-to-Buffer credits guarantee zero packet drop due to network congestion, while raw LUN presentation allows database engines to bypass file system abstraction and write directly to raw database extents.
2. Engineering CAD & Corporate Document Repositories: Thousands of client workstations across Windows and Linux environments need simultaneous access to shared directories, project drawings, and application configuration templates. A NAS solution using SMB 3.1.1 and NFS v4.2 is the proper architecture. Centralizing the file system on the NAS appliance allows granular Active Directory ACL enforcement, automatic snapshots, and multi-client locking.
3. Big Data Batch Analytics Scratch Worker Nodes: Large-scale distributed processing engines (such as Apache Hadoop or high-performance temporary scratch nodes) process massive sequential read/write operations locally. External SAN storage would introduce unneeded network fabric costs and switch port saturation. Dense Direct-Attached SAS/NVMe storage within each server chassis maximizes raw ingest bandwidth at the lowest cost per gigabyte.

---

8. Common Exam Traps & Pitfalls

• Exam Trap 1: Conflating SAN Fabric Zoning with Storage Array LUN Masking. Candidates often mistakenly believe that configuring soft zoning on a Brocade or Cisco switch grants host servers access to specific volumes. Switch zoning merely permits the host HBA to see the array controller target port. Without LUN masking configured on the storage array, the host cannot discover or mount any LUNs. Conversely, configuring LUN masking without switch zoning leaves the fabric unprotected against RSCN storms and device cross-talk.
• Exam Trap 2: Assuming iSCSI Requires Proprietary Cabling or Custom Switches. iSCSI operates over standard Ethernet switches and standard Cat6a copper or SFP+ fiber cabling using standard TCP/IP. While network best practices dictate dedicating separate VLANs, enabling Jumbo Frames (MTU 9000), and configuring Flow Control, proprietary hardware is never required.
• Exam Trap 3: Assuming DAS Cannot Provide High Availability. While basic internal DAS is limited to a single server, enterprise external DAS enclosures with dual SAS I/O modules and dual-ported SAS drives can connect to two independent servers simultaneously to form a high-availability failover cluster (such as a two-node Windows Server Failover Cluster).
• Exam Trap 4: Misunderstanding File System Ownership. In SAN and DAS topologies, the storage array has zero knowledge of the file system structure; the host operating system formats and manages the file system. In NAS, the storage appliance formats, owns, and manages the file system; client hosts only transmit network file requests.