11.3 Advanced NIO.2 Stream Operations and Directory Traversal

Advanced NIO.2 Stream Operations and Directory Traversal

Processing large file systems and high-volume text datasets using traditional iterative loops often results in excessive heap memory consumption, slow sequential disk I/O, and complex boilerplate code. Java NIO.2 bridges the physical file system directly with the Stream API, offering lazy, high-performance streaming factories on java.nio.file.Files.

On the Oracle Certified Professional: Java SE 21 Developer (1Z0-830) exam, candidates must understand the precise semantics of Files.lines(), Files.list(), Files.walk(), Files.find(), and DirectoryStream, the critical requirement of managing file descriptor resources using try-with-resources, and the manipulation of POSIX file permissions and attribute views.

---

1. Lazy Stream I/O Architecture & Resource Management

When reading lines from text files, developers must choose between eager in-memory loading and lazy stream processing:

• Files.readAllLines(Path path): Eager Loading. Reads the entire file content into a List<String> stored on the JVM heap. If invoked on multi-gigabyte files (such as enterprise transaction logs), this method will quickly trigger java.lang.OutOfMemoryError: Java heap space.
• Files.lines(Path path): Lazy Stream Processing. Returns a lazy Stream<String> that reads lines on demand as the terminal stream operation consumes them. Memory consumption remains strictly bounded.

// Correct idiom: Enclose in try-with-resources to prevent OS file descriptor leaks!
try (Stream<String> lines = Files.lines(Path.of("production-access.log"), StandardCharsets.UTF_8)) {
    long errorCount = lines
        .filter(line -> line.contains("HTTP 500"))
        .map(String::trim)
        .count();
    System.out.println("Total 500 Server Errors: " + errorCount);
}

---

2. Shallow Directory Traversal: Files.list() vs. DirectoryStream

Files.list(Path dir)

Files.list(Path dir) reads the entries of a single directory, returning a lazy Stream<Path>:

• Depth: Exactly 1 (shallow listing). It lists direct child files and subdirectories, but does not recurse into nested directories.
• Excludes Start Node: The returned stream contains only direct children, excluding the start directory dir itself.
• Ordering: Elements are emitted in an undefined, file-system-dependent order.

Path dir = Path.of("/var/log");
try (Stream<Path> entries = Files.list(dir)) {
    entries.filter(Files::isRegularFile)
           .map(Path::getFileName)
           .forEach(System.out::println);
}

DirectoryStream<Path> (The Iterable Alternative)

For traditional imperative for-each loops and built-in glob filtering, NIO.2 provides java.nio.file.DirectoryStream<Path>:

• Implements Iterable<Path> and AutoCloseable.
• Supports built-in file system glob expressions (e.g., "*.{log,txt}").
• Supports custom DirectoryStream.Filter<Path> lambda expressions.

Path dir = Path.of("/var/log");

// Glob pattern matching:
try (DirectoryStream<Path> ds = Files.newDirectoryStream(dir, "*.{log,txt}")) {
    for (Path entry : ds) {
        System.out.println("Matched log file: " + entry.getFileName());
    }
}

// Custom Filter lambda:
DirectoryStream.Filter<Path> sizeFilter = p -> Files.size(p) > 1_000_000L;
try (DirectoryStream<Path> ds = Files.newDirectoryStream(dir, sizeFilter)) {
    for (Path largeFile : ds) {
        System.out.println("Large file: " + largeFile);
    }
}

---

3. Deep Tree Traversal: Files.walk()

Files.walk() performs a depth-first recursive traversal (DFS) of a directory tree, returning a Stream<Path>.

Method Overloads

1. Files.walk(Path start, FileVisitOption... options): Traverses recursively to maximum depth Integer.MAX_VALUE.
2. Files.walk(Path start, int maxDepth, FileVisitOption... options): Traverses up to maxDepth levels deep.

Depth Calculation Rules for maxDepth

Understanding how maxDepth is calculated is essential for the exam:

• maxDepth = 0: Traverses only the start path itself (depth 0).
• maxDepth = 1: Traverses the start path (depth 0) plus its immediate children (depth 1).
• maxDepth = 2: Traverses start, direct children, and children's children.

Path rootDir = Path.of("/projects");

// Slicing directory tree up to depth 2:
try (Stream<Path> tree = Files.walk(rootDir, 2)) {
    tree.filter(p -> p.toString().endsWith(".java"))
        .forEach(System.out::println);
}

Files.list() vs. Files.walk(start, 1) Comparison

---

4. High-Performance Metadata Searching: Files.find()

Files.find() combines recursive directory tree traversal with direct file metadata evaluation:

public static Stream<Path> find(
    Path start,
    int maxDepth,
    BiPredicate<Path, BasicFileAttributes> matcher,
    FileVisitOption... options
) throws IOException

Why Files.find() Outperforms Files.walk().filter()

When using Files.walk().filter(p -> Files.size(p) > 1024), the JVM must issue a separate operating system system call (stat/fstat) for every single path encountered during traversal. In contrast, Files.find() retrieves the BasicFileAttributes in the same OS system call used to discover the directory entry, eliminating redundant disk I/O hits and running orders of magnitude faster on large directory trees.

Path startDir = Path.of("/var/data");

try (Stream<Path> matches = Files.find(startDir, 5, (path, attrs) -> {
    return attrs.isRegularFile() 
        && attrs.size() > 10_000_000L // Larger than 10 MB
        && path.getFileName().toString().endsWith(".log");
})) {
    matches.forEach(p -> System.out.println("Matching large log: " + p));
}

---

5. Symbolic Links and Circular References

By default, NIO.2 stream traversal methods (Files.walk(), Files.find(), and Files.list()) do not follow symbolic links.

• Default Behavior: A symbolic link is treated as a leaf file node; the target directory it references is not entered.
• Enabling Link Traversal: Pass FileVisitOption.FOLLOW_LINKS as an option argument to Files.walk() or Files.find().
• The Circular Link Hazard: If a symbolic link creates a circular reference (e.g., /app/dirA/link points back to /app), the JVM detects the cycle and throws java.nio.file.FileSystemLoopException.

try (Stream<Path> stream = Files.walk(rootDir, 10, FileVisitOption.FOLLOW_LINKS)) {
    stream.forEach(System.out::println);
} catch (FileSystemLoopException e) {
    System.err.println("Circular symbolic link detected: " + e.getFile());
}

---

6. File Attribute Views and POSIX Permissions

NIO.2 provides a clean architectural separation between read-only attribute snapshots and modifiable attribute views.

Read-Only Attribute Interfaces vs. Modifiable Views

• Read-Only Attributes: BasicFileAttributes, DosFileAttributes, PosixFileAttributes.
• Modifiable Views: BasicFileAttributeView, DosFileAttributeView, PosixFileAttributeView.

BasicFileAttributes Methods

Path file = Path.of("archive.tar.gz");
BasicFileAttributes attrs = Files.readAttributes(file, BasicFileAttributes.class);

System.out.println("Regular file: " + attrs.isRegularFile());
System.out.println("Directory:    " + attrs.isDirectory());
System.out.println("Symlink:      " + attrs.isSymbolicLink());
System.out.println("Other:        " + attrs.isOther());
System.out.println("Size (bytes): " + attrs.size());
System.out.println("Created:      " + attrs.creationTime());
System.out.println("Modified:     " + attrs.lastModifiedTime());
System.out.println("Accessed:     " + attrs.lastAccessTime());
System.out.println("File Key:     " + attrs.fileKey()); // Unique OS inode / file ID

Modifying File Timestamps with BasicFileAttributeView

Path file = Path.of("report.pdf");
BasicFileAttributeView view = Files.getFileAttributeView(file, BasicFileAttributeView.class);

FileTime now = FileTime.from(Instant.now());
// setTimes(lastModifiedTime, lastAccessTime, createTime)
// Passing null leaves that specific timestamp unchanged
view.setTimes(now, now, null);

POSIX File Permissions (Unix, Linux, macOS)

On POSIX-compliant file systems, permissions are managed using the PosixFilePermission enum and PosixFilePermissions helper:

Path script = Path.of("/opt/app/deploy.sh");

// 1. Reading POSIX permissions:
Set<PosixFilePermission> perms = Files.getPosixFilePermissions(script);
System.out.println("Current permissions: " + PosixFilePermissions.toString(perms)); // e.g. "rwxr-xr-x"

// 2. Modifying POSIX permissions via String representation:
Set<PosixFilePermission> newPerms = PosixFilePermissions.fromString("rwxr-x---");
Files.setPosixFilePermissions(script, newPerms);

// 3. Creating a new file with initial POSIX permissions:
FileAttribute<Set<PosixFilePermission>> attr = PosixFilePermissions.asFileAttribute(newPerms);
Files.createFile(Path.of("/opt/app/secure.key"), attr);

---

7. Traversal Methods Comparison Matrix