Java File I/O & Streams

Series: Java Core for Beginners Lecture: 11 of 12 Topics: File & Path · Reading files · Writing files · Byte streams vs character streams · try-with-resources · Working with directories

Table of Contents

The Java I/O Landscape

Java’s I/O story has evolved significantly across its history. Understanding which API to use requires knowing why there are multiple ones.

Three generations of file I/O in Java

java.io.File (Java 1.0) — the original. A single class that represents both files and directories. Useful but flawed: it cannot report why an operation failed (methods return false instead of throwing), has inconsistent behavior across operating systems, and lacks many common operations.

java.io streams (Java 1.0) — the original byte-level and character-level I/O: FileInputStream, FileOutputStream, FileReader, FileWriter, BufferedReader, BufferedWriter. Still widely used, still valid, but verbose.

java.nio.file — NIO.2 (Java 7+) — the modern API. Path, Paths, Files, and FileSystem replace java.io.File for path manipulation and file system operations. Much more expressive, throws meaningful exceptions, and integrates cleanly with try-with-resources.

What to use today:

Use Path and Files (NIO.2) for all file system operations — creating, deleting, copying, listing directories, checking existence.
Use java.io streams (wrapped in BufferedReader/BufferedWriter) when you need fine-grained stream control or are working with binary data.
For simple read/write of entire files, Files.readAllLines(), Files.writeString(), and Files.lines() from NIO.2 are the most concise and readable options.

This lecture covers all three tiers, showing where each fits.

Path and Files — The Modern API

Path — representing a file system location

Path is an interface representing a location in the file system — a sequence of directory names ending in either a file name or a directory name. Unlike java.io.File, a Path can represent paths on any file system, including ZIP archives and remote filesystems.

import java.nio.file.Path;
import java.nio.file.Paths;

// Creating paths
Path absolute  = Path.of("/home/alice/documents/report.txt");  // Java 11+
Path relative  = Path.of("data/config.properties");
Path fromParts = Path.of("src", "main", "java", "Main.java");  // joined automatically

// Legacy Paths.get() — same result, older style
Path legacy = Paths.get("/home/alice/documents/report.txt");

Path.of() (Java 11+) is the preferred factory method. Paths.get() (Java 7+) does the same thing — you will see both in existing code.

Navigating paths

Path p = Path.of("/home/alice/documents/report.txt");

p.getFileName()        // report.txt
p.getParent()          // /home/alice/documents
p.getRoot()            // /  (null on relative paths)
p.getNameCount()       // 4
p.getName(0)           // home
p.getName(3)           // report.txt
p.isAbsolute()         // true
p.toString()           // "/home/alice/documents/report.txt"

// Resolving relative paths
Path dir   = Path.of("/home/alice/documents");
Path file  = dir.resolve("report.txt");   // /home/alice/documents/report.txt
Path up    = dir.resolve("../photos");    // /home/alice/documents/../photos
Path clean = up.normalize();              // /home/alice/photos

// Relativizing
Path base  = Path.of("/home/alice");
Path child = Path.of("/home/alice/documents/report.txt");
Path rel   = base.relativize(child);     // documents/report.txt

Files — operating on the file system

java.nio.file.Files is a utility class of static methods for reading, writing, copying, moving, deleting, and inspecting files and directories. Every method that performs I/O throws a checked IOException — handle it or declare it.

import java.nio.file.Files;

Path path = Path.of("data.txt");

// Existence and type checks
Files.exists(path)          // true if the path exists
Files.notExists(path)       // true if the path definitely does not exist
Files.isRegularFile(path)   // true if it exists and is a file (not directory)
Files.isDirectory(path)     // true if it exists and is a directory
Files.isReadable(path)      // true if readable
Files.isWritable(path)      // true if writable
Files.isHidden(path)        // true if hidden (OS-dependent)

Reading Files

Reading all lines at once

For small to medium files (fits comfortably in memory), Files.readAllLines() is the simplest approach:

import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.charset.StandardCharsets;
import java.util.List;

Path path = Path.of("students.txt");

try {
    List<String> lines = Files.readAllLines(path, StandardCharsets.UTF_8);
    for (String line : lines) {
        System.out.println(line);
    }
} catch (IOException e) {
    System.err.println("Failed to read file: " + e.getMessage());
}

Always specify the charset explicitly. Use StandardCharsets.UTF_8 by default — it handles almost all text correctly. Relying on the system default charset produces code that behaves differently on different machines.

Reading entire file as a String (Java 11+)

try {
    String content = Files.readString(Path.of("notes.txt"), StandardCharsets.UTF_8);
    System.out.println(content);
} catch (IOException e) {
    System.err.println("Error: " + e.getMessage());
}

Reading large files line by line — BufferedReader

For large files (millions of lines), loading everything into memory at once is impractical. Read line by line with BufferedReader:

Path path = Path.of("large-dataset.csv");

try (BufferedReader reader = Files.newBufferedReader(path, StandardCharsets.UTF_8)) {
    String line;
    int lineNumber = 0;
    while ((line = reader.readLine()) != null) {
        lineNumber++;
        // process one line at a time — only one line in memory
        System.out.println(lineNumber + ": " + line);
    }
} catch (IOException e) {
    System.err.println("Error reading file: " + e.getMessage());
}

readLine() returns null when the end of the file is reached. The BufferedReader is auto-closed by try-with-resources when the block exits.

Streaming lines lazily — Files.lines() (Java 8+)

Files.lines() returns a Stream — each line is read on demand rather than all upfront. This is memory-efficient for large files and integrates with the Stream API (Lecture 12):

Path path = Path.of("data.csv");

try (var lines = Files.lines(path, StandardCharsets.UTF_8)) {
    lines.filter(line -> !line.startsWith("#"))   // skip comment lines
         .map(String::trim)                        // remove leading/trailing whitespace
         .filter(line -> !line.isEmpty())          // skip blank lines
         .forEach(System.out::println);
} catch (IOException e) {
    System.err.println("Error: " + e.getMessage());
}

Important: Files.lines() returns a Stream that holds an open file handle. Always use it inside try-with-resources — the Stream implements AutoCloseable and will close the file when the stream is closed.

Reading binary files

For binary data (images, PDFs, serialized objects), read raw bytes:

Path imagePath = Path.of("photo.jpg");

try {
    byte[] bytes = Files.readAllBytes(imagePath);
    System.out.println("File size: " + bytes.length + " bytes");
} catch (IOException e) {
    System.err.println("Error reading binary file: " + e.getMessage());
}

For large binary files, use FileInputStream with a buffer (covered in section 5).

Reading with Scanner

Scanner is convenient for parsing structured input — numbers, tokens, delimited values — but is slower than BufferedReader for simple line-by-line reading:

Path path = Path.of("numbers.txt");

try (Scanner scanner = new Scanner(path, StandardCharsets.UTF_8)) {
    int total = 0;
    int count = 0;
    while (scanner.hasNextInt()) {
        total += scanner.nextInt();
        count++;
    }
    System.out.printf("Sum: %d, Count: %d, Average: %.2f%n",
                      total, count, (double) total / count);
} catch (IOException e) {
    System.err.println("Error: " + e.getMessage());
}

Scanner uses a Path directly since Java 10, eliminating the need to wrap in a FileReader.

Writing Files

Writing a String (Java 11+)

import java.nio.file.StandardOpenOption;

Path path = Path.of("output.txt");
String content = "Hello, File I/O!\nLine 2\nLine 3\n";

try {
    Files.writeString(path, content, StandardCharsets.UTF_8);
    System.out.println("File written successfully.");
} catch (IOException e) {
    System.err.println("Failed to write file: " + e.getMessage());
}

By default, writeString creates the file if it does not exist and overwrites it if it does.

Appending to an existing file

Pass StandardOpenOption.APPEND to add content after existing content:

try {
    Files.writeString(path, "\nAppended line.", StandardCharsets.UTF_8,
                      StandardOpenOption.CREATE, StandardOpenOption.APPEND);
} catch (IOException e) {
    System.err.println("Error: " + e.getMessage());
}

Writing a list of lines

List<String> lines = List.of("Alice,30,Engineer",
                              "Bob,25,Designer",
                              "Charlie,35,Manager");

try {
    Files.write(Path.of("employees.csv"), lines, StandardCharsets.UTF_8);
} catch (IOException e) {
    System.err.println("Error writing file: " + e.getMessage());
}

Files.write() adds a line separator after each element automatically.

Writing large files — BufferedWriter

For writing large amounts of content or when you need fine-grained control over what and when to flush:

Path path = Path.of("report.txt");

try (BufferedWriter writer = Files.newBufferedWriter(path, StandardCharsets.UTF_8)) {
    writer.write("Sales Report — Q4 2024");
    writer.newLine();
    writer.write("=".repeat(30));
    writer.newLine();

    for (int i = 1; i <= 1000; i++) {
        writer.write(String.format("Record %d: value = %.2f%n", i, Math.random() * 1000));
    }
} catch (IOException e) {
    System.err.println("Error writing report: " + e.getMessage());
}

BufferedWriter accumulates writes in an internal buffer (8 KB by default) and flushes to disk in large chunks — far more efficient than writing one byte at a time. It is flushed and closed automatically by try-with-resources.

PrintWriter — printf-style writing

PrintWriter wraps a Writer and adds print(), println(), and printf() — identical to System.out but writing to a file:

Path path = Path.of("formatted-output.txt");

try (PrintWriter pw = new PrintWriter(
        Files.newBufferedWriter(path, StandardCharsets.UTF_8))) {
    pw.println("Name         Age   Salary");
    pw.println("-".repeat(35));
    pw.printf("%-12s %4d  %8.2f%n", "Alice",   30, 75000.0);
    pw.printf("%-12s %4d  %8.2f%n", "Bob",     25, 62500.0);
    pw.printf("%-12s %4d  %8.2f%n", "Charlie", 35, 90000.0);
} catch (IOException e) {
    System.err.println("Error: " + e.getMessage());
}

StandardOpenOption reference

Option	Meaning
`CREATE`	Create the file if it does not exist (default for most write operations)
`CREATE_NEW`	Create the file, fail if it already exists
`TRUNCATE_EXISTING`	Truncate to zero bytes before writing (default for overwrite)
`APPEND`	Append to the end of an existing file
`WRITE`	Open for writing
`READ`	Open for reading
`SYNC`	Synchronize file content and metadata to disk on every write

Byte Streams vs Character Streams

Java’s I/O system has two parallel hierarchies serving different purposes.

Byte streams — raw binary data

Byte streams work at the level of individual bytes (byte, 0–255). Use them for binary data: images, audio, video, serialized objects, any format where bytes have specific meaning:

InputStream (abstract)
    ├── FileInputStream        — reads bytes from a file
    ├── ByteArrayInputStream   — reads bytes from a byte array
    └── FilterInputStream
          └── BufferedInputStream — adds buffering

OutputStream (abstract)
    ├── FileOutputStream       — writes bytes to a file
    ├── ByteArrayOutputStream  — writes bytes to a byte array
    └── FilterOutputStream
          └── BufferedOutputStream — adds buffering

Path source      = Path.of("photo.jpg");
Path destination = Path.of("photo-copy.jpg");

try (InputStream  in  = new BufferedInputStream(new FileInputStream(source.toFile()));
     OutputStream out = new BufferedOutputStream(new FileOutputStream(destination.toFile()))) {

    byte[] buffer = new byte[8192];   // 8 KB read buffer
    int bytesRead;
    while ((bytesRead = in.read(buffer)) != -1) {
        out.write(buffer, 0, bytesRead);
    }
    System.out.println("File copied successfully.");
} catch (IOException e) {
    System.err.println("Copy failed: " + e.getMessage());
}

in.read(buffer) fills the buffer with up to buffer.length bytes and returns the number of bytes actually read, or -1 at end of file. This pattern is the standard idiom for binary file copying.

Note: For file copying, Files.copy() is far simpler — this example shows the underlying mechanism:

Files.copy(source, destination, StandardCopyOption.REPLACE_EXISTING);

Character streams — text data

Character streams work at the level of Unicode characters. They handle the encoding/decoding between bytes and characters transparently. Use them for all text files:

Reader (abstract)
    ├── FileReader             — reads chars from a file (uses system charset — avoid)
    ├── StringReader           — reads chars from a String
    └── BufferedReader         — adds buffering + readLine()

Writer (abstract)
    ├── FileWriter             — writes chars to a file (uses system charset — avoid)
    ├── StringWriter           — writes chars to a String
    ├── BufferedWriter         — adds buffering + newLine()
    └── PrintWriter            — adds print/println/printf

Avoid FileReader and FileWriter directly — they use the system default charset, producing code that behaves differently across platforms. Use Files.newBufferedReader() and Files.newBufferedWriter() instead — they let you specify the charset explicitly.

// Avoid
Reader reader = new FileReader("data.txt");  // system charset — unpredictable

// Prefer
BufferedReader reader = Files.newBufferedReader(Path.of("data.txt"), StandardCharsets.UTF_8);

The bridge: InputStreamReader and OutputStreamWriter

When you have a byte stream and need character-level access, bridge them with InputStreamReader and OutputStreamWriter, specifying the charset:

// Reading from a stream with explicit charset
InputStream rawStream = getSomeInputStream();
try (BufferedReader reader = new BufferedReader(
        new InputStreamReader(rawStream, StandardCharsets.UTF_8))) {
    String line;
    while ((line = reader.readLine()) != null) {
        System.out.println(line);
    }
}

// Writing to a stream with explicit charset
OutputStream rawStream = getSomeOutputStream();
try (PrintWriter writer = new PrintWriter(
        new OutputStreamWriter(rawStream, StandardCharsets.UTF_8))) {
    writer.println("Hello, stream!");
}

This pattern is common when working with network connections (Socket.getInputStream()) or HTTP responses.

Choosing between byte and character streams

Use byte streams when	Use character streams when
Copying files without interpreting content	Reading or writing text
Working with images, audio, video, PDFs	Processing CSV, JSON, XML, HTML, logs
Serializing Java objects	Any format where lines and characters matter
Network communication of binary protocols	Console I/O
Anything where encoding would corrupt the data	Files that humans read and write

Buffered Streams — Performance in Practice

Every call to read() or write() on an unbuffered stream goes directly to the operating system — a system call. System calls are orders of magnitude slower than in-memory operations. Buffering dramatically reduces the number of system calls by batching I/O into chunks.

Performance comparison

import java.time.Instant;

Path source = Path.of("large-file.bin");   // a 100 MB file
Path dest   = Path.of("copy.bin");

// Unbuffered — one system call per byte
long start = System.currentTimeMillis();
try (FileInputStream in = new FileInputStream(source.toFile());
     FileOutputStream out = new FileOutputStream(dest.toFile())) {
    int b;
    while ((b = in.read()) != -1) {
        out.write(b);
    }
}
System.out.println("Unbuffered: " + (System.currentTimeMillis() - start) + "ms");
// Typically: 60,000–120,000ms (minutes!) on a 100MB file

// Buffered — one system call per 8KB chunk
start = System.currentTimeMillis();
try (InputStream in   = new BufferedInputStream(new FileInputStream(source.toFile()));
     OutputStream out = new BufferedOutputStream(new FileOutputStream(dest.toFile()))) {
    byte[] buffer = new byte[8192];
    int bytesRead;
    while ((bytesRead = in.read(buffer)) != -1) {
        out.write(buffer, 0, bytesRead);
    }
}
System.out.println("Buffered: " + (System.currentTimeMillis() - start) + "ms");
// Typically: 200–600ms

The difference is dramatic — buffered I/O is typically 100–1000× faster for large files.

Buffer size guidelines

The default buffer size in BufferedInputStream/BufferedReader is 8 KB (8,192 bytes). For most purposes this is fine. For large binary file operations, a 64 KB or 128 KB buffer can improve throughput further:

// Custom buffer size — 64KB
new BufferedInputStream(new FileInputStream(path.toFile()), 65_536)

The optimal size depends on the file system and hardware. 8–64 KB covers the vast majority of use cases effectively.

Always buffer your streams

Rule of thumb: whenever you wrap a FileInputStream or FileOutputStream, always add a Buffered layer. The cost is negligible (a small heap allocation); the performance benefit is enormous.

The NIO.2 methods (Files.newBufferedReader, Files.newBufferedWriter) add buffering automatically — another reason to prefer them.

Working with Directories

Creating directories

Path dir     = Path.of("output/reports/2024");

// Create a single directory (parent must exist)
Files.createDirectory(Path.of("output"));

// Create all intermediate directories (like mkdir -p)
Files.createDirectories(dir);   // creates output/, output/reports/, output/reports/2024/

Listing directory contents

Path dir = Path.of("/home/alice/documents");

// List immediate children
try (var entries = Files.list(dir)) {
    entries.forEach(path -> System.out.println(path.getFileName()));
} catch (IOException e) {
    System.err.println("Error listing directory: " + e.getMessage());
}

// List with filtering
try (var entries = Files.list(dir)) {
    entries.filter(Files::isRegularFile)
           .filter(p -> p.toString().endsWith(".txt"))
           .forEach(System.out::println);
} catch (IOException e) {
    System.err.println("Error: " + e.getMessage());
}

Files.list() returns a Stream — use it inside try-with-resources just like Files.lines().

Walking a directory tree

Path root = Path.of("src");

// Walk recursively — all files and directories under root
try (var paths = Files.walk(root)) {
    paths.filter(Files::isRegularFile)
         .filter(p -> p.toString().endsWith(".java"))
         .sorted()
         .forEach(System.out::println);
} catch (IOException e) {
    System.err.println("Error walking directory: " + e.getMessage());
}

// Walk with max depth
try (var paths = Files.walk(root, 2)) {   // at most 2 levels deep
    paths.forEach(System.out::println);
} catch (IOException e) {
    System.err.println("Error: " + e.getMessage());
}

Finding files with a glob pattern

// Find all .log files in /var/log and its subdirectories
try (var matches = Files.find(
        Path.of("/var/log"),
        Integer.MAX_VALUE,
        (path, attrs) -> attrs.isRegularFile() && path.toString().endsWith(".log"))) {
    matches.forEach(System.out::println);
} catch (IOException e) {
    System.err.println("Error: " + e.getMessage());
}

Files.find() is similar to Files.walk() but accepts a BiPredicate — you can filter on both the path and file attributes (size, creation time, etc.) in one step.

Copying, moving, and deleting

Path source = Path.of("data.csv");
Path target = Path.of("backup/data.csv");

// Copy — fails if target exists by default
Files.copy(source, target);

// Copy — overwrite if target exists
Files.copy(source, target, StandardCopyOption.REPLACE_EXISTING);

// Copy — copy file attributes (timestamps, permissions) as well
Files.copy(source, target,
           StandardCopyOption.REPLACE_EXISTING,
           StandardCopyOption.COPY_ATTRIBUTES);

// Move (atomic rename when possible)
Files.move(source, target, StandardCopyOption.REPLACE_EXISTING);

// Delete — throws NoSuchFileException if file does not exist
Files.delete(target);

// Delete if exists — does not throw if missing
Files.deleteIfExists(target);

Deleting a non-empty directory tree

Files.delete() only works on empty directories. To delete a tree recursively:

Path dirToDelete = Path.of("temp/work");

try (var paths = Files.walk(dirToDelete)) {
    paths.sorted(Comparator.reverseOrder())   // files before their containing directories
         .forEach(path -> {
             try {
                 Files.delete(path);
             } catch (IOException e) {
                 System.err.println("Could not delete: " + path + " — " + e.getMessage());
             }
         });
} catch (IOException e) {
    System.err.println("Error during cleanup: " + e.getMessage());
}

Creating temp files and directories

// Temp file — deleted eventually by the OS or on JVM exit
Path tempFile = Files.createTempFile("prefix-", ".tmp");
System.out.println(tempFile);   // e.g. /tmp/prefix-3847162983472.tmp

// Temp directory
Path tempDir = Files.createTempDirectory("myapp-work-");
System.out.println(tempDir);    // e.g. /tmp/myapp-work-3847162983472

Use tempFile.toFile().deleteOnExit() or delete explicitly when done.

File Attributes and Metadata

import java.nio.file.attribute.BasicFileAttributes;

Path path = Path.of("report.pdf");

try {
    BasicFileAttributes attrs = Files.readAttributes(path, BasicFileAttributes.class);

    System.out.println("Size:          " + attrs.size() + " bytes");
    System.out.println("Created:       " + attrs.creationTime());
    System.out.println("Last modified: " + attrs.lastModifiedTime());
    System.out.println("Last accessed: " + attrs.lastAccessTime());
    System.out.println("Is file:       " + attrs.isRegularFile());
    System.out.println("Is directory:  " + attrs.isDirectory());
    System.out.println("Is symlink:    " + attrs.isSymbolicLink());

    // Size convenience method
    System.out.println("Size (direct): " + Files.size(path) + " bytes");

    // Last modified convenience method
    System.out.println("Modified:      " + Files.getLastModifiedTime(path));
} catch (IOException e) {
    System.err.println("Error: " + e.getMessage());
}

Comparing files

Path a = Path.of("original.txt");
Path b = Path.of("copy.txt");

// Are they the same file? (same inode, not just same content)
boolean sameFile = Files.isSameFile(a, b);

// Do they have the same content? (Java 12+)
boolean sameContent = Files.mismatch(a, b) == -1;
// Files.mismatch returns -1 if identical, or position of first differing byte

Standard Input and Output

Java’s System.in, System.out, and System.err are pre-opened streams connected to the terminal.

Reading from System.in

System.in is a InputStream. For line-by-line user input, wrap it in a BufferedReader or Scanner:

// Scanner — most common for simple interactive input
import java.util.Scanner;

Scanner scanner = new Scanner(System.in);
System.out.print("Enter your name: ");
String name = scanner.nextLine();
System.out.print("Enter your age: ");
int age = scanner.nextInt();
System.out.printf("Hello, %s! You are %d years old.%n", name, age);

// BufferedReader — faster for reading many lines
try (BufferedReader reader = new BufferedReader(new InputStreamReader(System.in))) {
    System.out.println("Enter lines (empty line to stop):");
    String line;
    while (!(line = reader.readLine()).isEmpty()) {
        System.out.println("You entered: " + line);
    }
}

Redirecting System.out

System.out is a PrintStream. You can redirect it to a file — useful for logging:

try (PrintStream fileOut = new PrintStream(
        new BufferedOutputStream(new FileOutputStream("app.log")),
        true,   // auto-flush
        StandardCharsets.UTF_8)) {

    System.setOut(fileOut);    // redirect System.out to file
    System.out.println("This goes to app.log, not the terminal.");
}
// After the try block, System.out is still redirected — reset if needed
System.setOut(new PrintStream(new FileOutputStream(FileDescriptor.out)));

System.err

System.err is the standard error stream — a separate stream from System.out. By convention, error and diagnostic messages go to System.err so they can be separated from normal output when redirecting:

System.out.println("Normal output");      // stdout
System.err.println("Error occurred!");    // stderr

Putting It Together — A Complete Example

Let us build a CSV data processor that demonstrates reading, parsing, processing, error handling, and writing results:

import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.*;
import java.util.*;

public class CsvProcessor {

    // Represents one row of employee data
    public record Employee(String name, int age, String department, double salary) {
        @Override
        public String toString() {
            return String.format("%-15s %3d  %-15s  %8.2f", name, age, department, salary);
        }
    }

    // Parse a single CSV line into an Employee, return null and log error if invalid
    private static Employee parseLine(String line, int lineNumber) {
        String[] parts = line.split(",");
        if (parts.length != 4) {
            System.err.printf("[Line %d] Expected 4 fields, got %d: %s%n",
                              lineNumber, parts.length, line);
            return null;
        }
        try {
            String name       = parts[0].trim();
            int    age        = Integer.parseInt(parts[1].trim());
            String department = parts[2].trim();
            double salary     = Double.parseDouble(parts[3].trim());

            if (name.isBlank()) throw new IllegalArgumentException("Name is blank");
            if (age < 18 || age > 70) throw new IllegalArgumentException("Age out of range: " + age);
            if (salary < 0) throw new IllegalArgumentException("Salary is negative: " + salary);

            return new Employee(name, age, department, salary);
        } catch (NumberFormatException e) {
            System.err.printf("[Line %d] Invalid number — %s: %s%n", lineNumber, e.getMessage(), line);
            return null;
        } catch (IllegalArgumentException e) {
            System.err.printf("[Line %d] Validation failed — %s: %s%n", lineNumber, e.getMessage(), line);
            return null;
        }
    }

    // Read and parse a CSV file, skipping the header and invalid rows
    public static List<Employee> loadEmployees(Path csvPath) throws IOException {
        List<Employee> employees = new ArrayList<>();
        try (BufferedReader reader = Files.newBufferedReader(csvPath, StandardCharsets.UTF_8)) {
            String header = reader.readLine();   // skip header line
            if (header == null) return employees;

            String line;
            int lineNumber = 1;
            while ((line = reader.readLine()) != null) {
                lineNumber++;
                line = line.trim();
                if (line.isEmpty() || line.startsWith("#")) continue;   // skip blank/comment lines
                Employee emp = parseLine(line, lineNumber);
                if (emp != null) employees.add(emp);
            }
        }
        return employees;
    }

    // Write a summary report to a file
    public static void writeReport(List<Employee> employees, Path outputPath) throws IOException {
        // Group by department
        Map<String, List<Employee>> byDept = new TreeMap<>();
        for (Employee emp : employees) {
            byDept.computeIfAbsent(emp.department(), k -> new ArrayList<>()).add(emp);
        }

        Files.createDirectories(outputPath.getParent());   // ensure output directory exists

        try (PrintWriter writer = new PrintWriter(
                Files.newBufferedWriter(outputPath, StandardCharsets.UTF_8))) {

            writer.println("EMPLOYEE REPORT");
            writer.println("=".repeat(60));
            writer.printf("Generated: %s%n%n", java.time.LocalDateTime.now()
                          .toString().substring(0, 19));

            // Per-department summary
            for (Map.Entry<String, List<Employee>> entry : byDept.entrySet()) {
                String dept = entry.getKey();
                List<Employee> deptEmployees = entry.getValue();

                double totalSalary = deptEmployees.stream()
                    .mapToDouble(Employee::salary).sum();
                double avgSalary = totalSalary / deptEmployees.size();

                writer.printf("Department: %s (%d employees)%n", dept, deptEmployees.size());
                writer.printf("  Average salary: $%,.2f | Total: $%,.2f%n", avgSalary, totalSalary);
                writer.println("  " + "-".repeat(55));

                deptEmployees.stream()
                    .sorted(Comparator.comparingDouble(Employee::salary).reversed())
                    .forEach(emp -> writer.println("  " + emp));

                writer.println();
            }

            // Overall summary
            double grandTotal = employees.stream().mapToDouble(Employee::salary).sum();
            writer.println("=".repeat(60));
            writer.printf("Total employees: %d%n", employees.size());
            writer.printf("Total payroll:   $%,.2f%n", grandTotal);
            writer.printf("Average salary:  $%,.2f%n", grandTotal / employees.size());
        }
    }

    public static void main(String[] args) {
        // Create a sample input CSV file
        Path inputPath  = Path.of("employees.csv");
        Path outputPath = Path.of("reports/employee-report.txt");

        String csvContent = """
                name,age,department,salary
                Alice,30,Engineering,95000
                Bob,25,Design,72000
                Charlie,35,Engineering,110000
                Dave,28,Marketing,68000
                Eve,40,Engineering,125000
                Frank,invalid_age,Design,75000
                Grace,22,Marketing,55000
                Henry,55,Design,88000
                ,30,Engineering,90000
                Iris,31,Marketing,71000
                """;

        try {
            Files.writeString(inputPath, csvContent, StandardCharsets.UTF_8);
            System.out.println("Input file written: " + inputPath.toAbsolutePath());

            List<Employee> employees = loadEmployees(inputPath);
            System.out.println("Loaded " + employees.size() + " valid employees.");

            writeReport(employees, outputPath);
            System.out.println("Report written to: " + outputPath.toAbsolutePath());

            // Print the report to console as well
            System.out.println("\n" + "=".repeat(60));
            System.out.println(Files.readString(outputPath, StandardCharsets.UTF_8));

        } catch (IOException e) {
            System.err.println("Fatal error: " + e.getMessage());
            e.printStackTrace();
        }
    }
}

This example demonstrates:

Files.writeString() to create the input file programmatically
Files.newBufferedReader() with explicit charset for reading
BufferedReader.readLine() for line-by-line processing
Error recovery — invalid rows are logged and skipped, not fatal
Files.createDirectories() to ensure the output path exists
PrintWriter wrapping a BufferedWriter for formatted output
Files.readString() to echo the result
try-with-resources on every stream — no resource leaks
TreeMap for sorted department grouping
Stream API preview — mapToDouble, sum, sorted, forEach (covered in full in Lecture 12)

Summary

Java has three generations of file I/O: java.io.File (legacy), java.io streams (still used), and java.nio.file NIO.2 (modern). Prefer NIO.2 for new code.
Path represents a file system location. Files provides static utility methods for all file system operations. Both live in java.nio.file.
For reading: Files.readAllLines() for small files, Files.newBufferedReader() for large line-by-line reading, Files.lines() for lazy stream processing, Files.readAllBytes() for binary data.
For writing: Files.writeString() for simple text, Files.write() for lists of lines, Files.newBufferedWriter() for large or structured output. Use StandardOpenOption.APPEND to append.
Always specify StandardCharsets.UTF_8 explicitly. Never rely on the system default charset.
Byte streams (InputStream/OutputStream) are for binary data. Character streams (Reader/Writer) are for text. Bridge them with InputStreamReader/OutputStreamWriter when needed.
Always buffer your streams. Unbuffered I/O makes one system call per byte — dramatically slow. BufferedInputStream/BufferedReader batch reads into chunks.
Use try-with-resources for every stream, reader, writer, and Files.lines()/Files.list()/Files.walk() call — they are all AutoCloseable.
Files.createDirectories() creates a full path of directories. Files.copy(), Files.move(), and Files.delete() handle common file operations. Walking a tree with Files.walk() enables recursive operations.
System.in, System.out, and System.err are standard streams. Use Scanner or BufferedReader on System.in for user input.

Exercises

Exercise 1 — File statistics Write a program that accepts a file path as a command-line argument (args[0]). If the file exists, print: its absolute path, size in bytes, last-modified time, number of lines, number of words (split on whitespace), number of characters (excluding newlines), and the 5 most frequent words (case-insensitive). Handle FileNotFoundException gracefully with a clear message.

Exercise 2 — CSV to formatted report Create a CSV file grades.csv with columns student,math,english,science and at least 8 rows of data. Write a program that reads it, computes each student’s average, finds the class average, identifies the top and bottom performers, and writes a formatted report to grades-report.txt. Use BufferedReader for reading and PrintWriter for writing.

Exercise 3 — File backup utility Write a BackupUtil class with a method backup(Path sourceDir, Path backupDir) that:

Creates backupDir if it does not exist
Recursively copies all .txt and .csv files from sourceDir to backupDir, preserving the relative directory structure
Skips files that already exist in backupDir and are newer than the source (use Files.getLastModifiedTime())
Prints a summary: files copied, files skipped, total bytes copied

Exercise 4 — Log file analyzer Write a LogAnalyzer class that reads a log file where each line has the format: [LEVEL] TIMESTAMP: message (e.g., [ERROR] 2024-01-15T10:30:00: Database connection failed). Use Files.lines() and the Stream API to:

Count lines by level (INFO, WARN, ERROR, DEBUG)
Extract all ERROR lines and write them to a separate errors.log file
Find the most common error message (the part after the timestamp)
Print a summary report to the console

Exercise 5 — Word frequency counter Write a program that reads a large text file (use any book from Project Gutenberg saved as a .txt file) using Files.lines() and counts the frequency of every word (case-insensitive, stripping punctuation). Store results in a HashMap. Write the top 20 words and their counts to word-frequency.txt, one per line, formatted as word: count and sorted by count descending. Compare the runtime with a version that reads with Files.readAllLines() — which is faster for large files?

Exercise 6 — Config file reader and writer Implement a ConfigManager class backed by a LinkedHashMap that:

Reads a config.properties file on construction (format: key=value, # for comments, blank lines ignored)
Provides get(key), get(key, defaultValue), set(key, value), and remove(key)
Writes the current state back to the file with save(), preserving comments and blank lines from the original
Detects if the file has been modified externally since last loaded (use Files.getLastModifiedTime()) and throws StaleConfigException on save() if so

Up next: Lecture 12 — Lambda, Stream API & Optional — the final lecture, where you will bring together everything from this series and learn to write clean, expressive, functional-style Java using lambdas, method references, the full Stream API pipeline, and Optional for null-safe code.