File I/O & Pathlib

1. Overview#

Almost every real program reads or writes files — configs, logs, datasets, reports. Python gives you two layers for this: the built-in open() for raw file handling, and the modern pathlib module for working with filesystem paths in an object-oriented, cross-platform way.

---

2. Reading & Writing Files#

2.1 The with Statement (Always Use This)#

Python
CopyAsk AI
with open("notes.txt", "w") as f:
    f.write("First line\n")
    f.write("Second line\n")
# File is automatically closed here, even if an exception occurs

2.2 File Modes#

2.3 Reading Patterns#

Python
CopyAsk AI
# Read entire file into memory
with open("notes.txt") as f:
    content = f.read()

    # Read line by line (memory-efficient for large files)
    with open("notes.txt") as f:
        for line in f:
            print(line.strip())

            # Read all lines into a list
            with open("notes.txt") as f:
                lines = f.readlines()

2.4 Appending#

Python
CopyAsk AI
with open("log.txt", "a") as f:
    f.write("New log entry\n")

---

3. Working with pathlib#

3.1 Creating and Inspecting Paths#

Python
CopyAsk AI
from pathlib import Path

p = Path("data/reports/summary.csv")

print(p.name)  # summary.csv
print(p.stem)  # summary
print(p.suffix)  # .csv
print(p.parent)  # data/reports
print(p.exists())  # True/False
print(p.is_file())  # True/False
print(p.is_dir())  # True/False

3.2 Building Paths (No More Manual String Concatenation)#

Python
CopyAsk AI
from pathlib import Path

base = Path("data")
file_path = base / "reports" / "summary.csv"  # / operator joins paths cleanly
print(file_path)  # data/reports/summary.csv

3.3 Reading/Writing Directly via Path#

Python
CopyAsk AI
from pathlib import Path

p = Path("notes.txt")
p.write_text("Hello, World!\n")
content = p.read_text()
print(content)

3.4 Directory Operations#

Python
CopyAsk AI
from pathlib import Path

folder = Path("data/output")
folder.mkdir(parents=True, exist_ok=True)  # creates nested dirs, no error if it exists

for file in Path("data").glob("*.csv"):  # find all CSVs in a directory
    print(file)

for file in Path("data").rglob("*.py"):  # recursive glob — searches subdirectories too
    print(file)

3.5 Absolute Paths & the Current Working Directory#

Python
CopyAsk AI
from pathlib import Path

print(Path.cwd())  # current working directory
print(Path("notes.txt").resolve())  # absolute path
print(Path.home())  # user's home directory

---

4. Structured File Formats#

4.1 JSON#

Python
CopyAsk AI
import json

data = {"name": "Kamal", "role": "Engineer", "skills": ["Python", "AI"]}

# Write
with open("profile.json", "w") as f:
    json.dump(data, f, indent=2)

# Read
with open("profile.json") as f:
    loaded = json.load(f)

print(loaded["skills"])

# String conversion (not file-based)
json_str = json.dumps(data)  # dict -> JSON string
parsed = json.loads(json_str)  # JSON string -> dict

4.2 CSV#

Python
CopyAsk AI
import csv

rows = [
    {"name": "Asha", "score": 92},
    {"name": "Ravi", "score": 85},
]

# Write
with open("scores.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["name", "score"])
    writer.writeheader()
    writer.writerows(rows)

    # Read
    with open("scores.csv") as f:
        reader = csv.DictReader(f)
        for row in reader:
            print(row["name"], row["score"])

---

5. Error Handling for File Operations#

Python
CopyAsk AI
from pathlib import Path

def load_config(path):
    file_path = Path(path)
    try:
        return file_path.read_text()
    except FileNotFoundError:
        print(f"Config file not found: {path}")
        return None
    except PermissionError:
        print(f"No permission to read: {path}")
        return None

print(load_config("missing_config.json"))

---

6. Common Pitfalls#

INCORRECT: Forgetting to Close Files#

Python
CopyAsk AI
f = open("notes.txt", "w")
f.write("data")
# File never closed if an exception happens before f.close()

CORRECT: Fix — Always Use with#

Python
CopyAsk AI
with open("notes.txt", "w") as f:
    f.write("data")

INCORRECT: Reading a Huge File Entirely into Memory#

Python
CopyAsk AI
with open("huge_log.txt") as f:
    lines = f.readlines()  # could consume gigabytes of RAM

CORRECT: Fix — Iterate Line by Line#

Python
CopyAsk AI
with open("huge_log.txt") as f:
    for line in f:
        process(line)  # one line in memory at a time

---

7. Summary & Best Practices Checklist#

• Always use with open(...) — never manually call .close().
• Prefer pathlib.Path over raw strings for any path manipulation.
• Use .read_text() / .write_text() for simple whole-file text operations.
• Iterate over large files line by line instead of loading them fully.
• Use json.dump/json.load for structured config or API-shaped data.
• Use csv.DictReader/DictWriter for tabular data with headers.
• Catch FileNotFoundError and PermissionError explicitly for user-facing tools.