File I/O & Pathlib

1. Overview#

Almost every real program reads or writes files — configs, logs, datasets, reports. Python gives you two layers for this: the built-in open() for raw file handling, and the modern pathlib module for working with filesystem paths in an object-oriented, cross-platform way.

Prefer pathlib.Path over string-based paths (os.path.join, manual "/" concatenation) in any new code — it's more readable and works identically on Windows, macOS, and Linux.

---

2. Reading & Writing Files#

2.1 The with Statement (Always Use This)#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
with open("notes.txt", "w") as f:
 f.write("First line\n")
 f.write("Second line\n")
# File is automatically closed here, even if an exception occurs

2.2 File Modes#

2.3 Reading Patterns#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
# Read entire file into memory
with open("notes.txt") as f:
 content = f.read()

# Read line by line (memory-efficient for large files)
with open("notes.txt") as f:
 for line in f:
 print(line.strip())

# Read all lines into a list
with open("notes.txt") as f:
 lines = f.readlines()

2.4 Appending#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
with open("log.txt", "a") as f:
 f.write("New log entry\n")

---

3. Working with pathlib#

3.1 Creating and Inspecting Paths#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from pathlib import Path

p = Path("data/reports/summary.csv")

print(p.name) # summary.csv
print(p.stem) # summary
print(p.suffix) # .csv
print(p.parent) # data/reports
print(p.exists()) # True/False
print(p.is_file()) # True/False
print(p.is_dir()) # True/False

3.2 Building Paths (No More Manual String Concatenation)#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from pathlib import Path

base = Path("data")
file_path = base / "reports" / "summary.csv" # / operator joins paths cleanly
print(file_path) # data/reports/summary.csv

3.3 Reading/Writing Directly via Path#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from pathlib import Path

p = Path("notes.txt")
p.write_text("Hello, World!\n")
content = p.read_text()
print(content)

3.4 Directory Operations#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from pathlib import Path

folder = Path("data/output")
folder.mkdir(parents=True, exist_ok=True) # creates nested dirs, no error if it exists

for file in Path("data").glob("*.csv"): # find all CSVs in a directory
 print(file)

for file in Path("data").rglob("*.py"): # recursive glob — searches subdirectories too
 print(file)

3.5 Absolute Paths & the Current Working Directory#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from pathlib import Path

print(Path.cwd()) # current working directory
print(Path("notes.txt").resolve()) # absolute path
print(Path.home()) # user's home directory

---

4. Structured File Formats#

4.1 JSON#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
import json

data = {"name": "Kamal", "role": "Engineer", "skills": ["Python", "AI"]}

# Write
with open("profile.json", "w") as f:
 json.dump(data, f, indent=2)

# Read
with open("profile.json") as f:
 loaded = json.load(f)

print(loaded["skills"])

# String conversion (not file-based)
json_str = json.dumps(data) # dict -> JSON string
parsed = json.loads(json_str) # JSON string -> dict

4.2 CSV#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
import csv

rows = [
 {"name": "Asha", "score": 92},
 {"name": "Ravi", "score": 85},
]

# Write
with open("scores.csv", "w", newline="") as f:
 writer = csv.DictWriter(f, fieldnames=["name", "score"])
 writer.writeheader()
 writer.writerows(rows)

# Read
with open("scores.csv") as f:
 reader = csv.DictReader(f)
 for row in reader:
 print(row["name"], row["score"])

Always pass newline="" when opening a CSV file for writing on Windows — otherwise the csv module can insert extra blank lines.

---

5. Error Handling for File Operations#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
from pathlib import Path

def load_config(path):
 file_path = Path(path)
 try:
 return file_path.read_text()
 except FileNotFoundError:
 print(f"Config file not found: {path}")
 return None
 except PermissionError:
 print(f"No permission to read: {path}")
 return None

---

6. Common Pitfalls#

INCORRECT: Forgetting to Close Files#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
f = open("notes.txt", "w")
f.write("data")
# File never closed if an exception happens before f.close()

CORRECT: Fix — Always Use with#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
with open("notes.txt", "w") as f:
 f.write("data")

INCORRECT: Reading a Huge File Entirely into Memory#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
with open("huge_log.txt") as f:
 lines = f.readlines() # could consume gigabytes of RAM

CORRECT: Fix — Iterate Line by Line#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
with open("huge_log.txt") as f:
 for line in f:
 process(line) # one line in memory at a time

---

7. Summary & Best Practices Checklist#

• Always use with open(...) — never manually call .close().
• Prefer pathlib.Path over raw strings for any path manipulation.
• Use .read_text() / .write_text() for simple whole-file text operations.
• Iterate over large files line by line instead of loading them fully.
• Use json.dump/json.load for structured config or API-shaped data.
• Use csv.DictReader/DictWriter for tabular data with headers.
• Catch FileNotFoundError and PermissionError explicitly for user-facing tools.