Chapter 12: Reading Files with Open
Master Python file handling! Learn how to open files safely using Python's built-in open() function, manage file descriptors with context managers (with statement), and efficiently parse text datasets using read(), readline(), and readlines().
Why Do We Need File I/O in Data Science?
In Data Science and analytics, raw data is almost always stored in external files on disk (such as .txt, .csv, .json, or log files). Python's open() function acts as a bridge between your program in RAM and files on your hard drive, allowing you to stream, extract, and transform information line by line.
12.1 Opening Files, Modes & Attributes
To access data from a file, we use Python's built-in open() function to create a file object (or file handle):
# Open file in read mode
file_object = open("Example1.txt", "r")
# Inspect attributes
print("File name:", file_object.name)
print("File mode:", file_object.mode)
# Always close files after usage!
file_object.close()
print("Is file closed?", file_object.closed)
The open() function takes two primary arguments:
- File Path: The filename or full directory path pointing to the target file.
- Access Mode:
'r'— Read mode (Default). Opens file for reading. Raises an error if file does not exist.'w'— Write mode. Overwrites existing contents or creates a new file.'a'— Append mode. Appends new data to the end of the file without erasing prior content.
Once instantiated, file objects have built-in data attributes to inspect their properties:
file_object.name— Returns the filename/path as a string.file_object.mode— Returns the mode string (e.g.'r').file_object.closed— Boolean indicating whether the file descriptor has been closed (True/False).
Select a file path and mode to open a file object in Python memory. Inspect its properties and test closing the handle manually.
12.2 The with Statement (Context Manager)
Calling file.close() manually can lead to memory leaks or locked files if your program crashes before reaching the close line. The best practice in Python is to use the with statement as a context manager:
with open("Example1.txt", "r") as file1:
file_stuff = file1.read()
print("Content:\n", file_stuff)
# Outside the indented block, file1 is automatically closed!
print("Is file closed outside with block?", file1.closed)
Key Rules of the with Statement:
- Python executes all code inside the indented block while keeping the file descriptor open.
- Upon exiting the indented block (even if an error occurs), Python automatically closes the file!
- You cannot read from the file object outside the indented block (raises
ValueError: I/O operation on closed file). - However, any variables created inside the block (like
file_stuff) remain stored in memory and accessible outside the block!
Step through code execution to see how the with block manages file handle closure and variable accessibility inside vs. outside the local scope.
with open("Example1.txt", "r") as file1:
file_stuff = file1.read()
print("file1.closed:", file1.closed)
print("Content stored:", file_stuff[:12])
12.3 Reading File Contents (read, readline, readlines)
Python provides three primary methods for reading data from a file object:
# 1. Read entire content
with open("Example1.txt", "r") as f:
content = f.read()
print("--- Entire File ---")
print(content)
# 2. Read line by line using readline()
with open("Example1.txt", "r") as f:
line1 = f.readline()
line2 = f.readline()
print("\n--- Readline Output ---")
print("Line 1:", line1.strip())
print("Line 2:", line2.strip())
# 3. Read into list of strings using readlines()
with open("Example1.txt", "r") as f:
lines = f.readlines()
print("\n--- Readlines List Output ---")
print(lines)
Try Reading Methods: Click "Try It Yourself" to run and compare read(), readline(), and readlines() outputs side-by-side.
file.read()
Reads the entire contents of the file and stores it as a single string variable, including newline characters (\n).
file.readline()
Reads one single line from the current position. Calling it multiple times sequentially advances through the file line by line.
file.readlines()
Reads all lines into a List of strings, where index 0 is Line 1, index 1 is Line 2, and so on.
Character Buffer Reading with read(N)
You can pass an integer argument N to read(N) to read only N characters at a time. Each call advances an internal file pointer cursor across character cells in the file grid:
Select a reading method or step through character buffer sizes (read(4), read(16), read(5), read(9)) to visualize the file cursor traversing character cells grid in memory.
12.4 Iterating Over File Lines with Loops
When dealing with massive multi-gigabyte datasets, reading the whole file at once with read() or readlines() can consume too much RAM. Instead, you can iterate over a file object directly using a for loop:
with open("Example1.txt", "r") as file1:
for line in file1:
print(line.strip())
This streams and loads one line at a time into memory, making file processing extremely fast and memory-efficient!
Step through the for line in file1: loop to see each line streamed into local iteration memory.
Practice Quiz
Validate your understanding of file objects, context managers, modes, and reading functions in Python.