Chapter 1 of ?
python 16 min read

Python for Data Science — Chapter 12: Reading Files with Open

Module 6 — File I/O & Data Operations

Chapter 12: Reading Files with Open

Master Python file handling! Learn how to open files safely using Python's built-in open() function, manage file descriptors with context managers (with statement), and efficiently parse text datasets using read(), readline(), and readlines().

Why Do We Need File I/O in Data Science?

In Data Science and analytics, raw data is almost always stored in external files on disk (such as .txt, .csv, .json, or log files). Python's open() function acts as a bridge between your program in RAM and files on your hard drive, allowing you to stream, extract, and transform information line by line.

12.1 Opening Files, Modes & Attributes

To access data from a file, we use Python's built-in open() function to create a file object (or file handle):

# Open file in read mode
file_object = open("Example1.txt", "r")

# Inspect attributes
print("File name:", file_object.name)
print("File mode:", file_object.mode)

# Always close files after usage!
file_object.close()
print("Is file closed?", file_object.closed)

The open() function takes two primary arguments:

  • File Path: The filename or full directory path pointing to the target file.
  • Access Mode:
    • 'r'Read mode (Default). Opens file for reading. Raises an error if file does not exist.
    • 'w'Write mode. Overwrites existing contents or creates a new file.
    • 'a'Append mode. Appends new data to the end of the file without erasing prior content.

Once instantiated, file objects have built-in data attributes to inspect their properties:

  • file_object.name — Returns the filename/path as a string.
  • file_object.mode — Returns the mode string (e.g. 'r').
  • file_object.closed — Boolean indicating whether the file descriptor has been closed (True/False).
Interactive File Handle Inspector

Select a file path and mode to open a file object in Python memory. Inspect its properties and test closing the handle manually.

File Object State (File 1) Unopened
file.name:None
file.mode:None
file.closed:True
Python ConsoleMEMORY LOG
>>> # Click open() to initialize file handle

12.2 The with Statement (Context Manager)

Calling file.close() manually can lead to memory leaks or locked files if your program crashes before reaching the close line. The best practice in Python is to use the with statement as a context manager:

with open("Example1.txt", "r") as file1:
    file_stuff = file1.read()
    print("Content:\n", file_stuff)

# Outside the indented block, file1 is automatically closed!
print("Is file closed outside with block?", file1.closed)

Key Rules of the with Statement:

  1. Python executes all code inside the indented block while keeping the file descriptor open.
  2. Upon exiting the indented block (even if an error occurs), Python automatically closes the file!
  3. You cannot read from the file object outside the indented block (raises ValueError: I/O operation on closed file).
  4. However, any variables created inside the block (like file_stuff) remain stored in memory and accessible outside the block!
Interactive Scope & Context Manager Visualizer

Step through code execution to see how the with block manages file handle closure and variable accessibility inside vs. outside the local scope.

Scope: Outside with Block
# Python Context Scope
with open("Example1.txt", "r") as file1:
    file_stuff = file1.read()
print("file1.closed:", file1.closed)
print("Content stored:", file_stuff[:12])
Variable & Memory State
file1.closed:True (No Handle)
file_stuff:Uninitialized
Can Read Handle?No
Scope Execution Log
Ready to step execution...

12.3 Reading File Contents (read, readline, readlines)

Python provides three primary methods for reading data from a file object:

# 1. Read entire content
with open("Example1.txt", "r") as f:
    content = f.read()
    print("--- Entire File ---")
    print(content)

# 2. Read line by line using readline()
with open("Example1.txt", "r") as f:
    line1 = f.readline()
    line2 = f.readline()
    print("\n--- Readline Output ---")
    print("Line 1:", line1.strip())
    print("Line 2:", line2.strip())

# 3. Read into list of strings using readlines()
with open("Example1.txt", "r") as f:
    lines = f.readlines()
    print("\n--- Readlines List Output ---")
    print(lines)

Try Reading Methods: Click "Try It Yourself" to run and compare read(), readline(), and readlines() outputs side-by-side.

file.read()

Reads the entire contents of the file and stores it as a single string variable, including newline characters (\n).

file.readline()

Reads one single line from the current position. Calling it multiple times sequentially advances through the file line by line.

file.readlines()

Reads all lines into a List of strings, where index 0 is Line 1, index 1 is Line 2, and so on.

Character Buffer Reading with read(N)

You can pass an integer argument N to read(N) to read only N characters at a time. Each call advances an internal file pointer cursor across character cells in the file grid:

Interactive Character Cursor & Reading Lab

Select a reading method or step through character buffer sizes (read(4), read(16), read(5), read(9)) to visualize the file cursor traversing character cells grid in memory.

Python Console OutputBUFFER STATE
>>> # Click a reading method above to read file content

12.4 Iterating Over File Lines with Loops

When dealing with massive multi-gigabyte datasets, reading the whole file at once with read() or readlines() can consume too much RAM. Instead, you can iterate over a file object directly using a for loop:

with open("Example1.txt", "r") as file1:
    for line in file1:
        print(line.strip())

This streams and loads one line at a time into memory, making file processing extremely fast and memory-efficient!

Interactive Loop Line Streamer

Step through the for line in file1: loop to see each line streamed into local iteration memory.

Iteration: 0 / 3
Loop Stream ConsoleLINE BY LINE
>>> # Click 'Step Loop Iteration' to stream lines

Practice Quiz

Validate your understanding of file objects, context managers, modes, and reading functions in Python.

1. What is the default mode of Python's open() function if not explicitly specified?
2. The with statement automatically closes the file object upon exiting the indented block, even if an exception occurs.
3. Which method reads all lines of a file into a List of strings?
Chapter 11
Done with this chapter?
Mark it complete to track your progress and unlock your certificate.
Next Up

Learner Reviews

Write a Review
Share your experience to help other learners.
Your Rating *