Working with Text Files: open, with, Reading and Writing

What will we learn?#

• Opening a file with open and opening modes.
• Why a file must be closed, and how with does it automatically.
• Reading: the whole file, line by line.
• Writing and appending.
• A robust path, one that does not depend on where you run from (we promised in the topic on the computer, the operating system and the network!).

Why do we need this?#

The topic on the computer, the operating system and the network: “everything in RAM disappears when the program ends”. Files are the basic way to save information on Storage: settings, reports, history, data for import and export. The counter that always went back to 1 in memory_test.py? Today we will make it survive between runs.

And in the OOP project you will save all of the system’s information to a file.

Explanation from the basics#

Opening a file#

open(path, mode) asks the operating system (a System Call!) for access to the file, and returns a file object through which you read and write.

Opening modes#

Why does a file need to be closed?#

• Writing to a file is first collected in a buffer in memory, and actually written to the disk only when the file is closed or when the buffer fills up. Without closing, part of the information may not be saved.
• The operating system limits how many files a process can keep open. Files that are not closed “leak”.
• On Windows, an open file is sometimes locked, and other programs cannot access it.

with — automatic closing#

with open("notes.txt", "r") as file:
    content = file.read()
# here the file is already closed - always

with guarantees that the file is closed the moment you leave the block: at a normal end, at a return, and even when an error is thrown. Remember finally from the exceptions and logging topic? with does exactly that, in a short form. Always open files with with.

Ways to read#

\n is the newline character. Every line in the file ends with it. line.strip() removes it (and the spaces at the edges).

Writing#

file.write(text) writes exactly the text, without going to a new line automatically (unlike print). Want a new line? Add "\n" yourself. And write accepts only a string: convert numbers first with str().

A robust path with pathlib#

In the topic on the computer, the operating system and the network we saw that a relative path is measured from the folder you ran from. The professional solution: build the path relative to the location of the .py file itself.

from pathlib import Path

BASE_DIR = Path(__file__).parent      # the folder this .py file is in
DATA_FILE = BASE_DIR / "data.txt"     # "/" joins path parts, on every OS

• __file__ — a built-in variable: the path of the current file.
• Path(...).parent — the folder that contains it.
• / between a Path and a string — joins path parts correctly on every operating system (\ on Windows, / on Mac).

Now DATA_FILE points to the same file no matter where you run from.

Syntax#

from pathlib import Path

path = Path(__file__).parent / "notes.txt"

# write (creates or overwrites)
with open(path, "w", encoding="utf-8") as file:
    file.write("first line\n")

# append (adds to the end)
with open(path, "a", encoding="utf-8") as file:
    file.write("another line\n")

# read everything
with open(path, "r", encoding="utf-8") as file:
    content = file.read()

# read line by line
with open(path, "r", encoding="utf-8") as file:
    for line in file:
        print(line.strip())

path.exists()     # True / False

encoding="utf-8" — always state it. Without it, on Windows the default is a different encoding, and non-English text (like Hebrew) or emoji will be written as gibberish or throw a UnicodeDecodeError.

First code example#

week-2/files/diary.py:

from pathlib import Path

DIARY_FILE = Path(__file__).parent / "diary.txt"

with open(DIARY_FILE, "w", encoding="utf-8") as file:
    file.write("Day 1: started the course\n")
    file.write("Day 2: learned the terminal\n")

with open(DIARY_FILE, "a", encoding="utf-8") as file:
    file.write("Day 3: networks!\n")

with open(DIARY_FILE, "r", encoding="utf-8") as file:
    content = file.read()

print(content)
print("File size in characters:", len(content))

Output:

Day 1: started the course
Day 2: learned the terminal
Day 3: networks!

File size in characters: 71

And the file diary.txt was created in the folder. Open it in VS Code.

Breaking down the code line by line#

Try it: run it again. Did the file grow? No, because the "w" mode at the start of the script empties it on every run. Change the first opening to "a" and run it twice. What happens?

Another example#

A run counter that really survives (the problem from the topic on the computer, the operating system and the network!) + processing a data file line by line.

week-2/files/run_counter.py:

from pathlib import Path

COUNTER_FILE = Path(__file__).parent / "counter.txt"

def read_count():
    if not COUNTER_FILE.exists():
        return 0
    with open(COUNTER_FILE, "r", encoding="utf-8") as file:
        return int(file.read().strip())

def save_count(count):
    with open(COUNTER_FILE, "w", encoding="utf-8") as file:
        file.write(str(count))

def main():
    count = read_count() + 1
    save_count(count)
    print("This program has run", count, "times")

if __name__ == "__main__":
    main()

Three runs:

This program has run 1 times
This program has run 2 times
This program has run 3 times

Now it works, because the number is saved on Storage between runs, and not only in RAM.

Processing a grades file. Create week-2/files/grades.txt:

Dana,90
Avi,72
Noa,abc
Tom,85

Rina,64

week-2/files/grades_report.py:

from pathlib import Path

GRADES_FILE = Path(__file__).parent / "grades.txt"
REPORT_FILE = Path(__file__).parent / "report.txt"

def load_grades(path):
    grades = {}
    with open(path, "r", encoding="utf-8") as file:
        for line_number, line in enumerate(file, start=1):
            line = line.strip()
            if line == "":
                continue
            name, grade_text = line.split(",")
            try:
                grades[name] = int(grade_text)
            except ValueError:
                print("Skipping line", line_number, "- bad grade:", repr(grade_text))
    return grades

def write_report(grades, path):
    with open(path, "w", encoding="utf-8") as file:
        for name, grade in grades.items():
            status = "PASS" if grade >= 65 else "FAIL"
            file.write(name + ": " + str(grade) + " " + status + "\n")
        average = sum(grades.values()) / len(grades)
        file.write("Average: " + str(round(average, 1)) + "\n")

def main():
    grades = load_grades(GRADES_FILE)
    write_report(grades, REPORT_FILE)
    print("Report written to", REPORT_FILE.name)

if __name__ == "__main__":
    main()

Output on the screen:

Skipping line 3 - bad grade: 'abc'
Report written to report.txt

report.txt:

Dana: 90 PASS
Avi: 72 PASS
Tom: 85 PASS
Rina: 64 FAIL
Average: 77.8

Breakdown:

• for line_number, line in enumerate(file, start=1) — goes over the file line by line, with a line number, to report where the problem is.
• line.strip() — removes the \n. Without it, int("90\n") actually works, but the name would arrive with extra spaces.
• if line == "": continue — skip empty lines.
• name, grade_text = line.split(",") — splitting by comma, and Unpacking into two variables.
• "PASS" if grade >= 65 else "FAIL" — a Conditional Expression: a short if/else in one line, which returns one of two values.
• REPORT_FILE.name — only the file’s name, without the full path.
• Path is not only “a path”. It has useful methods like .exists() and .name.

Common mistake#

Mistake 1: "w" instead of "a" — and deleting data. We saw it above. Before every open(..., "w"), ask yourself: “am I fine with the previous content being deleted?”

Mistake 2: FileNotFoundError because of a relative path.

with open("grades.txt") as file:

Works only when you run from the same folder. From the Run button in VS Code, or from another folder:

FileNotFoundError: [Errno 2] No such file or directory: 'grades.txt'

The fix: Path(__file__).parent / "grades.txt". And for files that may not exist, check .exists() or catch FileNotFoundError.

Mistake 3: writing a number with write.

file.write(42)

TypeError: write() argument must be str, not int

The fix: file.write(str(42)).

Mistake 4: forgetting \n.

file.write("line one")
file.write("line two")

In the file: line oneline two, on one line. write does not add a line break.

Mistake 5: reading twice from the same open file.

with open(path, encoding="utf-8") as file:
    first = file.read()
    second = file.read()
print(repr(second))   # ''

The file “remembers” where it is reading from (a read position). After read() the position is at the end, so the second read returns an empty string.

Practice#

The exercises use only what was taught in this lesson and the lessons before it. Try on your own first, and only then open the solution.

Exercise 1: Writing and reading#

Write two lines to the file diary.txt: Day 1: started and Day 2: networks. Then read the file and print its content.

Expected output:

Day 1: started
Day 2: networks

with open("diary.txt", "w", encoding="utf-8") as file:
    file.write("Day 1: started\n")
    file.write("Day 2: networks\n")

with open("diary.txt", "r", encoding="utf-8") as file:
    print(file.read(), end="")

Day 1: started
Day 2: networks

Exercise 2: w versus a#

with open("log.txt", "w", encoding="utf-8") as file:
    file.write("first\n")
with open("log.txt", "w", encoding="utf-8") as file:
    file.write("second\n")
with open("log.txt", "a", encoding="utf-8") as file:
    file.write("third\n")

What will be in the file at the end? Explain.

with open("log.txt", "w", encoding="utf-8") as file:
    file.write("first\n")
with open("log.txt", "w", encoding="utf-8") as file:
    file.write("second\n")
with open("log.txt", "a", encoding="utf-8") as file:
    file.write("third\n")

with open("log.txt", "r", encoding="utf-8") as file:
    print(file.read(), end="")

second
third

Exercise 3: Line after line#

In the file grades.txt every line is name,grade. Write code that reads the file line after line, and prints every name with its grade. Skip empty lines.

Create the file like this:

with open("grades.txt", "w", encoding="utf-8") as file:
    file.write("dana,90\n\navi,75\nnoa,88\n")

Expected output:

dana 90
avi 75
noa 88

with open("grades.txt", "w", encoding="utf-8") as file:
    file.write("dana,90\n\navi,75\nnoa,88\n")

with open("grades.txt", "r", encoding="utf-8") as file:
    for line in file:
        line = line.strip()
        if not line:
            continue
        name, grade_text = line.split(",")
        print(name, int(grade_text))

dana 90
avi 75
noa 88

Exercise 4: Broken input#

The same grades file, but now it has a broken line. Read the file: a line whose grade is not a number — skip it and print a message with the line number. At the end, print the average of the valid grades.

with open("grades.txt", "w", encoding="utf-8") as file:
    file.write("dana,90\navi,abc\nnoa,80\n")

Expected output:

line 2 skipped: avi,abc
Average: 85.0

with open("grades.txt", "w", encoding="utf-8") as file:
    file.write("dana,90\navi,abc\nnoa,80\n")

grades = []
with open("grades.txt", "r", encoding="utf-8") as file:
    for number, line in enumerate(file, start=1):
        line = line.strip()
        name, grade_text = line.split(",")
        try:
            grades.append(int(grade_text))
        except ValueError:
            print("line", number, "skipped:", line)

print("Average:", sum(grades) / len(grades))

line 2 skipped: avi,abc
Average: 85.0

Exercise 5: A file that does not exist#

1. What will happen if we try to open for reading a file that does not exist?
2. Write a function load_tasks(path) that returns the file’s lines as a list, and an empty list if the file does not exist. Use Path(path).exists().

print(load_tasks("todo.txt"))
with open("todo.txt", "w", encoding="utf-8") as file:
    file.write("buy milk\ncall mom\n")
print(load_tasks("todo.txt"))

Expected output:

[]
['buy milk', 'call mom']

from pathlib import Path


def load_tasks(path):
    if not Path(path).exists():
        return []
    tasks = []
    with open(path, "r", encoding="utf-8") as file:
        for line in file:
            tasks.append(line.strip())
    return tasks


print(load_tasks("todo.txt"))
with open("todo.txt", "w", encoding="utf-8") as file:
    file.write("buy milk\ncall mom\n")
print(load_tasks("todo.txt"))

[]
['buy milk', 'call mom']

Summary#

• open(path, mode, encoding="utf-8"): "r" reading (the default), "w" writing that deletes, "a" appending at the end.
• Always with: the file is closed automatically, even when there is an error.
• Reading: read() (everything), for line in file (line by line, recommended). strip() removes the \n.
• write writes only a str, and without a line break. Add "\n" yourself.
• A robust path: Path(__file__).parent / "file.txt". Check existence with .exists() or catch FileNotFoundError.

Official documentation#

• Reading and Writing Files
• Methods of File Objects
• open()