Learn
B2.5.1 | FILE PROCESSING
01 | KEEP DATA AFTER THE PROGRAM ENDS
A quiz program can keep a score in a variable while it runs. When that process ends, the variable is gone. Saving text to a file lets a later run retrieve stored data instead of starting from nothing.
A text file stores characters using an encoding. A program opens the file through a file object or stream, performs operations, then closes that resource. Closing is different from deleting: the stored file remains.
This common SL/HL lesson constructs text-file programs using reading, writing and appending. We will process lines sequentially and keep the file’s contents separate from the program’s variables.
02 | SET UP A SMALL PROJECT YOU CAN INSPECT
In VS Code, open a project folder and create a Python file such as file_practice.py. Beside it, create scores.txt with three lines containing 8, 5 and 7. Save both files before running the examples.
These examples use the relative path "scores.txt". A relative path is resolved from the program’s current working directory, which is not automatically always the script’s folder. If a file is not found, check the folder from which the program is running.
Run the Python file with the Run button, then inspect the text file in the editor after each completed write. For the independent examples below, reset the sample contents when the instructions require it.
03 | CHOOSE THE MODE BEFORE OPENING
| Python mode | Main purpose | Existing file | Missing file |
|---|---|---|---|
| "r" | Read text | Contents preserved | Raises FileNotFoundError |
| "w" | Write replacement text | Truncated immediately on successful open | Creates file if parent/location permits |
| "a" | Append text | Writes added at end | Creates file if parent/location permits |
Opening in "w" can erase existing contents before a single write occurs. Appending in "a" preserves the existing text, but does not automatically insert a newline before the new text.
The parent folder must exist and permissions must allow the operation. Choosing a creation mode does not make every path valid.
Use a mode matching the requirement: reading stored data, replacing a report, or adding a new log entry are different jobs.
04 | OPEN, READ AND CLOSE EXPLICITLY
file = open("scores.txt", "r", encoding="utf-8")
try:
contents = file.read()
finally:
file.close()
print(contents)
open returns a file object associated with the open file. read reads the remaining text and returns a string; it does not turn each number into an integer. finally ensures this opened resource is closed even if reading raises an exception.
close releases the resource. Later operations using that closed file object fail; opening the same path again creates a new file object with a new reading position.
A simple sequence of read then close can miss closing if an intervening exception occurs. The next example expresses the same cleanup more conveniently.
05 | WITH: CLOSE AUTOMATICALLY
with open("scores.txt", "r", encoding="utf-8") as file:
contents = file.read()
print(contents)
The with statement manages the file resource. Once control leaves its block, the file is closed, including when the block raises an exception. The variable contents still holds the returned string and can be used afterwards.
Indent operations that need the open file inside the block. Printing a stored string afterwards is fine; trying file.read() afterwards would use a closed resource.
encoding="utf-8" makes the text encoding explicit. A program must decode a file using an appropriate encoding for the bytes it contains.
06 | READ IS NOT THE SAME AS PRINT
For the sample file, read returns one string represented as "8\n5\n7\n". Here \n represents a newline character; it is not two visible characters stored as a backslash and n.
print(contents) displays those newline characters as line breaks. Since print adds its own ending newline by default, an extra blank line can appear after text that already ends with a newline.
To inspect the exact string, print(repr(contents)) shows an escaped representation. To display the contents without adding another newline, use print(contents, end="").
07 | READLINE: ONE LINE FROM THE CURRENT POSITION
with open("scores.txt", "r", encoding="utf-8") as file:
first = file.readline()
second = file.readline()
rest = file.read()
print(repr(first))
print(repr(second))
print(repr(rest))
For the initial sample, the representations are '8\n', '5\n' and '7\n'. Reading advances the current position, so the second call does not reread the first line.
readline includes the line-ending newline when present. A final line without a newline is still returned as ordinary nonempty text.
After reading the whole file, another read or readline returns an empty string. That is the end-of-file indication in these Python text-reading operations.
08 | EMPTY LINE AND END OF FILE ARE DIFFERENT
| Returned Python string | Meaning |
|---|---|
| "8\n" | A line containing 8 and a newline |
| "\n" | A blank line; still a line |
| "7" | A final line without a newline |
| "" | No more characters available at end of file |
A blank line is not the same as end of file. If you strip a line before checking the EOF condition, a blank line can become "" and accidentally stop processing too early.
Check the raw returned line against "" first, then clean or validate the line’s contents according to the file format.
09 | CONSTRUCT A READLINE LOOP
with open("scores.txt", "r", encoding="utf-8") as file:
line = file.readline()
while line != "":
print(line.rstrip("\n"))
line = file.readline()
The first read primes the loop. Each iteration processes the current line and obtains the next one. The loop ends when readline returns "", not when it sees a blank line.
rstrip("\n") removes trailing newline characters for display. General strip() also removes other leading and trailing whitespace, which can be useful for numbers but may change meaningful spaces in other data.
Forgetting the final read repeats the same line forever. Reading twice without processing the intermediate result can skip data.
10 | ITERATE OVER LINES DIRECTLY
with open("scores.txt", "r", encoding="utf-8") as file:
for line in file:
print(line.rstrip("\n"))
The file iterator supplies each line in turn and stops at end of file. It avoids manually priming and updating a readline loop while retaining sequential processing.
The file stays open during iteration and closes when the with block ends. The program need not load the entire file into a list first.
This is often a clear choice when the requirement is simply to visit every line once. Knowing the explicit readline version is still useful when tracing file position and EOF.
11 | CONVERT STORED TEXT BEFORE CALCULATING
total = 0
count = 0
with open("scores.txt", "r", encoding="utf-8") as file:
for line in file:
score = int(line.strip())
total = total + score
count = count + 1
print("Total:", total)
if count > 0:
print("Average:", total / count)
else:
print("No scores")
For the initial sample, total is 20 and the average is 20 / 3. int converts each cleaned text line into a number. Without conversion, adding strings could concatenate text instead of calculating a numeric total.
This introductory format assumes every line contains a valid integer. A blank or invalid line would raise ValueError; the later robust example applies an explicit invalid-line policy.
Initialize total and count before the loop. Each update keeps the previous running total and adds the new score.
12 | WRITE REPLACEMENT CONTENTS
with open("report.txt", "w", encoding="utf-8") as file:
file.write("Quiz summary\n")
file.write("Total: 20\n")
The resulting file contains two lines: Quiz summary and Total: 20. write takes a string and adds exactly that text. It does not automatically add spaces or newlines.
The file is truncated when successfully opened in "w", even if the body later writes nothing. Use replacement mode only when replacing the previous contents is intended.
A numeric value must be converted to text, for example file.write(str(total) + "\n") or file.write(f"Total: {total}\n").
13 | APPEND A NEW RECORD
with open("log.txt", "a", encoding="utf-8") as file:
file.write("Practice completed\n")
Every run adds another entry at the end of log.txt. It does not begin again at the start and overwrite the earlier entries. If the file is absent, append mode can create it.
The existing file must follow a compatible record convention. If the last stored line has no final newline, this write joins onto that line before adding its own newline.
For a simple one-record-per-line log, make every normal writer include the terminating newline. Do not assume append mode repairs an existing malformed boundary.
14 | INTERACTIVE: A VIRTUAL TEXT FILE
Experiment with modes on a simulated file. Opening in w clears the virtual contents immediately. Read operations move the reading position; write adds exactly the supplied text. Newline characters are shown escaped in returned strings, while the contents panel displays real line breaks. The simulation does not access your device’s files.
VIRTUAL FILE CONTENTS
OPERATION HISTORY
Try reading a line twice, reading the remainder, then reading again at EOF. Close before reopening in another mode. Reset restores the original three-line sample. This small model uses one handle at a time and omits filesystem permissions and OS-level details.
15 | TRACE A COMPLETE FILE POSITION EXAMPLE
| Operation on initial sample | Returned string / effect | Remaining unread text |
|---|---|---|
| Open r | New handle at start | "8\n5\n7\n" |
| readline() | "8\n" | "5\n7\n" |
| readline() | "5\n" | "7\n" |
| read() | "7\n" | "" |
| readline() | "" (EOF) | "" |
| close() | Handle closed; contents unchanged | Not reading |
Reopening in read mode begins at the start again. read does not always mean “the complete original file”; it reads what remains from the current position.
A trace should show both data returned to variables and effects on the handle or file. Reading does not remove the stored characters from the file.
16 | HANDLE MISSING FILES AND INVALID RECORDS
total = 0
count = 0
try:
with open("scores.txt", "r", encoding="utf-8") as file:
for number, line in enumerate(file, start=1):
try:
score = int(line.strip())
except ValueError:
print("Skipped invalid line", number)
continue
total = total + score
count = count + 1
except FileNotFoundError:
print("scores.txt was not found")
else:
print("Valid scores:", count)
print("Total:", total)
This policy skips invalid lines, including blank ones, and reports their line numbers. Another application might reject the whole file instead; choose and document the required policy.
The outer handler covers a missing input file. Other I/O failures, such as permission problems or decoding errors, are not silently turned into “file missing”.
The outer else reports totals only if the file-processing try completes normally. The with block still closes an opened file when processing leaves it.
17 | COMPLETE PROGRAM: FILE TO SUMMARY REPORT
def create_summary(source_path, report_path):
total = 0
count = 0
with open(source_path, "r", encoding="utf-8") as source:
for line in source:
score = int(line.strip())
total = total + score
count = count + 1
with open(report_path, "w", encoding="utf-8") as report:
report.write(f"Count: {count}\n")
report.write(f"Total: {total}\n")
if count > 0:
report.write(f"Average: {total / count:.2f}\n")
else:
report.write("Average: no data\n")
create_summary("scores.txt", "summary.txt")
For the initial sample, summary.txt contains Count: 3, Total: 20 and Average: 6.67 on separate lines. Source and report paths must refer to different files. The source format here requires a valid integer on every line.
The program finishes reading and closes the input before opening the output for replacement. Invalid source data raises an error before the report is opened, rather than publishing a partial numeric summary.
This is not a guarantee of atomic saving: an I/O failure during output can still leave a partial replacement report. Atomic replacement is a further design concern beyond this introductory example.
18 | STORE AND RETRIEVE A SIMPLE RECORD
name = "Ada"
score = 8
with open("results.txt", "w", encoding="utf-8") as file:
file.write(f"{name},{score}\n")
with open("results.txt", "r", encoding="utf-8") as file:
for line in file:
stored_name, stored_score = line.rstrip("\n").split(",")
print(stored_name, int(stored_score))
The output is Ada 8. A delimiter separates fields within a line, and the newline separates records. Reading must follow the same format that writing used.
This deliberately simple format assumes names contain no comma or newline and each record has exactly two fields. Arbitrary names need proper quoting and escaping, typically through the csv module rather than naive split(",").
Do not confuse a sequential text-file format with a relational database table. The application is responsible for this file’s parsing and record conventions.
19 | JAVA: WRITE AND APPEND WITH FILEWRITER
import java.io.FileWriter;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
public class WriteExample {
public static void main(String[] args) {
try (FileWriter file = new FileWriter("notes.txt", StandardCharsets.UTF_8)) {
file.write("First entry\n");
} catch (IOException error) {
System.out.println("Could not write: " + error.getMessage());
return;
}
try (FileWriter file = new FileWriter("notes.txt", StandardCharsets.UTF_8, true)) {
file.write("Second entry\n");
} catch (IOException error) {
System.out.println("Could not append: " + error.getMessage());
}
}
}
This Java 11-or-later example uses explicit UTF-8 constructors. The first writer replaces or creates notes.txt; the second passes true for append and adds a second line.
Try-with-resources closes each FileWriter when its block ends, including exceptional exits. Running the entire example again replaces the old file at its first stage, so it ends with two lines rather than accumulating all earlier runs.
20 | JAVA: BUFFEREDREADER AND END OF FILE
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
public class ReadExample {
public static void main(String[] args) {
try (BufferedReader file = new BufferedReader(
new FileReader("scores.txt", StandardCharsets.UTF_8))) {
String line = file.readLine();
while (line != null) {
System.out.println(line);
line = file.readLine();
}
} catch (IOException error) {
System.out.println("Could not read: " + error.getMessage());
}
}
}
Java BufferedReader.readLine() returns line text without the line terminator and returns null at EOF. A blank line is an empty string, not null. This differs from Python readline, which usually includes the newline and uses "" at EOF.
This Java 11-or-later example reads sequentially and closes both the BufferedReader and its wrapped reader through resource cleanup. Do not transfer a language’s EOF condition without checking that API’s contract.
21 | JAVA: SCANNER AS ANOTHER LINE READER
import java.io.File;
import java.io.FileNotFoundException;
import java.nio.charset.StandardCharsets;
import java.util.Scanner;
public class ScanExample {
public static void main(String[] args) {
try (Scanner file = new Scanner(new File("scores.txt"),
StandardCharsets.UTF_8.name())) {
while (file.hasNextLine()) {
String line = file.nextLine();
System.out.println(line);
}
} catch (FileNotFoundException error) {
System.out.println("scores.txt was not found or could not be opened");
}
}
}
hasNextLine checks whether another line is available; nextLine obtains it without the line separator. Scanner can also parse tokens, but this example consistently uses line operations.
Keep parsing and error handling aligned with the chosen file format. A line reader does not automatically validate a numeric score or the number of fields in a record.
22 | STREAM LARGE FILES WHEN YOU CAN
read() stores all remaining text in memory, so its result size grows with the file. Iterating over lines can process a running total without retaining every previous line.
For n characters, a complete scan has O(n) character-processing work under the usual model. A streaming solution’s working memory can depend on the longest line and parser state rather than the whole file.
If you append every line to a list anyway, you again retain growing data. The benefit comes from processing and discarding previous lines when the objective allows it.
23 | UPDATE RECORDS BY REWRITING THE TEXT
A simple sequential text file does not offer an automatic “update this record” instruction. A common approach reads and parses records, changes the intended value, then writes a revised file.
Opening the original in "w" before reading it destroys its old contents. Keep input and output responsibilities clear and validate the revised data before replacing a valuable original.
For production data, a temporary file followed by a carefully handled replacement can reduce the chance of losing the original on failure. This lesson’s basic write examples illustrate modes rather than a complete durable storage system.
24 | TEST CONTENTS AS WELL AS SCREEN OUTPUT
| Test | Expected observation |
|---|---|
| Read initial sample | Three lines processed; total 20 |
| Empty input file | No records; no division by zero |
| Final line has no newline | Final record still processed |
| Blank line | Handled by format policy; not confused with EOF |
| Open existing file with w | Old contents removed immediately |
| Append twice | Earlier text preserved; two additions |
| Missing read file | Appropriate missing-file response |
| Wrong working directory | Path checked; clear failure rather than unrelated data |
A program can print a success message even when its stored contents are wrong. Reopen the output and compare exact text, separators, record count and expected numeric values.
Use small disposable test files while learning modes so you can inspect each effect directly.
25 | COMMON ERRORS TO REPAIR
| Error | Why it matters | Correction |
|---|---|---|
| Use w to add a log entry | Old contents are truncated | Use append mode |
| Forget newline in repeated writes | Records can run together | Write the required separator explicitly |
| Add text scores without conversion | Concatenation or type error | Parse numeric values first |
| Treat blank line as EOF after stripping | Later records can be skipped | Check raw EOF first |
| Read again after close | Resource is no longer open | Keep I/O inside its managed block |
| Use a relative path as if fixed | Working directory may differ | Verify the intended path |
| Assume write means safe permanent save immediately | Buffering and failures can intervene | Close resources and handle the required failure policy |
Closing normally flushes buffered output, but it is not a universal guarantee against power loss or every storage failure. Keep the file-operation contract distinct from stronger durability requirements.
26 | CONSTRUCT A COMPLETE FILE SOLUTION
Specify the text format, encoding, path, opening mode and missing/invalid-data policy. Then open the required resource, perform the sequential processing and ensure it is closed on exit.
For a read task, track what each operation returns and how it advances the reading position. For write or append, show the exact strings and separators stored, including newline characters.
Test the stored result as well as the display. A complete constructed program covers the empty case, correct mode choice, necessary type conversions and cleanup.