Skip to main content

How to read and write large files in Python

How to read and write large files in Python.

Here is a step-by-step tutorial on how to read and write large files in Python:

Reading Large Files

Reading large files in Python can be a memory-intensive task. To efficiently read large files, we can use the following approach:

  1. Open the file using the open() function, specifying the file path and the mode as r (for reading).

    file_path = 'path/to/your/file.txt'
    file = open(file_path, 'r')
  2. To read the file line by line, use a for loop and iterate over the file object:

    for line in file:
    # process each line
    print(line)

    This approach reads one line at a time, minimizing memory usage.

  3. After reading the file, close it using the close() method:

    file.close()

    It is important to close the file to release system resources.

Here's an example that reads a large file and prints each line:

file_path = 'path/to/your/file.txt'
file = open(file_path, 'r')

for line in file:
print(line)

file.close()

Writing Large Files

Writing large files in Python can also be memory-intensive. To write large files efficiently, follow these steps:

  1. Open the file using the open() function, specifying the file path and the mode as w (for writing).

    file_path = 'path/to/your/file.txt'
    file = open(file_path, 'w')

    If the file already exists, it will be overwritten.

  2. Use the write() method to write data to the file. You can write strings or use a loop to write multiple lines.

    file.write('Hello, World!\n')
    file.write('Welcome to Python file writing.\n')

    Note that you need to add the newline character \n at the end of each line if you want to write multiple lines.

  3. After writing the file, close it using the close() method:

    file.close()

    Closing the file is important to ensure that all data is written and resources are released.

Here's an example that writes multiple lines to a large file:

file_path = 'path/to/your/file.txt'
file = open(file_path, 'w')

file.write('Hello, World!\n')
file.write('Welcome to Python file writing.\n')

file.close()

Remember to replace 'path/to/your/file.txt' with the actual file path you want to read from or write to.

That's it! You now know how to efficiently read and write large files in Python using a step-by-step approach.