Today I need to preserve the EOF (End Of the File) of each file when processing multi-file read and merge, and I want to be able to read line by line to avoid the burden of loading the entire file at once on the memory.
In the case, we have to exclude “open() and then read()” or “readlines()“, because these methods will all be loaded whole file into memory.
Suppose the file format I want to read is as follows:
t
o
d
a
y
i
s
a
n
i
c
e
d
a
y
I would have thought the easiest way would be:
# coding: utf-8
def main():
f = open("file.txt", "r")
for line in f:
print(repr(line))
f.close()
if __name__ == "__main__":
main()
Output:
't\n'
'o\n'
'd\n'
'a\n'
'y\n'
'\n'
'i\n'
's\n'
'\n'
'a\n'
'\n'
'n\n'
'i\n'
'c\n'
'e\n'
'\n'
'd\n'
'a\n'
'y\n'
Then, you can see that I did not catch the EOF at the end. In Python, EOF should be treated as an empty string.
Later, I found that it is more appropriate to use readline() (not readlines()) to deal with this kink of situation to confirm EOF.
Use the following equation code:
# coding: utf-8
def main():
f = open("file.txt", "r")
while True:
line = f.readline()
print(repr(line))
if not line:
break
f.close()
if __name__ == "__main__":
main()
Output:
't\n'
'o\n'
'd\n'
'a\n'
'y\n'
'\n'
'i\n'
's\n'
'\n'
'a\n'
'\n'
'n\n'
'i\n'
'c\n'
'e\n'
'\n'
'd\n'
'a\n'
'y\n'
''
As the output shows, this time we caught an empty string at the end of the file which can be considered as the EOF position. This is the effect I want to achieve.
Maybe it is a useless and tricky record, but it is really a good question that made me understand more about python’s mechanism for reading files. Now the problem bas been solved, so I write it down here as a record.
References
- https://stackoverflow.com/questions/15599639/what-is-the-perfect-counterpart-in-python-for-while-not-eof
- https://www.geeksforgeeks.org/correcting-eof-error-in-python-in-codechef/