I am trying to open a file read it's content and write to it by using the contents that were read earlier. I am opening the file in 'a+' mode. I can't use 'r+' mode since it won't create a file if it doesn't exist.
a+ will put the pointer in the end of the file.
You can save it with tell() for later writing.
Then use seek(0,0) to return to file beginning for reading.
tell()
seek()
Default open
Using the default a(+) option, it is not possible, as provided in the documentation:
''mode is an optional string that specifies the mode in which the file
is opened. It defaults to 'r' which means open for reading in text
mode. Other common values are 'w' for writing (truncating the file if
it already exists), 'x' for creating and writing to a new file, and
'a' for appending (which on some Unix systems, means that all writes
append to the end of the file regardless of the current seek position).''
Alternative
Using the default open, this is not possible.However we can of course create our own file handler, that will create a file in r and r+ mode when it doesn't exists.
A minimal working example that works exactly like open(filename, 'r+', *args, **kwargs), would be:
import os
class FileHandler:
def __init__(self, filename, mode='r', buffering=None, encoding=None, errors=None, newline=None, closefd=True):
self.filename = filename
self.mode = mode
self.kwargs = dict(buffering=buffering, encoding=encoding, errors=errors, newline=newline, closefd=closefd)
if self.kwargs['buffering'] is None:
del self.kwargs['buffering']
def __enter__(self):
if self.mode.startswith('r') and not os.path.exists(self.filename):
with open(self.filename, 'w'): pass
self.file = open(self.filename, self.mode, **self.kwargs)
return self.file
def __exit__(self, exc_type, exc_val, exc_tb):
self.file.close()
Now when you use the following code:
with FileHandler("new file.txt", "r+") as file:
file.write("First line\n")
file.write("Second line\n")
file.seek(0, 0)
file.write("Third line\n")
It will generate a new file new file.txt, when it doesn't exists, with the context:
Third line
Second line
If you would use the open you will receive a FileNotFoundError, if the file doesn't exists.
Notes
I am only creating a new file when the mode starts with an r, all other files are handled as would be by the normal open function.
For some reason passing buffering=None, directly to the open function crashes it with an TypeError: an integer is required (got type NoneType), therefore I had to remove it from the key word arguments if it was None. Even though it is the default argument according to the documentation (if any one knows why, please tell me)
Edit
The above code didn't handle the following cases:
file = FileHandler("new file.txt", "r+")
file.seek(0, 0)
file.write("Welcome")
file.close()
In order to support all of the open use cases, the above class can be adjusted by using __getattr__ as follows:
import os
class FileHandler:
def __init__(self, filename, mode='r', buffering=None, encoding=None, errors=None, newline=None, closefd=True):
self.filename = filename
self.mode = mode
self.kwargs = dict(buffering=buffering, encoding=encoding, errors=errors, newline=newline, closefd=closefd)
if self.kwargs['buffering'] is None:
del self.kwargs['buffering']
if self.mode.startswith('r') and not os.path.exists(self.filename):
with open(self.filename, 'w'): pass
self.file = open(self.filename, self.mode, **self.kwargs)
def __enter__(self):
return self.file
def __exit__(self, exc_type, exc_val, exc_tb):
self.file.close()
def __getattr__(self, item):
if hasattr(self.file, item):
return getattr(self.file, item)
raise AttributeError(f"{type(self).__name__}, doesn't have the attribute {item!r}")
Related
I am using the following way to capture the output of the python script in a file:
class Logger(object):
def __init__(self, filename="Default.log"):
self.terminal = sys.stdout
self.log = open(filename, "w")
def write(self, message):
self.terminal.write(message)
self.log.write(message)
def flush(self):
pass
sys.stdout = Logger("logfile.log")
After the output is logged in a file I am then using using the following method to read from the logfile to write its contents to another file
def read_file():
count = 0
with open('logfile.log', 'r') as firstfile, open('file.txt', 'a') as secondfile:
for line in firstfile:
secondfile.write(line)
The problem I am facing is the output of the log file keeps getting refreshed whenever I try to capture the output. How can I create a duplicate file as I want to search for a string and print its occurrences.
Okay I found the issue I had to open the file and the file mode as r+. So it would be:
class Logger(object):
def __init__(self, filename="Default.log"):
self.terminal = sys.stdout
**self.log = open(filename, "r+")**
def write(self, message):
self.terminal.write(message)
self.log.write(message)
def flush(self):
pass
sys.stdout = Logger("logfile.log")
I am making an app which will return one random line from the .txt file. I made a class to implement this behaviour. The idea was to use one method to open file (which will remain open) and the other method which will close it after the app exits. I do not have much experience in working with files hence the following behaviour is strange to me:
In __init__ I called self.open_file() in order to just open it. And it works fine to get self.len. Now I thought that I do not need to call self.open_file() again, but when I call file.get_term()(returns random line) it raises IndexError (like the file is empty), But, if I call file.open_file() method again, everything works as expected.
In addition to this close_file() method raises AttributeError - object has no attribute 'close', so I assumed the file closes automatically somehow, even if I did not use with open.
import random
import os
class Pictionary_file:
def __init__(self, file):
self.file = file
self.open_file()
self.len = self.get_number_of_lines()
def open_file(self):
self.opened = open(self.file, "r", encoding="utf8")
def get_number_of_lines(self):
i = -1
for i, line in enumerate(self.opened):
pass
return i + 1
def get_term_index(self):
term_line = random.randint(0, self.len-1)
return term_line
def get_term(self):
term_line = self.get_term_index()
term = self.opened.read().splitlines()[term_line]
def close_file(self):
self.opened.close()
if __name__ == "__main__":
print(os.getcwd())
file = Pictionary_file("pictionary.txt")
file.open_file() #WITHOUT THIS -> IndexError
file.get_term()
file.close() #AttributeError
Where is my mistake and how can I correct it?
Here in __init__:
self.open_file()
self.len = self.get_number_of_lines()
self.get_number_of_lines() actually consumes the whole file because it iterates over it:
def get_number_of_lines(self):
i = -1
for i, line in enumerate(self.opened):
# real all lines of the file
pass
# at this point, `self.opened` is empty
return i + 1
So when get_term calls self.opened.read(), it gets an empty string, so self.opened.read().splitlines() is an empty list.
file.close() is an AttributeError, because the Pictionary_file class doesn't have the close method. It does have close_file, though.
I would like to create my own pytest fixture where I can insert what I want it to do in the setup and teardown phase.
I am looking for something like this (in this example, i create a file that's needed for the test):
#pytest.fixture
def file(path, content):
def setup():
# check that file does NOT exist
if os.path.isfile(path):
raise Exception('file already exists')
# put contents in the file
with open(path, 'w') as file:
file.write(content)
def teardown():
os.remove(path)
and I would like to be able to use it like this:
def test_my_function(file):
file('/Users/Me/myapplication/info.txt', 'ham, eggs, orange juice')
assert my_function('info') == ['ham', 'eggs', 'orange juice']
I am aware there is already a tempdir fixture in pytest that has similar functionality. Unfortunately, that fixture only creates files somewhere within the /tmp directory, and I need files in my application.
Thanks!
UPDATE:
I'm getting pretty close. The following almost works, but it doesn't set the PATH variable global to the fixture like I expected. I'm wondering if I can create a class instead of a function for my fixture.
#pytest.fixture
def file(request):
PATH = None
def setup(path, content):
PATH = path
# check that file does NOT exist
if os.path.isfile(PATH):
raise Exception('file already exists')
# put contents in the file
with open(PATH, 'w+') as file:
file.write(content)
def teardown():
os.remove(PATH)
request.addfinalizer(teardown)
return setup
This is a bit crazy, but here is a solution:
#pytest.fixture
def file(request):
class File:
def __call__(self, path, content):
self.path = path
# check that file does NOT exist
if os.path.isfile(self.path):
raise Exception('file already exists')
# put contents in the file
with open(self.path, 'w+') as file:
file.write(content)
def teardown(self):
os.remove(self.path)
obj = File()
request.addfinalizer(obj.teardown)
return obj
I'm not quite sure how to implement a context manager for a custom class. Basically it's just accepting a file name in __init__, opening the file in __enter__ and closing it in __exit__.
E.g., like:
class BlaFile:
def __init__(self, file_name):
self.file_name = file_name
def __enter__(self):
self.file = open(self.file_name, "rb")
return self
def __exit__(self, exc_type, exc_val, exc_tb):
self.file.close()
def do_stuff_with_file():
# This will fail when I'm using this class without 'with'.
return self.file.read(1)
However, when I'm not going to instantiate / use the class with the 'with' statement, I will not be able to use any functions of it which access the file, as __enter__ has never been called; and thus the file is not open.
Maybe I'm too much oriented towards C#'s using keyword here; but shouldn't I be able to correctly use an instance of the class even when I'm not using it with the with keyword? Right now I'm forced to use it with it - is that the typical usage in Python?
If you want the file to be opened regardless of whether or not it is used in a with, you need to open the file in your __init__ method. Since the user might forget to close the file, I would suggest that you register the closing to happen at exit:
from atexit import register
class BlaFile:
def __init__(self, file_name):
self.file_name = file_name
self.file = open(file_name, 'rb')
register(self.file.close)
def __enter__(self):
pass
def __exit__(self, exc_type, exc_val, exc_tb):
self.file.close()
def do_stuff_with_file(self):
return self.file.read(1)
You need to use open()
to open with reading you need to change to:
class BlaFile:
def __init__(self, file_name):
self.file_name = file_name
self.file = open(self.file_name, "r+b")
def close(self):
self.file.close()
def do_stuff_with_file(self):
return self.file.read(1)
That should do the trick.
I need to generate a temporary file to replace a system file with new contents.
I have the code, and it works, but I was thinking if there is some module that automatically does that, along the lines of:
with tempfile.NamedTemporaryFileFor('/etc/systemfile', delete=False) as fp:
...
This would create a temporary file with the same permissions and owner of the original file and in the same folder. Then I would write my new contents and atomically replace the original systemfile with the new one. Heck, the context manager could do the replacement for me when the file is closed!
My current code is:
with tempfile.NamedTemporaryFile(delete=False, dir='/etc') as fp:
tempfilename = fp.name
fp.write('my new contents')
orig_stat = os.stat('/etc/systemfile')
os.chown(tempfilename, orig_stat.st_uid, orig_stat.st_gid)
os.chmod(tempfilename, orig_stat.st_mode)
os.replace(tempfilename, '/etc/systemfile')
There is no such context manager in tempfile but it's not too difficult to write your own context manager:
class NamedTemporaryFileFor(object):
def __init__(self, orig_path, delete=True, dir=None):
self.orig_path = orig_path
self.named_temp_file = tempfile.NamedTemporaryFile(delete=delete, dir=dir)
def __enter__(self):
self.named_temp_file.__enter__()
return self
def write(self, *args, **kwargs):
return self.named_temp_file.write(*args, **kwargs)
def __exit__(self, exc, value, tb):
orig_stat = os.stat(self.orig_path)
os.chown(self.named_temp_file.name, orig_stat.st_uid, orig_stat.st_gid)
os.chmod(self.named_temp_file.name, orig_stat.st_mode)
self.named_temp_file.__exit__(exc, value, tb)
if __name__ == "__main__":
with NamedTemporaryFileFor(sys.argv[1], delete=False, dir="/etc") as fp:
f.write(b'my new content')
(Note that I've had to pass a byte string to the write method to make the example work)