I currently coding a discord bot which reads a local log file which keeps getting updated. Every time a new line is added and matches a certain pattern the bot (should) post a messages with that content. I tried the following or my current stand is the following code. The current problem with my code is that the file is opened all the time. Is there a way to just read the file if its gets updated since the last reading? or in other words is there a way to realize my solution with a solution which does not need to have the file opened at all times/beeing busy with reading the file consitantly?
while not BOT.is_closed():
for log in logs:
file_path = config["path_to_log"]
logfiles= []
for filename in glob.glob(os.path.join(file_path,f'_{log}*')):
logfiles.append(filename)
latest_file = max(logfiles, key=os.path.getmtime)
""" #second try doenst work if to much content gets added at the same time
with open(latest_file,"rb") as f:
if(len(messagelist)>1000):
messagelist = []
f.seek(-2, os.SEEK_END)
while f.read(1) != b'\n':
f.seek(-2, os.SEEK_CUR)
last_line = f.readline().decode()
for word in words:
if word in last_line and last_line not in messagelist:
messagelist.append(last_line)
print(last_line)
await channel.send(last_line)
else:
time.sleep(1)
"""
# first try it works but i dont think its the best solution
try:
fp = open(latest_file, 'r')
except:
end_program("Error while reading the log file")
for line in (fp.readlines() [-10:]):
new = line
for word in words:
if word in new and new not in messagelist:
messagelist.append(new)
print(new)
await channel.send(new)
else:
time.sleep(1)
###
fp.close()
You could use a generator.
import time, os
def follow(filename):
with open(filename) as f:
f.seek(0, os.SEEK_END)
while True:
line = f.readline()
if not line:
time.sleep(1)
continue
if line.startswith('a'):
yield line
The generator above will yield the lines with a newline character left at the end. You may want to slightly modify yield line to exclude the newline character, say, yield line[:-1]. line.startswith('a') is just an example to show how to yield lines that match a pattern. You will need to update this with your pattern.
In your code, you could use the generator as:
for line in follow('/path/to/file'):
# do something with line
Related
So, I'm trying to answer a coding question. It's supposed to create a random knock knock joke from an external text file, but I can't figure out how to get the joke randomized. It just prints the first joke.
The below is my code:
# Saving filepath to a variable
# makes a smoother transition to the Sandbox
filepath = "KnockKnock.txt"
# When finished copy all code after this line into the Sandbox
# Open the file as read-only
inFile = open(filepath, "r")
# Get the first line and do something with it
line = inFile.readline()
# Write your program below
print("Knock-Knock")
print("Who's there?")
print (line)
print(line + "who?")
line = inFile.readline()
print(line)
line = inFile.readline()
inFile.close()
Any idea how to get a random joke instead of it just doing the first one in the file?
Assuming your file KnockKnock.txt has the jokes in pairs, every other line, then we can read all of the jokes into a list of 2-tuples, containing the setup and punchline.
import random
...
# read in file and make a list of jokes
with open('KnockKnock.txt', 'r') as infile:
# make a list of lines from file
in_lines = infile.readlines()
# pair every line with every other line - setup and punchline
jokes = list(zip(in_lines[0::2], in_lines[1::2]))
# choose a random joke
setup, punchline = random.choice(jokes)
# print the joke
print("Knock-Knock")
print("Who's there?")
print(setup)
print(setup + " who?")
print(punchline)
I am trying to edit a specific line of a notepad file using Python 3. I can read from any part of the file and write to the end of it, however whenever I have tried editing a specific line, I am given the error message 'TypeError: 'int' object is not subscriptable'. Does anybody know how I could fix this?
#(This was my first attempt)
f = open('NotepadTester.txt', 'w')
Edit = input('Enter corrected data')
Line = int(input('Which line do you want to edit?'))
f.write(Edit)[Line-1]
f.close()
main()
#(This was my second attempt)
f = open('NotepadTester.txt', 'w')
Line = int(input('Which line do you want to edit?'))
Edit = input('Enter corrected data')
f[Line-1] = (Edit)
main()
you can't directly 'edit' a line in a text file as far as I know. what you could do is read the source file src to a variable data line-by-line, edit the respective line and write the edited variable to another file (or overwrite the input file) dst.
EX:
# load information
with open(src, 'r') as fobj:
data = fobj.readlines() # list with one element for each text file line
# replace line with some new info at index ix
data[ix] = 'some new info\n'
# write updated information
with open(dst, 'w') as fobj:
fobj.writelines(data)
...or nice and short (thanks to Aivar Paalberg for the suggestion), overwriting the input file (using open with r+):
with open(src, 'r+') as fobj:
data = fobj.readlines()
data[ix] = 'some new info\n'
fobj.seek(0) # reset file pointer...
fobj.writelines(data)
You should probably load all the lines into memory first, modify it from there, and then write the whole thing to a file.
f = open('NotepadTester.txt', 'r')
lines = f.readlines()
f.close()
Which_Line = int(input('Which line do you want to edit? '))
Edit = input('Enter corrected data: ')
f = open("NotepadTester.txt",'w')
for i,line in enumerate(lines):
if i == Which_Line:
f.writelines(str(Edit)+"\n")
else:
f.writelines(line)
f.close()
I've been learning Python and I wanted to write a script to count the number of characters in a text and calculate their relative frequencies. But first, I wanted to know the length of the file. My intention is that, while the script goes from line to line counting all the characters, it would print the current line and the total number of lines, so I could know how much it is going to take.
I executed a simple for loop to count the number of lines, and then another for loop to count the characters and put them in a dictionary. However, when I run the script with the first for loop, it stops early. It doesn't even go into the second for loop as far as I know. If I remove this loop, the rest of the code goes on fine. What is causing this?
Excuse my code. It's rudimentary, but I'm proud of it.
My code:
import string
fname = input ('Enter a file name: ')
try:
fhand = open(fname)
except:
print ('Cannot open file.')
quit()
#Problematic bit. If this part is present, the script ends abruptly.
#filelength = 0
#for lines in fhand:
# filelength = filelength + 1
counts = dict()
currentline = 1
for line in fhand:
if len(line) == 0: continue
line = line.translate(str.maketrans('','',string.punctuation))
line = line.translate(str.maketrans('','',string.digits))
line = line.translate(str.maketrans('','',string.whitespace))
line = line.translate(str.maketrans('','',""" '"’‘“” """))
line = line.lower()
index = 0
while index < len(line):
if line[index] not in counts:
counts[line[index]] = 1
else:
counts[line[index]] += 1
index += 1
print('Currently at line: ', currentline, 'of', filelength)
currentline += 1
listtosort = list()
totalcount = 0
for (char, number) in list(counts.items()):
listtosort.append((number,char))
totalcount = totalcount + number
listtosort.sort(reverse=True)
for (number, char) in listtosort:
frequency = number/totalcount*100
print ('Character: %s, count: %d, Frequency: %g' % (char, number, frequency))
It looks fine the way you are doing it, however to simulate your problem, I downloaded and saved a Guttenberg text book. It's a unicode issue. Two ways to resolve it. Open it as a binary file or add the encoding. As it's text, I'd go the utf-8 option.
I'd also suggest you code it differently, below is the basic structure that closes the file after opening it.
filename = "GutenbergBook.txt"
try:
#fhand = open(filename, 'rb')
#open read only and utf-8 encoding
fhand = open(filename, 'r', encoding = 'utf-8')
except IOError:
print("couldn't find the file")
else:
try:
for line in fhand:
#put your code here
print(line)
except:
print("Error reading the file")
finally:
fhand.close()
For the op, this is a specific occasion. However, for visitors, if your code below the for state does not execute, it is not a python built-in issue, most likely to be: an exception error handling in parent caller.
Your iteration is inside a function, which is called inside a try except block of caller, then if any error occur during the loop, it will get escaped.
This issue can be hard to find, especially when you dealing with intricate architecture.
I have approximately 160,000 text files in a directory. My first objective is to create a list of files that contain at least one item from a list of about 50 keywords. My current code is
import os
ngwrds= [list of words]
for filename in os.listdir(os.getcwd()):
with open(filename, 'r') as searchfile:
for line in searchfile:
if any(x in line for x in ngwrds):
with open("keyword.txt", 'a') as out:
out.write(filename + '\n')
Which works but sends out duplicate filenames. Ideally what I would like is for the loop to stop once it hits the first keyword, write the file name to 'keyword.txt', and move on to the next file in the directory. Any thoughts on how to do this?
A more in depth answer to #strubbly's comment, you would simply add a break in the 2nd for loop
with open(filename, 'r') as searchfile:
for line in searchfile:
if any(x in line for x in ngwrds):
with open("keyword.txt", 'a') as out:
out.write(filename + '\n')
break
What does the break do? from the python3 docs:
The break statement, like in C, breaks out of the smallest enclosing for or while loop.
for more information on break go to the control flow documentation :https://docs.python.org/3/tutorial/controlflow.html
Can you explain what is going on in this code? I don't seem to understand
how you can open the file and read it line by line instead of all of the sentences at the same time in a for loop. Thanks
Let's say I have these sentences in a document file:
cat:dog:mice
cat1:dog1:mice1
cat2:dog2:mice2
cat3:dog3:mice3
Here is the code:
from sys import argv
filename = input("Please enter the name of a file: ")
f = open(filename,'r')
d1ct = dict()
print("Number of times each animal visited each station:")
print("Animal Id Station 1 Station 2")
for line in f:
if '\n' == line[-1]:
line = line[:-1]
(AnimalId, Timestamp, StationId,) = line.split(':')
key = (AnimalId,StationId,)
if key not in d1ct:
d1ct[key] = 0
d1ct[key] += 1
The magic is at:
for line in f:
if '\n' == line[-1]:
line = line[:-1]
Python file objects are special in that they can be iterated over in a for loop. On each iteration, it retrieves the next line of the file. Because it includes the last character in the line, which could be a newline, it's often useful to check and remove the last character.
As Moshe wrote, open file objects can be iterated. Only, they are not of the file type in Python 3.x (as they were in Python 2.x). If the file object is opened in text mode, then the unit of iteration is one text line including the \n.
You can use line = line.rstrip() to remove the \n plus the trailing withespaces.
If you want to read the content of the file at once (into a multiline string), you can use content = f.read().
There is a minor bug in the code. The open file should always be closed. I means to use f.close() after the for loop. Or you can wrap the open to the newer with construct that will close the file for you -- I suggest to get used to the later approach.