I'm kind of on a time crunch, but this was one of my problems in my homework assignment. I am stuck, and I don't know what to do or how to proceed.
Our assignment was to open various text files and within each of the text files, we are supposed to add each word into a dictionary in which the key is the document number it came from, and the value is the word.
For example, one text file would be:
1
Hello, how are you?
I am fine and you?
Each of the text files begin with a number corresponding to it's title (for example, "document1.txt" begins with "1", "document2.txt" begins with "2", etc)
My teacher gave us this coding to help with stripping the punctuation and the lines, but I am having a hard time figuring out where to implement it.
data = re.split("[ .,:;!?\s\b]+|[\r\n]+", line)
data = filter(None, data)
I don't really understand where the filter(None, data) stuff comes into play, because all it does is return a code line of what it represents in memory.
Here's my code so far:
def invertFile(list_of_file_names):
import re
diction = {}
emplist = []
fordiction = []
for x in list_of_file_names:
afile = open(x, 'r')
with afile as f:
for line in f:
savedSort = filterText(f)
def filterText(line):
import re
word_delimiters = [' ', ',', ';', ':', '.','?','!']
data = re.split("[ .,:;!?\s\b]+|[\r\n]+", f)
key, value = data[0], data[1:]
diction[key] = value
How do I make it so each word is appended into a dictionary, where the key is the document it comes from, and the value are the words in the document? Thank you.
Related
For instance, the .txt file includes 2 lines, separated by commas:
John, George, Tom
Mark, James, Tom,
Output should be:
[George, James, John, Mark, Tom]
The following will create the list and store each item as a string.
def test(path):
filename = path
with open(filename) as f:
f = f.read()
f_list = f.split('\n')
for i in f_list:
if i == '':
f_list.remove(i)
res1 = []
for i in f_list:
res1.append(i.split(', '))
res2 = []
for i in res1:
res2 += i
res3 = [i.strip(',') for i in res2]
for i in res3:
if res3.count(i) != 1:
res3.remove(i)
res3.sort()
return res3
print(test('location/of/file.txt'))
Output:
['George', 'James', 'John', 'Mark', 'Tom']
Your file opening is fine, although the 'r' is redundant since that's the default. You claim it's not, but it is. Read the documentation.
You have not described what task is so I have no idea what's going on there. I will assume that it is correct.
Rather than populating a list and doing a membership test on every iteration - which is O(n^2) in time - can you think of a different data structure that guarantees uniqueness? Google will be your friend here. Once you discover this data structure, you will not have to perform membership checks at all. You seem to be struggling with this concept; the answer is a set.
The input data format is not rigorously defined. Separators may be commas or commas with trailing spaces, and may appear (or not) at the end of the line. Consider making an appropriate regular expression and using its splitting feature to split individual lines, though normal splitting and stripping may be easier to start.
In the following example code, I've:
ignored task since you've said that that's fine;
separated actual parsing of file content from parsing of in-memory content to demonstrate the function without a file;
used a set comprehension to store unique results of all split lines; and
used a generator to sorted that drops empty strings.
from io import StringIO
from typing import TextIO, List
def parse(f: TextIO) -> List[str]:
words = {
word.strip()
for line in f
for word in line.split(',')
}
return sorted(
word for word in words if word != ''
)
def parse_file(filename: str) -> List[str]:
with open(filename) as f:
return parse(f)
def test():
f = StringIO('John, George , Tom\nMark, James, Tom, ')
words = parse(f)
assert words == [
'George', 'James', 'John', 'Mark', 'Tom',
]
f = StringIO(' Han Solo, Boba Fet \n')
words = parse(f)
assert words == [
'Boba Fet', 'Han Solo',
]
if __name__ == '__main__':
test()
I came up with a very simple solution if anyone will need:
lines = x.read().split()
lines.sort()
new_list = []
[new_list.append(word) for word in lines if word not in new_list]
return new_list
with open("text.txt", "r") as fl:
list_ = set()
for line in fl.readlines():
line = line.strip("\n")
line = line.split(",")
[list_.add(_) for _ in line if _ != '']
print(list_)
I think that you missed a comma after Jim in the first line.
You can avoid the use of a loop by using split property :
content=file.read()
my_list=content.split(",")
to delete the occurence in your list you can transform it to set :
my_list=list(set(my_list))
then you can sort it using sorted
so the finale code :
with open("file.txt", "r") as file :
content=file.read()
my_list=content.replace("\n","").replace(" ", "").split(",")
result=sorted(list(set(my_list)))
you can add a key to your sort function
I am trying to write a script to print specific words after a particular string.
Here is the input file
Theyare "playing in the ground", with friends
Theyare "going to Paris", with family
Theyare "motivating to learn new things", by themselves
In the output I am trying to select "are" as the keyword and after "are" I want the text which is in the "" and I want to add the text before space to the "".
output should be
They playing in the ground
They going to Paris
They motivating to learn new things
I can print the rest of the line with the below code but not certain words. So far I have
with open ('input.txt', 'r') as f:
for lines in f:
a = re.search(r'\bare', f):
if a:
print (lines)
Any help would be appreciated
Use a regular expression to extract the parts of the line you want.
with open ('input.txt', 'r') as f:
for lines in f:
m = re.match(r'(.*?) are "(.*?)"')
if m:
print m.group(1) + " " + m.group(2)
The groups in m return the parts of the line that matches the patterns between ().
If your lines always look like the examples you provided you can use string manipulations:
s = 'They are "playing in the ground", with friends'
are_split = s.split('are')
# are_split = ['They ', ' "playing in the ground", with friends']
quote_split = are_split[1].split('"')
# quote_split = [' ', 'playing in the ground', ', with friends']
print(are_split[0] + quote_split[1])
# 'They playing in the ground'
The question is to:
Firstly,find the number of all words in a text file
Secondly, delete the common words like, a, an , and, to, in, at, but,... (it is allowed to write a list of these words)
Thirdly, find the number of the remaining words (unique words)
Make a list of them
the file name should be used as the parameter of the function
I have done the first part of the question
import re
file = open('text.txt', 'r', encoding = 'latin-1')
word_list = file.read().split()
for x in word_list:
print(x)
res = len(word_list)
print ('The number of words in the text:' + str(res))
def uncommonWords (file):
uncommonwords = (list(file))
for i in uncommonwords:
i += 1
print (i)
The code shows till the number of the words and nothing appears after that.
you can do it like this
# list of common words you want to remove
stop_words = set(["is", "the", "to", "in"])
# set to collect unique words
words_in_file = set()
with open("words.txt") as text_file:
for line in text_file:
for word in line.split():
words_in_file.add(word)
# remove common words from word list
unique_words = words_in_file - stop_words
print(list(unique_words))
First, you may want to get rid of punctuation : as showed in this answer, you should do :
nonPunct = re.compile('.*[A-Za-z0-9].*')
filtered = [w for w in text if nonPunct.match(w)]
then, you could do
from collections import Counter
counts = Counter(filtered)
you can then access the list of unique words with list(counts.keys()) and then you can chose to ignore the words you don't want with
[word for word in list(counts.keys()) if word not in common_words]
Hope this answers your question.
I need to search for a name in a file and in the line starting with that name, I need to replace the fourth item in the list that is separated my commas. I have began trying to program this with the following code, but I have not got it to work.
with open("SampleFile.txt", "r") as f:
newline=[]
for word in f.line():
newline.append(word.replace(str(String1), str(String2)))
with open("SampleFile.txt", "w") as f:
for line in newline :
f.writelines(line)
#this piece of code replaced every occurence of String1 with String 2
f = open("SampleFile.txt", "r")
for line in f:
if line.startswith(Name):
if line.contains(String1):
newline = line.replace(str(String1), str(String2))
#this came up with a syntax error
You could give some dummy data which would help people to answer your question. I suppose you to backup your data: You can save the edited data to a new file or you can backup the old file to a backup folder before working on the data (think about using "from shutil import copyfile" and then "copyfile(src, dst)"). Otherwise by making a mistake you could easily ruin your data without being able to easily restore them.
You can't replace the string with "newline = line.replace(str(String1), str(String2))"! Think about "strong" as your search term and a line like "Armstrong,Paul,strong,44" - if you replace "strong" with "weak" you would get "Armweak,Paul,weak,44".
I hope the following code helps you:
filename = "SampleFile.txt"
filename_new = filename.replace(".", "_new.")
search_term = "Smith"
with open(filename) as src, open(filename_new, 'w') as dst:
for line in src:
if line.startswith(search_term):
items = line.split(",")
items[4-1] = items[4-1].replace("old", "new")
line = ",".join(items)
dst.write(line)
If you work with a csv-file you should have a look at the csv module.
PS My files contain the following data (the filenames are not in the files!!!):
SampleFile.txt SampleFile_new.txt
Adams,George,m,old,34 Adams,George,m,old,34
Adams,Tracy,f,old,32 Adams,Tracy,f,old,32
Smith,John,m,old,53 Smith,John,m,new,53
Man,Emily,w,old,44 Man,Emily,w,old,44
I need to make a program in a single def that opens a text file 'grades' where first, last and grade are separated by comas. Each line is a separate student. Then it displays students and grades as well as class average. Then goes on to add another student and grade and saves it to the text file while including the old students.
I guess I just don't understand the way python goes through the text file. If i comment out 'lines' I see it prints the old_names but its as if everything is gone after. When lines is not commented out 'old_names' is not printed which makes me think the file is closed? or empty? however everything is still in the txt file as it should be.
currently i get this error.... Which I am pretty sure is telling me I'm retarded there's no information in 'line'
File "D:\Dropbox\Dropbox\1Python\Batch Processinga\grades.py", line 45, in main
first_name[i], last_name[i], grades[i] = line.split(',')
ValueError: need more than 1 value to unpack
End goal is to get it to give me the current student names and grades, average. Then add one student, save that student and grade to file. Then be able to pull the file back up with all the students including the new one and do it all over again.
I apologize for being a nub.
def main():
#Declare variables
#List of strings: first_name, last_name
first_name = []
last_name = []
#List of floats: grades
grades = []
#Float grade_avg, new_grade
grade_avg = new_grade = 0.0
#string new_student
new_student = ''
#Intro
print("Program displays information from a text file to")
print("display student first name, last name, grade and")
print("class average then allows user to enter another")
print("student.\t")
#Open file “grades.txt” for reading
infile = open("grades.txt","r")
lines = infile.readlines()
old_names = infile.read()
print(old_names)
#Write for loop for each line creating a list
for i in len(lines):
#read in line
line = infile.readline()
#Split data
first_name[i], last_name[i], grades[i] = line.split(',')
#convert grades to floats
grades[i] = float(grades[i])
print(first_name, last_name, grades)
#close the file
infile.close()
#perform calculations for average
grade_avg = float(sum(grades)/len(grades))
#display results
print("Name\t\t Grade")
print("----------------------")
for n in range(5):
print(first_name[n], last_name[n], "\t", grades[n])
print('')
print('Average Grade:\t% 0.1f'%grade_avg)
#Prompt user for input of new student and grade
new_student = input('Please enter the First and Last name of new student:\n').title()
new_grade = eval(input("Please enter {}'s grade:".format(new_student)))
#Write new student and grade to grades.txt in same format as other records
new_student = new_student.split()
new_student = str(new_student[1] + ',' + new_student[0] + ',' + str(new_grade))
outfile = open("grades.txt","w")
print(old_names, new_student ,file=outfile)
outfile.close()enter code here
File objects in Python have a "file pointer", which keeps track of what data you've already read from the file. It uses this to know where to start looking when you call read or readline or readlines. Calling readlines moves the file pointer all the way to the end of the file; subsequent read calls will return an empty string. This explains why you're getting a ValueError on the line.split(',') line. line is an empty string, so line.split(",") returns a list of length 0, but you need a list of length 3 to do the triple assignment you're attempting.
Once you get the lines list, you don't need to interact with the infile object any more. You already have all the lines; you may as well simply iterate through them directly.
#Write for loop for each line creating a list
for line in lines:
columns = line.split(",")
first_name.append(columns[0])
last_name.append(columns[1])
grades.append(float(columns[2]))
Note that I'm using append instead of listName[i] = whatever. This is necessary because Python lists will not automatically resize themselves when you try to assign to an index that doesn't exist yet; you'll just get an IndexError. append, on the other hand, will resize the list as desired.