Related
A sample script of Game of Thrones was taken from the page and stored in the file conv.txt in the dataset provided. Make sure to complete all the tasks and follow the submission guidelines.
Tasks:
Find out the number of unique dialogue speakers in the sample conversation?
Create a new text file by the name of the dialogue speaker and store the unique words spoken by that character in the respective text file. Make sure there is only one word
every line.
WILL: I’ve never seen wildlings do a thing like this. I’ve never seen a thing like this, not ever in my life.
WAYMAR ROYCE: How close did you get?
WILL: Close as any man would.
GARED: We should head back to the wall.
ROYCE: Do the dead frighten you?
GARED: Our orders were to track the wildlings. We tracked them. They won’t trouble us no more.
ROYCE: You don’t think he’ll ask us how they died? Get back on your horse.
WILL: Whatever did it to them could do it to us. They even killed the children.
ROYCE: It’s a good thing we’re not children. You want to run away south, run away. Of course, they will behead you as a deserter … If I don’t catch you first. Get back on your horse. I won’t say it again.
ROYCE: Your dead men seem to have moved camp.
WILL: They were here.
GARED: See where they went.
ROYCE: What is it?
JON: Go on. Father’s watching.
JON: And your mother.
SEPTA MORDANE: Fine work, as always. Well done.
SANSA: Thank you.
SEPTA MORDANE: I love the detail that you’ve managed to get in this corners.
NED: And which one of you was a marksman at ten? Keep practicing, Bran. Go on.
JON: Don’t think too much, Bran.
ROBB: Relax your bow arm.
CASSEL: Lord Stark. My lady. A guardsman just rode in from the hills. They’ve captured a deserter from the Night’s Watch.
NED: Get the lads to saddle their horses.
CATELYN: Do you have to?
NED: He swore an oath, Cat.
CASSEL: The law is law, my lady.
NED: Tell Bran he’s coming, too.
CATELYN: Ned. Ten is too young to see such things.
NED: He won’t be a boy forever. And winter is coming.
ROBB: Lad, go run back and get the rest.
NED: King of the Andals and the First Men …
JON: Father will know if you do.
NED: Lord of the Seven Kingdoms and protector of the realm, I, Eddard of the House Stark, Lord of Winterfell and Warden of the North, sentence you to die.
JON: You did well.
NED: You understand why I did it?
BRAN: Jon said he was a deserter.
NED: But do you understand why I had to kill him?
BRAN: Our way is the old way?
NED: The man who passes the sentence should swing the sword.
BRAN: Is it true he saw the White Walkers?
NED: The White Walkers have been gone for thousands of years.
BRAN: So he was lying?
NED: A madman sees what he sees.
JON: What is it?
THEON: Mountain lion?
NED: There are no mountain lions in these woods.
THEON: It’s a freak.
NED: It’s a direwolf.
NED: Tough old beast.
ROBB: There are no direwolves south of the Wall.
JON: Now there are five.
JON: You want to hold it?
BRAN: Where will they go? Their mother’s dead.
CASSEL: They don’t belong down here.
NED: Better a quick death. They won’t last without their mother.
THEON: Right. Give it here.
BRAN: NO!
ROBB: Put away your blade.
THEON: I take orders from your father, not you.
BRAN: Please, father!
NED: I’m sorry, Bran.
JON: Lord Stark? There are five pups. One for each of the Stark children. The direwolf is the sigil of your House. They were meant to have them.
NED: You will train them yourselves. You will feed them yourselves. And if they die, you will bury them yourselves.
BRAN: What about you?
JON: I’m not a Stark. Get on.
ROBB: What is it?
CERSEI: What if Jon Arryn told someone?
JAIME: But who would he tell?
CERSEI: My husband.
JAIME: If he told the king, both our heads would be skewered on the city gates by now. Whatever Jon Arryn knew or didn’t know, it died with him. And Robert will choose a new Hand of the king, someone to do his job while he’s off fucking boars and hunting whores. Or is it the other way around? And life will go on.
CERSEI: You should be the Hand of the king.
JAIME: That’s an honor I can do without. Their days are too long, their lives are too short.
CATELYN: All these years and I still feel like an outsider when I come here.
NED: You have five northern children. You’re not an outsider.
CATELYN: I wonder if the old gods agree.
NED: It’s your gods with all the rules.
CATELYN: I am so sorry, my love.
NED: Tell me.
ROBB: I hear the prince is a right royal prick.
THEON: Think of all those southern girls he gets to stab with his right royal prick.
ROBB: Go on, Tommy, shave him good. He’s never met a girl he likes better than his own hair.
CATELYN: Gods, but they grow fast.
BRAN: I saw the king! He’s got hundreds of people!
CATELYN: How many times have I told you No climbing!
BRAN: But he’s coming right now! Down our road!
CATELYN: I want you to promise me. No more climbing.
NED: Your Grace.
ROBERT: You’ve got fat.
ROBERT: Cat!
CATELYN: Your Grace.
ROBERT: Nine years. Why haven’t I seen you? Where the hell have you been?
NED: Guarding the North for you, Your Grace. Winterfell is yours.
ARYA: Where’s the Imp?
SANSA: Will you shut up?
ROBERT: Who have we here? You must be Robb.
ARYA: Arya.
ROBERT: Ooh. Show us your muscles. You’ll be a soldier.
ARYA: That’s Jaime Lannister. The queen’s twin brother.
SANSA: Would you please shut up.
NED: My queen.
CATELYN: My queen.
ROBERT: Take me to your crypt. I want to pay my respects.
CERSEI: We’ve been riding for a month, my love. Surely the dead can wait.
ROBERT: Ned.
ARYA: Where’s the Imp?
CERSEI: Where is our brother? Go find the little beast.
NED: Tell me about Jon Arryn.
ROBERT: One minute he was fine, and then … Burned right through him, whatever it was. I loved that man.
NED: We both did.
ROBERT: He never had to teach you much, but me … You remember me at 16? All I wanted to do was crack skulls and fuck girls. He showed me what was what.
NED: Aye.
the name of the dialogue speaker and store the unique words spoken by that character in the respective text file.
Before actually getting into the problems, here is how we read the file:
import string
filename = "script.txt"
with open(filename) as f:
dialogue_list = [x for x in f if len(x.strip()) > 0] # ignore empty lines
This actually consists of two problems:
1. finding the names of the speakers
For this, we first extract the names using split(':') and then take its first element [0], then we use set to take the unique names:
# take the speaker names
speaker_list = [x.split(':')[0] for x in dialogue_list]
# unique speakers
speaker_set = set(speaker_list)
print("number of unique speakers: {}".format(len(speaker_set)))
2. finding the unique word of each speaker
We use nltk.tokenize.word_tokenize to extract the tokens, ignore the punctuations, and then we use set to take the unique elements.
from nltk.tokenize import word_tokenize
for speaker_name in speaker_set:
this_speaker_sentences = [x for x in dialogue_list if x.split(':')[0]==speaker_name]
this_speaker_tokenized_sentences = [word_tokenize(x) for x in this_speaker_sentences]
this_speaker_words = [
word for l in this_speaker_tokenized_sentences for word in l # flatten the list
if word not in string.punctuation # ignore punctuations
]
this_speaker_words_set = set(this_speaker_words)
with open("{}.txt".format(speaker_name), 'w') as f:
f.writelines([x+'\n' for x in this_speaker_words_set])
I am trying BerTopic on a cluster of sentences. I have actually employed Agglomerative clustering using Bert sentence embeddings, the result has many clusters one of them is this
docs=["PARIS:France’s trade unions called for mass protests and strikes over pension reform that have brought much of the country to a halt to carry on next week, piling more pressure on President Emmanuel Macron.Commuters faced severe disruption getting to work on Friday, hospitals have been left understaffed and Paris City Hall said dozens of schools in the capital would stay closed, as unions dug in over Macron’s plans to streamline one of the developed world’s most generous pension systems.Transport workers went on strike on Thursday and took to the streets – joined by teachers, doctors, police, firemen and civil servants. Smoke and tear gas swirled through parts of Paris and Nantes as protests turned violent.Union leaders said public workers should maintain their industrial action until Tuesday when they urged members to flood the streets once again.“Unions will meet on Tuesday evening to decide on our next actions if by then Macron and (Prime Minister) Edouard Philippe has not reversed course and opened negotiations,” Catherine Perret of the hard-left CGT union told reporters.The strike pits Macron, a 41-year-old former investment banker who took office in 2017 on a promise of opening up France’s highly regulated economy, against powerful unions who say he is set on dismantling worker protections.“We’re going to protest for a week at least, and at the end of that week it’s the government that’s going to back down,” said 50-year-old Paris transport employee Patrick Dos Santos.The outcome depends on who blinks first – the unions who risk losing public support if the disruption goes on for too long, or the president whose two-and-a-half years in office have been rocked by waves of social unrest.Macron’s pension tsar Jean-Paul Delevoye is due to hold talks with the unions on Monday before the prime minister presents the broad outlines of the proposal to the public mid-week.Education Minister Jean-Michel Blanquer said far-reaching reform was needed to put the generous pension system on a sustainable footing. Fewer teachers went on strike on Friday, education ministry data showed.“It would be much easier for us to do nothing,” Blanquer told BFM TV. “We could see through this five-year term without enacting deep reform. But if every presidency reasons in this way, our children will not have an acceptable pension system.”Police had used tear gas in central Paris on Thursday afternoon when hooded protesters on the fringes of the trade unions’ march threw fireworks at officers, ransacked bus stops, and set fire to rubbish bins.More than 800,000 people rallied in protests countrywide on Thursday. Union leaders put the numbers higher.“There’s a noise in the streets, I hope the windows of the Elysee are open,” said Philippe Martinez, secretary-general of the CGT union, referring to the president’s office.Macron wants to simplify the unwieldy pension system, which comprises more than 40 different plans. Rail workers and mariners can, for instance, retire up to a decade earlier than the average worker.The president says the system is unfair and too costly and that the French will have to work longer, though he appears reluctant to simply raise the retirement age of 62.One alternative is to curb benefits for those who stop working before 64 and give a benefits boost to those who leave later.",
"The French -- and particularly Parisians -- are face to face with what may be the largest strike in the country's history.On the heals of the Yellow Vest protests, employees of various sectors are preparing to go on indefinite strike beginning Thursday to protest pension reforms by the government of French President Emmanuel Macron.The walkout was sealed when the government announced its determination to implement pension reform despite pushback.According to France’s National Institute of Statistics and Economic Studies, Macron has further fueled the sense of anger and rebellion among French people against their presidents, with his economic policies that have given the wealthy a greater share of national income since his inauguration on May 17, 2017.He has been facing the biggest crisis since the yellow vest protests.The reform would lift the privileges granted to civil servants and gradually increasing the retirement age from 62 to 64. It is expected to adversely affect many sectors.Long list of strikersAmong the strikers will be employees of national carrier Air France, state-owned Parisian public transport operator RATP, electricity company EDF that is largely owned by the government, state-owned national railway firm SNCF, and automobile manufacturer Renault.Police, healthcare professionals, teachers, lawyers, taxi and freight drivers, postal workers, farmers, civil servants, refinery workers and students will also participate.Over half of all schools across the country will be suspended, while nearly all commuter trains and buses will halt and or work in intermittently. Air France will cancel 30% of its flights.The Yellow Vest protests started Nov. 17, 2018 in reaction to rising fuel costs and economic injustice, later spiraling into deadly anti-government riots.Protesters used yellow vests, part of the standard safety kit in French cars, to make their members more easily visible.The demonstrations left 11 dead and more than 4,000 injured including protesters and the police, according to government figures.Activists claim that 24 protesters were blinded in one eye and that five lost one of their hands.At least some 8,400 people have been arrested since the beginning of the Yellow Vest protests, and 2,000 were remanded into custody.A total of 17 protestors were arrested in Toulouse and five people -- two police and three civilians -- were injured.",
"PARIS-The Eiffel Tower shut down, France’s vaunted high-speed trains stood still and several thousand people protested in Paris as unions launched open-ended, nationwide strikes Thursday over the government’s plan to overhaul the retirement system.Paris authorities barricaded the presidential palace and deployed 6,000 police as activists - many in yellow vests representing France’s year-old movement for economic justice - gathered in the capital in a mass outpouring of anger at President Emmanuel Macron and his centerpiece reform.Unions and their supporters fear that the changes to how and when workers can retire will threaten the hard-fought French way of life. Macron himself remained “calm and determined” to push it through, according to a top presidential official.The Louvre Museum warned of strike disruptions, and subway stations across Paris shut their gates. Many visitors - including the U.S. energy secretary - canceled plans to travel to one of the world’s most-visited countries amid the strike. Unprepared tourists discovered historic train stations standing empty Thursday, with about nine out of 10 of high-speed TGV trains canceled. Signs at Paris’ Orly Airport showed “canceled” notices, as the civil aviation authority announced 20% of flights were grounded.Some travelers showed support for the striking workers, but others complained about being embroiled in someone else’s fight. “I had no idea about the strike happening, and I was waiting for two hours in the airport for the train to arrive and it didn’t arrive,” said vacationer Ian Crossen, from New York. “I feel a little bit frustrated. And I’ve spent a lot of money. I’ve spent money I didn’t need to, apparently.”Vladimir Madeira, a Chilean tourist vacationing in Paris, said the strike has been “a nightmare.” He hadn’t heard about the protest until he arrived, and transport disruptions foiled his plans to travel directly to Zurich.Beneath the closed Eiffel Tower, tourists from Thailand, Canada and Spain echoed those sentiments. Bracing for possible violence along the route of the Paris march, police ordered all businesses, cafes and restaurants in the area to close. Authorities banned protests in the more sensitive neighborhoods around the Champs-Elysees avenue, presidential palace, parliament and Notre Dame Cathedral.Police carried out security checks of more than 6,000 people arriving for the protest and detained 65 even before it started. Embassies warned tourists to avoid the protest area. The mood was impassioned in the crowd massed on Boulevard Magenta in eastern Paris.Health workers showed up to decry conditions in hospitals. Students pointed to recent student suicides and demanded government action. Environmentalists emphasized that climate justice and social justice are one and the same. And young and old roundly condemned the new retirement plan, which they fear would take money out of their pockets and reduce the period of repose the French expect in the last decades of their lives.Eric Mettling, who joined the yellow vests at the start of their movement, said the general strike had brought together social movements across France in a manner unprecedented in recent memory to denounce “the social crisis.”Skirmishes broke out between police firing tear gas and protesters throwing flares at a protest in the western French city of Nantes, and thousands of red-vested union activists marched through cities from Marseille on the Mediterranean to Lille in the north.Lacking public transport, commuters used shared bikes or electric scooters despite near-freezing temperatures. Many workers in the Paris region worked from home or took a day off to stay with their children, since 78% of teachers in the capital were on strike.The big question is how long the strike will last. Transport Minister Elisabeth Borne said she expects the travel troubles to be just as bad Friday, and unions said they’ll maintain the Paris subway system strike at least through Monday. Public sector workers fear Macron’s reform will force them to work longer and shrink their pensions. Some private sector workers share their worries, while others welcome the reform.Joseph Kakou, who works an overnight security shift in western Paris, walked an hour to get home to the eastern side of town Thursday morning. “It doesn’t please us to walk. It doesn’t please us to have to strike,” Kakou told The Associated Press. “But we are obliged to, because we can’t work until 90 years old.”To Macron, the retirement reform is central to his plan to transform France so it can compete globally in the 21st century. The government argues France’s 42 retirement systems need streamlining. While Macron respects the right to strike, he “is convinced that the reform is needed, he is committed, that’s the project he presented the French in 2017” during his election campaign, the presidential official said. The official was not authorized to be publicly named.After extensive meetings with workers, the high commissioner for pensions is expected to detail reform proposals next week, and the prime minister will release the government’s plan days after that.",
"Protesters mobilized across France on Thursday in a nationwide strike challenging President Emmanuel Macron’s controversial pension reform plans.The Interior Ministry said 806,000 people took part, while labor unions put the number at nearly 1.5 million.Some 250,000 people took part in the protests in Paris, where police used smoke bombs to disperse the crowd.The unlimited strike impacted all public transport systems in the country, according to local media reports.A total of 90 people have been arrested so far in Paris, police said.Some train, subway and bus services were canceled and many schools were closed while the law and order situation led to the cancelation of 20% of flights to the country.In a tweet, the Paris Police Department said it had conducted 6,476 checks. Labor unions said the strike will continue until Monday.The Gare du Nord, a station of the SNCF railway network in Paris, was almost empty in the morning, according to broadcaster France 24.Protesters, however, made their way to the Gare du Nord in the afternoon to attend the main march to Place de la Nation square.They included police, healthcare professionals, teachers, lawyers, taxi and freight drivers, postal workers, farmers, civil servants, refinery workers and students, according to the Le Monde daily.The walkout came after the government announced its determination to implement pension reform despite a nationwide outcry.According to France’s National Institute of Statistics and Economic Studies, Macron has further fueled the sense of anger and rebellion among French people against their president with his economic policies that have given wealthy people a greater share of national income since his inauguration on May 17, 2017.He has been facing the biggest crisis since the beginning of the Yellow Vest protests in October last year.Proposed reformFrance currently has 42 different pension programs for different sectors, but the government proposed to unify them into one pension scheme.France’s current program is based on the principle of solidarity between generations under which the working population finances the pensioners of that year.But due to the aging population, fewer people are paying into the current system.To fix this, the government introduced a point-based system that would compensate workers with pension points for every day they work or every euro they contribute.The reform would lift the privileges granted to civil servants and gradually increase the retirement age from 62 to 64, a move expected to adversely affect many sectors.Workers will get a full pension if they retire at the age of 64. If they retired before, they would lose 5% of their pensions for every year they retire early.They would also gain a 5% increase in their pensions for every year if they retire after the age of 64.The demonstrations and strikes have been supported by numerous labor and police unions as well as the Yellow Vests.Macron paused his overseas visits for a while to focus on a solution to the problems caused by the strikes and demonstrations.",
"Paris-A strike over planned pension reforms that paralysed France on Thursday has entered its second day.Several unions, including rail and metro workers, voted to extend the strike action, meaning another day of major disruptions to key services.It comes after more than 800,000 people protested on Thursday, with violent clashes reported in a number of cities.Workers are angry about planned pension reforms that would see them retiring later or facing reduced payouts.France currently has 42 different pension schemes across its private and public sectors, with variations in retirement age and benefits. President Emmanuel Macron says his plans for a universal points-based system would be fairer, but many disagree.Rail workers voted to extend their strike through Friday, while unions at the Parisian bus and metro operator said their walkout would continue until at least Monday.Numerous rush-hour trains into Paris were cancelled on Friday and 10 out of 16 metro lines were closed, while others ran limited services, Reuters news agency reports.Traffic jams of more than 350km (217 miles) were reported on major roads in and around the capital.A number of flights have also been disrupted, while many schools are expected to remain shuttered and hospitals understaffed. Protesters sang songs against President Macron in ParisMr Macron’s government has reportedly made plans to deal with the strike action at the weekend.Some trade union leaders have vowed to strike until Mr Macron abandons his campaign promise to overhaul the retirement system.“We’re going to protest for a week at least, and at the end of that week it’s the government that’s going to back down,” 50-year-old Paris transport employee Patrick Dos Santos told Reuters.What happened on Thursday?French police gave the figure of 800,000 people taking to the streets across the country, including 65,000 in Paris. Union leaders put the numbers higher, with the CGT union saying 1.5m people turned out across France.The disruption meant popular tourist sites in Paris, including the Eiffel Tower, were closed for the day and usually busy transport hubs like the Gare du Nord were unusually quiet.",
"Paris (AFP): France was on Saturday expecting its most serious nationwide strike in years to paralyse the country over the weekend, with unions warning the turmoil would last well into next week.",
"PARIS: The French government on Friday expressed determination to plough ahead with far-reaching pension reforms in the face of the biggest strikes in years, which have brought public transport in much of the country to a standstill.The strikes, which began on Thursday, have seen most high-speed trains cancelled, flights affected and most of the Paris metro shut down in a major challenge to the ambitious reform agenda of President Emmanuel Macron.The turmoil is expected to continue over the weekend and through until at least Tuesday when unions have called more nationwide protests to follow mass rallies on Thursday that brought over 800,000 people onto the streets.With Macron not yet speaking publicly about the strikes and seeking for now to rise above the fray, Prime Minister Edouard Philippe insisted that the government would not abandon a plan which would require the French “to work a bit longer.” He pledged to work with trade unions to introduce a single “fairer”, points-based pension scheme for all, scrapping the 42 more advantageous plans currently enjoyed by train drivers, soldiers and a host of other workers in the process.The centre-right premier added that the government was “very determined” to implement the reform, adding he did not believe the French would always accept a situation where some retire earlier, and with more money than others doing comparable jobs.But he emphasised that the changes, which he said would be unveiled on Wednesday, would be introduced “progressively, without harshness”. “My logic will never be one of confrontation,” he said.Dozens of trains, metros and flights were cancelled, many schools were again closed or offering only daycare, and four of the country’s eight oil refineries remained blocked on Friday.Rail operator SNCF has already halted ticket sales through the weekend, with 90 percent of high-speed TGV trains again cancelled on Friday and little improvement expected over the weekend.Half of the Eurostar trains between Paris and London were dropped, and just two of three Thalys trains serving Paris, Brussels and Amsterdam were running.“I was supposed to take a train to Metz (northeast France), I reserved my ticket three days ago but it’s been cancelled and I’ve gotten no information,” Rachel Pallamidessi said at a deserted station in the city of Strasbourg.Several airlines cancelled flights as air traffic controllers walked off the job, with Air France cancelling 30 percent of domestic flights and 10 percent of nearby international routes.In Paris, nine of the capital’s 16 metro lines were shut while many others were running only during rush hours, prompting commuters to turn to bicycles, electric scooters and other alternatives or to work from home.It remains to be seen if the protests will match the magnitude of the 1995 strikes against pension overhauls when France was paralysed for three weeks from November to December, ultimately forcing the government to back down. The walkout is the latest test of Macron’s mettle after months of protests from teachers, hospital workers, police and firefighters, capping a year of social unrest triggered by the “yellow vest” protest movement.Unions say Macron’s proposal for a single pension system would force millions of people in both the public and private sectors to work well beyond the official retirement age of 62.At least 800,000 took part in rallies around the country on Thursday, according to the interior ministry, one of the biggest demonstrations of union strength in nearly a decade.Another day of strikes and rallies has been called for Tuesday, a day after union leaders are to meet again with government officials over the pension reform.“There were lots of people on strike, now we need even more if we want to influence these decisions,” Philippe Martinez of the hard-line CGT union told LCI television.While most of the rallies were peaceful, police fired tear gas to disperse dozens of black-clad protesters smashing windows and throwing stones during the Paris march, with one construction trailer set on fire.Several dozens of people were arrested, and three journalists were injured after reportedly being hit by tear gas or stun grenades, including a Turkish journalist who was struck in the face.Published in Dawn, December 7th, 2019Copyright © 2019, DawnScribe Publishing Platform",
]
The code is as follows.
from bertopic import BERTopic
# docs=[i for i in all_text if type(i)==str]
# docs=docs.T
topic_model = BERTopic()
topics, probs = topic_model.fit_transform(docs)
Any help is much appreciated
Thanks
So, I'm currently doing a Web Dev bootcamp, I'm about halfway done and I'm trying to come up with a personal project to challenge myself and also have something useful that'll stand out. Anyways,
I have this odd hobby of monitoring everyone who's been arrested per week in Hillsborough County FL. Sometimes if I see an old friend who has been arrested, I immediately go to the clerk of court website for Hillsborough county and look at their court case and even get their CRA(Criminal Report Affidavit) aka Police Report redacted so I can see what they actually did. I'm really big into all of this, call it weird but I got sick of having to look up the arrest on florida.arrests.org or from
webapps.hcso.tampa.fl.us/ArrestInquiry and then having to go immediately to hover.hillsclerk.com to search court records and all court info regarding said person. I thought it'd be cool to construct a website where on the index page it's just 2 textboxes and 1 submit button. 1 text box being for First Name and the second being for Last Name. Once submitted, the First and Last Name are compared to a database and all arrests for Joe Macgyver come up and Joe Macgyver's court cases(open and closed) which include court dates, evidence submitted by the state, basically everything that happened in court on whatever day. So all arrests for Joe Macgyver in Hillsborough county come up and on the same page all court cases. I figured it'd be cool and it'd save me some time but I don't even know where to begin.
It looks like florida.arrests.org is gonna be more practical than webapps.hcso.tampa.fl.us/ArrestInquiry because they just recently made some changes and the sheriff's office is only allowing arrests within a 3 month time period to remain public record that can be accessed for free online. They also have a captcha that needs to be entered for every search done However, Florida.arrests logs every arrest made and doesn't delete anything and has records going back to the late 90s. I can't seem to find any way to get an API key for florida.arrests.org so I know its possible to do what i wanna do, I just dont know where to start and how to go about it. A friend suggested web scraping, I did some research and got a little confused. Can anyone possibly help point me in the right direction? I figured if I kept the searches down to only hillsborough county, it should make my idea a little easier.
Thank you,
SparklingCoffee
so i'm trying to clean up a file for my assignment by converting to lowercases and removing punctuations.
content:
I'm a fan of soft serve ice cream and Guptill's Coney Express has delicious ice cream with many flavors. I've tried Kurver Kreme in Colonie, Tastee Freeze in Delmar and Country Drive Inn in Clifton Park, but I think that this place has the best soft serve ice cream. The portions are generous and the taste is very rich. For example, the brownie sundae is decadently delicious but likely too much for one person. They also have cupcake sundaes which I am looking to try soon!
The nurses here were very attentive and wonderful. I was able to have the same surgical nurse that I had for another procedure a few years ago. The anesthesiologist listened to my concerns and acting skillfully and compassionately. I felt well taken care of there.
Pretty great! Okay, so this place is obviously not Vegan since they have a bunch of cheese and egg offerings, BUT I see that they do offer plenty of vegan alternatives.\n\nI was sort of skeptical being here because the prices were pretty hefty, I felt. I guess it looked like a fast-food/diner joint, but it charged a little more. \n\nAnyway, their homemade hot sauce is AMAZING. I got the eggs benedict for dinner and J got an omelet. Both were really good. I do love their homefries.. but the next time I come here, I want onion rings or fries. Those onion rings looked amazing.\n\nLastly, the food came relatively quickly.\n\nNot a fan of the service. They tried to seat us at this edge facing the stoves, without asking, so I asked for a booth. Then at the booth, the server didn't refill waters very well but didn't feel bad emphasizing over and over whether or not we wanted their $5-7 desserts. Honestly, a slice of pie for $6.50? Veggie Galaxy, you are t r i p p i n !\n\nBut great food! (especially breaky!)
The Tale of the 4-Starred, Up Close and Personal Bar.\n\nThe Back Abbey is a bar that is located in the charming West Village part of Claremont. The area resembles the old shopping strip-meets-new shopping strip of San Luis Obispo. What they did -- or so I've been told -- is that they've managed to preserve the old strip, leaving it looking the way it presumably did in the 70s. Rhino Records is located in the older part. The Back Abbey in the new part. Both distinct areas of their aesthetic and consumer pull. \n\nThe Back Abbey specializes in Belgian draft beer. They carry some German and English beer, as well.\n\nWow, the selection of beer. Let me tell you! A lof of them I have never heard of. I tasted the Augustijn (Belgian) and the Hofbrau Original (German). Both were good, but I prefer the Hofbrau.They come in different sized glasses, with the Hofbrau coming in a long glass. The Augustijn was $9, and the Hofbrau $6. The Hofbrau came in the larger glass, so the Augustijn must be considered some premium brew for it to be in a smaller glass and come knocking down doors at a mighty $9. \n\nI liked the overall aesthetics of the place. There is an up-close-and-personal feel to the bar. Along with the standard bar and barstools set-up, there's a long table that goes from one end of the bar to the other. People sit across from each other and, since the tables are not wide, you're not far at all from the person sitting in front of you. Like I wrote earlier, up close and personal is what this bar has going for it.\n\nThey have a food menu, as well. Above their draft selection of beer, they have a chalkboard placed high up with a list of what they serve. For those veggies among us, they even make a veggie burger.\n\nLastly, the patrons all seem to know each other. Maybe it's the small town mentality. But it's not exactly a backwards, hick town. It's very much modern in certain ways, and people just love to talk. Couple that with brew, and there's a lot of talk going on.\n\nI had a discussion with someone at the bar, who bartends at a bar not too far from The Back Abbey, and he told me that the Augustijn is the oldest beer recipe known to man. (Ah, ok. Old is vintage. Vintage is expensive. Thus, Augustijn is expensive. And two + two = four. I got this down, partna'.) Again, can we give it up for \
As a vegan, I try to support places that are not wholly vegetarian, but that made the effort to put a vegetarian section on their menu, to show them that it's worth keeping. Thai Singha House does indeed have a section of their menu that reads vegetarian, but, unfortunately, their definition of vegetarian is different from mine.\n\nAll of the vegetarian curries, I was fortunately informed upon asking, do indeed contain fish sauce, which is an ingredient in the curry itself, and cannot be omitted.\n\nMy server suggested the pad thai as a fish-free vegetarian alternative, which I ordered, holding the egg. When it arrived, there appeared to be tofu in there and no egg, which was a good sign. But after a few blind, trusting bites I quickly realized that there was also chicken or pork mixed in as well. My server graciously took it back and honored my request for a house salad, of which I wasn't charged.\n\nI don't blame Thai Singha House for not understanding the definition of vegetarian (sadly, I know a few \
On a recent visit to SLO, I got taken out to the Natural Cafe, which bears a distinct resemblance to some of the places I remember from growing up in Berkeley. Very natural woodsy, with just enough sprouts on the sandwich to add a little crunch. It's casual, with counter service only.\n\nMy turkey burger was not too dry, but the guacamole and special sauce helped it along in the moistness department. I didn't walk away from the meal feeling all gross and over full like you would with a normal burger and fries. Yeah for whole grains and veggies!
Great place! \n\nI have to say the menu and the outdoor seating keep us coming back. The food is good -- had breakfast both times but some friends had lunch items. Definitely a great selection. We've been at off-peak times so no waiting and better service. \n\nAll in all, it's no DZ Akins but it's definitely worth trying!
Went to Rebecca's Cafe today during my lunch break today. I have to say, my sandwich was delicious! I had a Chicken Caprese panini (grilled chicken, tomato, mozzarella cheese, spinach, and balsamic vinaigrette). The staff was friendly and the service was quick, however the price is a little high for my taste ($7 or so for the sandwich only). Overall, I would definitely go back to try some different items (or even get the same sandwich), but it won't be too often.
The only thing I've ever eaten at Time-Out is their ice cream cookie sandwich thing, and that's really all I need. It's about a half-gallon of ice cream sandwiched between two large-ish chocolate chip cookies. I don't know if they make it, or if it comes off a truck, or what... but it's frozen so solid you usually have to wait a few minutes before chowing down so you don't break a tooth.\n\nIf nothing else, they are one of the few old standards on Franklin Street that have not succumbed to the high-end shopping takeover, so they are worth being excited about.\n\nDo watch out for drunken Tarheels, though. This place has more disgusting drunkenness issues late at night than any bar I've been in.
Pros: Very clean and pretty little place. Really sweet-natured and attentive servers. Clean bathrooms (Haha it says a lot, ok?) Reasonable prices \n\nCons: Pho broth started ok but after a while starts to taste bitter and weird and left my mouth really dry. Meat is kinda chewy.\n\nThis place could have been SO AWESOME. A pho place so close to campus is one of those \
This is my favorite place of all time. I've driven from LA to SLO just to eat here. The barbecue sauce is AMAZING! Everything here is AMAZING! I can never decide what to get when I go here. Here's what I rotate between; tri tip sandwich, chicken wings and the ABC burger. Their fries are delicious also! Wings are fried and spicy. ABC burger is ten times more delicious then In N Out. I don't even know what to say about the trip tip sandwich. Words can not describe it. If you are a meat eater, you'll love it. It's bread, bbq sauce and like 2 inches of steak. Just amazing! My mouth is watering trying to describe the food.
After enduring years of crappy, undercooked, bad pizza in and around Ann Arbor (a la pizza house, cottage inn, and the like), Silvio's brings real pizza to town. Real pizza should be thin, have a crispy, bubbly, sourdough crust and be full of flavor, not grease, and this is the only place to get it.\n\nDon't be fooled by the \
The average rating for this place is right on point: a-ok. We came here with a relatively large group on a Friday night a little after ten o'clock. The first floor of the place was packed, but we found plenty of space upstairs in the corner, which is a pretty weird room considering how removed it is from everything. Or waiter was good about taking everyone's drink orders and being patient about it, though at one point he gave the right drinks to the wrong people. He was also cool about serving us while we played a stupid game with post its on our foreheads. We didn't order any food, but I was impressed that their kitchen is open so late and the menu is pretty big. \n\nWe were having a fun time until the band came on down stairs. The blaring sound was inescapable, even though we were kind of removed from where everything was happening. The music was so loud it was almost impossible to hear what anyone was saying right next to you. In case you're planning to pay with Discover, they DO accept it; not so cooperative on the check splitting though.
For the grand finale of my LA trip, we head over to Firestone Grill immediately after Splash Cafe.\n\nI like this place for many reasons. Obviously the tri-tip sandwich was just off da hook delicious even after clam chowder & calamari literally 5 mins ago. I also think this is one of the largest, most laid back sports bar/grills I've been to. The ladies at the register and the bartenders were very friendly in answering my questions. There is a large dining area inside and the humongous patio outside, but I don't doubt that people may end up spilling onto the sidewalk during game days as it was already quite packed for a Sunday evening. Despite the noise factor, I felt there was good energy from everyone and I was pretty comfortable. They have plenty of wide screen LCD TVs hung along the walls for your viewing pleasure (and the picture is clear :P). I don't think they have servers so you have to walk up to the counter or bar for service. Not a big deal to me IMO. \n\nSome might say that it was premature to give this a 5 star rating, but I have faith that the next time I visit (and I will go back), I will not be disappointed.
I've used this service for years. The best parts are that it tends to be on time and that there are bike racks on the front of the buses. Unfortunately the drivers must navigate roads filled with undergraduates on bicycles who think they are immortal. I've seen some close calls, but the drivers get serious kudos for being able to drive in that environment all day.\n\nThe drivers tend to be helpful and friendly, though some of them don't speak excellent English. There are a few routes I get a bit car-sick on, but that is due to the twisty roads and frequent stopping-accelerating. \n\nThe routes change, especially if there is construction going on. Make sure you check to see what is running during the summer and holidays. I've waited for shuttles that weren't going to come for days before.
Cannot believe how highly rated this place is, wow. Takes forever for them to get orders out even when its super slow. Our pizza took something like 65 minutes from the time we ordered it. Now i will wait that long if the pizza is going to be amazing, like at pizza popilis down in greek town. But this pizza was average at best. I think that the gluten free offerings are great but I really expected alot more. Great renovations btw the place looks amazing.
We went here a few weeks ago on the premiere weekend of Horrible Bosses. Loved that movie! \nMy wife wanted to come to a real movie theater that was more of a classic than one of these \
One of the best nights of my life. Hands down.\n\nIt was just amazing... The amount of people that gathered all for the same thing, The music pumping through your body everywhere you walked, The day turned to night, and the fireworks that illuminated the sky as you danced without a care to the blissful beats of Deadmau5 and others. It was a night that I will never forget.\n\nYes, it was not perfect, but what event of that size ever has been. I mean you don't show up to the largest rave in america and not expect there to be a shit load of people there. The numbers have been increasing every year, so why would this one be any different? If not, it would only be even bigger, since the biggest rising dance artist of the year, and the biggest name in trance are headlining...\n\nAnd as for all that crap about fence jumping and people getting hurt, and that one girl even died!.. OK, my condolences go out to that girls family, having a death in the family is hard, i cant even imagine it being your child.... BUT you cannot blame her taking drugs on an event, any more than video-games for kids shooting other kids. Its their own ignorance and apathy. As for all those people who got hurt. More than likely their own fault. Some 250 out of 180,000 people messed up, and now theres this huge dim view on raves. Cause there have been less riots and fights due to hockey, or football, or basketball, etc. right? (note: sarcasm above)\n\nMy final word/opinion: Definitely the best massive I've ever been to. Just as good, if not better than last year! Can't wait for next year!
I was in a desperate need of a good falafel and once again Yelp didn't let me down.\n\nOnce in the small deliciously smelling restaurant, I decided to go with the lamb gyro. I read many good things and decided to try it out, and was praying it wouldn't turn my entire mouth yellow(as stated by a few yelpers). \n\nI am happy to report that the gyro was deliciously seasoned and had huge pieces of lamb wrapped up inside. I couldn't tell you if my mouth turned yellow because I made it a point to not
file_input = open('yelp.dat', 'r')
convert_lowercase = [line.lower() for line in file_input]
with open('yelp.dat', 'w') as out:
out.writelines(sorted(convert_lowercase))
import string
remove = dict.fromkeys(map(ord, '\n ' + string.punctuation))
with open('yelp.dat') as file_input:
f = file_input.read()translate(remove)
i ran it but i checked the file there's still punctuation. where did i go wrong?
A couple issues I noticed.In your code, you open your file with file_input = open("file", "r"). This opens the file in read mode, but you don't receive the contents of this file until you do file_input.read(). Additionally, when using open() without a context manager, you should close the file with file_input.close(). Some code that would work:
import string
file_input = open("yelp.dat", "r")
file_content = file_input.read() # Read the contents of the file
convert_lowercase = file_content.lower() # No need to convert each line individually.
file_input.close() # Close the file
for punctuation in string.punctuation: # Go through the punctuation and replace it with an empty string (remove it) in your content.
convert_lowercase.replace(punctuation, "")
with open("yelp.dat", "w") as file:
file.write(convert_lowercase) # This overwrites all current content that is inside the file.
I do not understand the part that "smaller max_feature reduce overfitting in Random forest" ?
Also, what does it mean by bagging in Random Forest ?
I'll do my best to explain to you. You seem like you're new to Machine Learning algorithms. I'm sure you know what overfitting is! So let me explain why this happens very easily with Random Forest. Copying this users answer: https://cs.stackexchange.com/users/30903/kyle-hale,
Once there was a girl named Mel. And every day Mel played with a different friend, and every day she played it was a sunny, wonderful day.
Mel played with
Jordan on Monday,
Lily on Tuesday,
Mimi on Wednesday,
Olive on Thursday .. and then
on Friday Mel played with Brianna, and it rained. It was a terrible thunderstorm!
More days, more friends!
Mel played with Kwan on Saturday,
Grayson on Sunday,
Asa on Monday... and then
on Tuesday Mel played with Brooke and it rained again, even worse than before!
Now Mel's mom made a time-table for Mel deciding who she should play with and when, so that night during dinner she starts telling Mel all about the new time-table she has lined up. "Luis on Wednesday, Ryan on Thursday, Jemini on Friday, Bianca on Saturday -"
Mel frowned.
Mel's mom asked, "What's the matter, Mel, don't you like Bianca?"
Mel replied, "Oh, sure, she's great, but every time I play with a friend whose name starts with B, it rains!"
What's wrong with Mel's answer?
Well, it might not rain on Saturday.
Well, I don't know, I mean, Brianna came and it rained, Brooke came and it rained...
Yeah, I know, but rain doesn't depend on your friends.
This is exactly what happens with Random Forest. When you add too many features which is represented by "max_feature" variable, you will try very hard to find patterns where they don't exist and end up being too exact with the existing data that you overfit. When you're using decision trees (Random Forest) you will always get the best fit, since you're breaking the tree into branches and each branch will literally take in every feature and every possibility (if that makes sense).
For what is Bagging in Random Forest? You must first understand how the Random Forest traverses down the tree (it makes many trees and takes some kind of average) and decides which feature to consider important and so this requires you to FULLY UNDERSTAND how it works. You should probably YouTube it.