I am reading data from .csv data and want to write parts of that data into an output file.
When I execute the program and print the results, I get the complete data set of the input file.
However, when I hit print() again, only the last line of the input file is shown.
When I write the print-result into another csv file, as well only the last line is transfered
Basically I am new at this and struggle to understand how data is stored in cache and passed on.
import csv
with open("path to the input file") as file:
reader = csv.reader(file)
for line in file:
input_data = line.strip().split(";")
print(input_data)
with open(os.path.join("path to the output file"), "w") as file1:
toFile = input_data
file1.write(str(toFile))
There is no error messages, just not the expected result. I expect 10 lines to be transferred, but only the last makes it to the output .csv
Thank you for your help!
When you loop over the lines in the csv each iteration you assign the value of that line to input data, overwriting the value previously stored in input_data.
I would recommend something like the following:
import csv
with open('path to input file', 'r') as input, open('path to output file', 'w') as output:
reader = csv.reader(input)
for line in reader:
ouput.write(line.strip().split(';'))
you can open multiple files in a single with clause like I showed in the example. Then for each line in the file you write the stripped and split string to the file.
This should do it. You created the reader object correctly but didn't use it. I hope my example will better your understanding of the reader class.
#!/usr/bin/env python
import csv
from os import linesep
def write_csv_to_file(csv_file_name, out_file_name):
# Open the csv-file
with open(csv_file_name, newline='') as csvfile:
# Create a reader object, pass it the csv-file and tell it how to
# split up values
reader = csv.reader(csvfile, delimiter=',', quotechar='|')
# Open the output file
with open(out_file_name, "w") as out_file:
# Loop through the rows that the reader found
for row in reader:
# Join the row values using a comma as separator
r = ', '.join(row)
# Print row and write to output file
print(r)
out_file.write(r + linesep)
if __name__ == '__main__':
csv_file = "example.csv"
out_file = "out.txt"
write_csv_to_file(csv_file, out_file)
So, I've got a CSV file (league.csv') in my directory that I want to access through Python.
When I do this:
with open('league.csv', 'r') as csvFile:
csvReader = csv.reader(csvFile)
next(csvReader)
for line in csvReader:
print(line[0])
It works great. But when I try to make it into a function it doesn't work:
def createLeague(csv):
with open(csv, 'r') as csvFile:
csvReader = csv.reader(csvFile)
next(csvReader)
for line in csvReader:
print(line[0])
return
If I call this using:
createLeague('league.csv')
I get the error
csvReader = csv.reader(csvFile)
AttributeError: 'str' object has no attribute 'reader'
This makes no sense because type(csvFile) is not str, and I'm doing exactly the same as above?
Could someone help me, I'm getting really frustrated!
Thanks!
See inline comment what goes wrong.
import csv # "csv" variable at "with open..." overwrites
# cvs module.
from csv import reader # required to use the function "reader" from csv.
def createLeague(csv): # "csv" variable overwrites cvs module.
with open(csv, 'r') as csvFile:
csvReader = reader(csvFile) # skip the csv. here and it works for you.
next(csvReader)
for line in csvReader:
print(line[0])
return
Alternatively:
Below example with changed csv filename does not overwrite the csv module that you just have loaded. The mix-up triggers the error to occur.
import csv
#from csv import reader
def createLeague(csv_file):
with open(csv_file, 'r') as csvFile: # changed variable "csv" filename.
csvReader = csv.reader(csvFile)
next(csvReader)
for line in csvReader:
print(line[0])
return
Don’t use the name of the library as the name of the variable in the function definition.
My Code:
import csv
import requests
url = 'https://xxxxxxxxxxxxxxxxxxx.csv'
r = requests.get(url)
text = r.iter_lines()
reader = csv.reader(text, delimiter=',')
for row in reader:
print(row)
I am getting the following error:
_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)
Just save the csv separately and then read in with pandas:
import pandas
pd.read_csv('SO/AN_LATEST_ANNOUNCED.csv')
Result:
(To save the file separately just open the link in a browser.)
I'm new to python and I'm trying to import some URL's i scraped into a csv file but it is parsing every character in the web addresses into a difference cell. Here's my code:
import csv
with open('test.csv', 'w') as f:
csv_writer = csv.writer(f)
csv_writer.writerow(['Web Address'])
csv_write.writerows(filter_records)
If i put brackets around the filter_records variable, it just returns the entire list of URLs in a single cell
Any guidance would be great.
Thanks
Garrett
You can do something like this:
import csv
filter_records = ['www.google.com', 'www.stackoverflow.com', 'www.facebook.com']
with open('test.csv', 'w') as f:
csv_writer = csv.writer(f)
csv_writer.writerow(['Web Address'])
[csv_writer.writerow([record]) for record in filter_records]
or
import csv
filter_records = ['www.google.com', 'www.stackoverflow.com', 'www.facebook.com']
with open('test.csv', 'w') as f:
csv_writer = csv.writer(f)
csv_writer.writerow(['Web Address'])
csv_writer.writerows([record] for record in filter_records)
This happens because python a string is a list of chars. writerow() method receives a list as param, and writerows() receives a list of lists. So, you will get an comma-split string in every row.
I have data which is being accessed via http request and is sent back by the server in a comma separated format, I have the following code :
site= 'www.example.com'
hdr = {'User-Agent': 'Mozilla/5.0'}
req = urllib2.Request(site,headers=hdr)
page = urllib2.urlopen(req)
soup = BeautifulSoup(page)
soup = soup.get_text()
text=str(soup)
The content of text is as follows:
april,2,5,7
may,3,5,8
june,4,7,3
july,5,6,9
How can I save this data into a CSV file.
I know I can do something along the lines of the following to iterate line by line:
import StringIO
s = StringIO.StringIO(text)
for line in s:
But i'm unsure how to now properly write each line to CSV
EDIT---> Thanks for the feedback as suggested the solution was rather simple and can be seen below.
Solution:
import StringIO
s = StringIO.StringIO(text)
with open('fileName.csv', 'w') as f:
for line in s:
f.write(line)
General way:
##text=List of strings to be written to file
with open('csvfile.csv','wb') as file:
for line in text:
file.write(line)
file.write('\n')
OR
Using CSV writer :
import csv
with open(<path to output_csv>, "wb") as csv_file:
writer = csv.writer(csv_file, delimiter=',')
for line in data:
writer.writerow(line)
OR
Simplest way:
f = open('csvfile.csv','w')
f.write('hi there\n') #Give your csv text here.
## Python will convert \n to os.linesep
f.close()
You could just write to the file as you would write any normal file.
with open('csvfile.csv','wb') as file:
for l in text:
file.write(l)
file.write('\n')
If just in case, it is a list of lists, you could directly use built-in csv module
import csv
with open("csvfile.csv", "wb") as file:
writer = csv.writer(file)
writer.writerows(text)
I would simply write each line to a file, since it's already in a CSV format:
write_file = "output.csv"
with open(write_file, "wt", encoding="utf-8") as output:
for line in text:
output.write(line + '\n')
I can't recall how to write lines with line-breaks at the moment, though :p
Also, you might like to take a look at this answer about write(), writelines(), and '\n'.
To complement the previous answers, I whipped up a quick class to write to CSV files. It makes it easier to manage and close open files and achieve consistency and cleaner code if you have to deal with multiple files.
class CSVWriter():
filename = None
fp = None
writer = None
def __init__(self, filename):
self.filename = filename
self.fp = open(self.filename, 'w', encoding='utf8')
self.writer = csv.writer(self.fp, delimiter=';', quotechar='"', quoting=csv.QUOTE_ALL, lineterminator='\n')
def close(self):
self.fp.close()
def write(self, elems):
self.writer.writerow(elems)
def size(self):
return os.path.getsize(self.filename)
def fname(self):
return self.filename
Example usage:
mycsv = CSVWriter('/tmp/test.csv')
mycsv.write((12,'green','apples'))
mycsv.write((7,'yellow','bananas'))
mycsv.close()
print("Written %d bytes to %s" % (mycsv.size(), mycsv.fname()))
Have fun
What about this:
with open("your_csv_file.csv", "w") as f:
f.write("\n".join(text))
str.join() Return a string which is the concatenation of the strings in iterable.
The separator between elements is
the string providing this method.
In my situation...
with open('UPRN.csv', 'w', newline='') as out_file:
writer = csv.writer(out_file)
writer.writerow(('Name', 'UPRN','ADMIN_AREA','TOWN','STREET','NAME_NUMBER'))
writer.writerows(lines)
you need to include the newline option in the open attribute and it will work
https://www.programiz.com/python-programming/writing-csv-files