I have been using this code to extract data from website and store it into a csv file. However, I would like to know how can I add an empty row between the data without using hardcode, for instance a empty row that seperate the title 'Total' and the data extracted from Total.text. (Like this picture.) This is the link I extra data "https://finance.yahoo.com/quote/0700.HK/sustainability?p=0700.HK"
with open(f'{stockname}_sustainability.csv', 'w', newline='') as csvfile:
fieldnames = ['Total','Type','Percentile','ERS','SRS','GRS','mclevel']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
writer.writerow({'Total':int(Total.text),'Type' : Type.text,'Percentile':int(Percentile[0:2]), 'ERS' : float(ERS.text),'SRS':float(SRS.text),'GRS':float(GRS.text),'mclevel':int(mclevel.text)})
import csv
with open('Book1.csv', 'w', newline='') as csvfile:
writer = csv.writer(csvfile, delimiter=',')
fieldnames = ['Total', 'Type', 'Percentile', 'ERS', 'SRS', 'GRS', 'mclevel']
writer.writerow("".join(fieldnames))
writer.writerow("\n")
writer.writerow({'Total':int(Total.text),'Type' : Type.text,'Percentile':int(Percentile[0:2]), 'ERS' : float(ERS.text),'SRS':float(SRS.text),'GRS':float(GRS.text),'mclevel':int(mclevel.text)})
By using write.writerow("\n") you will create one empty row.
Related
I have a CSV file where the first row in the first column is blank with some numbers in the second and third row. This whole column is useless and I need to remove it so I can convert the data into a JSON file. I just need to know how to remove the first column of data so I can parse it. Any help is greatly appreciated!
My script is as follows
#!/usr/bin/python3
import pandas as pd
import csv, json
xls = pd.ExcelFile(r'C:\Users\Andy-\Desktop\Lab2Data.xlsx')
df = xls.parse(sheetname="Sheet1", index_col=None, na_values=['NA'])
df.to_csv('file.csv')
file = open('file.csv', 'r')
lines = file.readlines()
file.close()
data = {}
with open('file.csv') as csvFile:
csvReader = csv.DictReader(csvFile)
for rows in csvReader:
id = rows['Id']
data[id] = rows
with open('Lab2.json', 'w') as jsonFile:
jsonFile.write(json.dumps(data, indent=4))
I don't know much about json files but this will remove the first column from your csv file.
with open ('new_file.csv', 'w') as out_file :
with open ('file.csv') as in_file :
for line in in_file :
test_string = line.strip ('\n').split (',')
out_file.write (','.join (test_string [1:]) + '\n')
I tried many ways/combinations to insert headers (I need to add two column headers) but the file doesn't accept headers at file creation. At best, I get headers as rows come into the file one after the other. I can't see how to enter the headers only once persistently. Can you see in the code below where I could make some change please? Thanks.
with open(MYFILE, "w", newline='') as csvWriter, open('read.csv', 'r', newline='') as csvReader:
if keyword != "q":
fieldnames = [engine, keyword]
#fieldnames = ['Engine', 'Keywords']
# writer = csv.DictWriter(csvWriter, fieldnames=[engine, keyword], extrasaction='ignore')
writer = csv.DictWriter(csvWriter, fieldnames=fieldnames)
writer.writeheader()
reader = csv.DictReader(csvReader, fieldnames=fieldnames)
writer.writerow({"Engine": engine, "Keywords": keyword})
writer.writerows(reader)
I'm trying to save data into a csv file. I have two columns but no headers currently. I'd need some column titles like in any spreadsheet basically.
UPDATE ==============================================
I've tried to insert to no avail the first block hereafter before and after the 2nd one. There is certainly something I'm not doing right but I don't know. Any suggestion, please?
STATUS = "quit"
print("Press [q] to quit at any time")
menu_engine = {}
menu_engine['1'] = "Google Search - Large Results"
menu_engine['2'] = "Google Search - Small Results"
menu_engine['3'] = "Yahoo Search - Large Results"
menu_engine['4'] = "Yahoo Search - Small Results"
while True:
options = menu_engine.keys()
for entry in options:
print(entry, menu_engine[entry])
engine = input("Enter your Type of Search: ")
while STATUS != "q":
keyword = input("Enter keyword(s): ")
with open(MYFILE, "a", newline='') as csvWriter:
if keyword != "q":
fieldnames = [engine, keyword]
writer = csv.DictWriter(csvWriter, fieldnames=fieldnames, extrasaction='ignore')
writer.writeheader()
THE EXPECTED OUTPUT in the CSV File:
Engine Number,Keywords
4,man man
4,mate mate
you write the header with writeheader() then you can just write the rows
import csv
with open('new.dat', "w", newline='') as csvWriter, open('test.dat', 'r', newline='') as csvReader:
fieldnames = ['Engine', 'Keywords']
writer = csv.DictWriter(csvWriter, fieldnames=fieldnames)
reader = csv.DictReader(csvReader, fieldnames=fieldnames)
writer.writeheader()
writer.writerows(reader)
INPUT FILE
sef,56
sfd,67
eryt,67
sdfsf,34
OUTPUT FILE
Engine,Keywords
sef,56
sfd,67
eryt,67
sdfsf,34
UPDATE
I felt there should be an easier way to do this and seems you can use fileinput module from the standard python library to edit the same file inplace and insert the record at the start. this will save you having to the move or rename files.
import fileinput
headers = ['Engine','Type']
with fileinput.input('test.dat', inplace=1) as my_file:
for line in my_file:
if my_file.isfirstline():
print(",".join(headers))
print(line, end='')
Try:
df["new_column"]=" "
df.to_csv("your_file.csv", index= False)
print(df) #to see if it's what you need
when i export from a list to CSV strings such as "1/1" or "1/3", they are automatically converted to dates, ie "01-Jan".
How can i keep as "1/1"?
Here is my code:
import csv
my_list = [["1/1", "1/3"]]
with open("my_csv.csv", "a", newline="") as csvfile:
writer = csv.writer(csvfile)
for i in my_list:
writer.writerow(i)
Thanks.
This has nothing to do with python. Most likely the program you're viewing the .csv in is formatting the output.
Also, are you sure you want the output that way? Currently the output is:
1,/,1
1,/,3
You can remove the loop and just write the list to the file
import csv
my_list = ["1/1", "1/3"]
with open("my_csv.csv", "a", newline="") as csvfile:
writer = csv.writer(csvfile)
writer.writerow(my_list)
for this output:
1/1,1/3
I am trying to read a csv and then transpose one column into a row.
I tried following a tutorial for reading a csv and then one for writing but the data doesnt stay saved to the list when I try to write the row.
import csv
f = open('bond-dist-rep.csv')
csv_f = csv.reader(f)
bondlength = []
with open("bond-dist-rep.csv") as f:
for row in csv_f:
bondlength.append(row[1])
print (bondlength)
print (len(bondlength))
with open('joined.csv', 'w', newline='') as csvfile:
csv_a = csv.writer (csvfile, delimiter=',',quotechar='"',
quoting=csv.QUOTE_ALL)
csv_a.writerow(['bondlength'])
with open('joined.csv') as csvfile:
readCSV = csv.reader(csvfile, delimiter=',')
for row in readCSV:
print(row)
print(row[0])
f.close()
The matter is that you only read the first value of each line and write only a string in the new file.
In order to transpose the read lines, you can use the zip function.
I also delete the first open function which is useless because of the good use of with for opening the file.
Here the final code:
import csv
bondlength = []
with open("bond-dist-rep.csv") as csv_f:
read_csv = csv.reader(csv_f)
for row in read_csv:
bondlength.append(row)
# delete the header if you have one
bondlength.pop(0)
with open('joined.csv', 'w') as csvfile:
csv_a = csv.writer (csvfile, delimiter=',')
for transpose_row in zip(*bondlength):
csv_a.writerow(transpose_row)
I have a small script that reads a pipe '|' delimited csv file and then writes it to another file as comma delimited. It works fine. Code, input file and output file below:
import csv
ifile = "d:\\python\\project\\cars-original.csv"
ofile = "d:\\python\\project\\cars.csv"
with open(ifile, 'r') as f:
reader = csv.reader(f, delimiter='|')
with open(ofile, 'w', newline='') as of:
writer = csv.writer(of, delimiter=',')
for row in reader:
writer.writerow(row)
Reading|Make|Model|Type|Value
Reading 0|Toyota|Previa|distance|19.83942
Reading 1|Dodge|Intrepid|distance|31.28257
Reading,Make,Model,Type,Value
Reading 0,Toyota,Previa,distance,19.83942
Reading 1,Dodge,Intrepid,distance,31.28257
I now want to modify the script so that it can auto-read the delimiter type.
I have found several examples online, but when I run mine I get a blank file for output. No errors, just blank. Not sure what I am doing wrong.
My modified (broken) script:
import csv
ifile = "d:\\python\\projects\\cars-original.csv"
ofile = "d:\\python\\projects\\cars.csv"
with open(ifile, 'r') as f:
reader = csv.reader(f)
dialect = csv.Sniffer().sniff(f.read(1024), delimiters=',|')
with open(ofile, 'w', newline='') as of:
writer = csv.writer(of, dialect)
for row in reader:
writer.writerow(row)
In the example in Python documentation (https://docs.python.org/3.6/library/csv.html) file pointer is moved to the beginning of the file, right after the dialect detection.
with open('example.csv', newline='') as csvfile:
dialect = csv.Sniffer().sniff(csvfile.read(1024))
csvfile.seek(0)
reader = csv.reader(csvfile, dialect)
This works:
import csv
ifile = "d:\\python\\project\\cars-original.csv"
ofile = "d:\\python\\project\\cars.csv"
with open(ifile, 'r') as f:
dialect = csv.Sniffer().sniff(f.readline(), [',', '|'])
f.seek(0)
reader = csv.reader(f, dialect)
with open(ofile, 'w', newline='') as of:
writer = csv.writer(of, delimiter=',')
for row in reader:
writer.writerow(row)