Using Tweepy, I am writing to a csv file with python and the header repeats every other row
x=0
x+=1
with open('NAME' + str(x) + '.csv', 'w' , newline='') as f:
for user in tweepy.Cursor(api.followers, screen_name="Name").items(5):
thewriter = csv.writer(f)
thewriter.writerow(['Username', 'location'])
thewriter = csv.writer(f)
thewriter.writerow([user.screen_name , user.location])
Your script should change to this:
x=0
x+=1
with open('NAME' + str(x) + '.csv', 'w' , newline='') as f:
thewriter = csv.writer(f)
thewriter.writerow(['Username', 'location'])
for user in tweepy.Cursor(api.followers, screen_name="Name").items(5):
thewriter.writerow([user.screen_name , user.location])
You only need to create thewriter object one time, and of course, you only want to create the headers once, not every other row as you saw. Moving things out of the for loop where you are looping through the rows enables that.
Related
I've got a CSV of client details for a bank project in Python 3. I've managed to create a function in which you can edit the client details but I want to exclude the last 2 columns as and and can't figure out how.
Example of CSV data:
first_name,last_name,title,pronouns,dob,occupation,account_balance,overdraft_limit
Garner,Coupman,Ms,Male,14/04/2022,General Manager,2200.76,2.28
Jens,Eldrid,Honorable,Male,13/11/2021,Research Associate,967.64,79.15
Edit function:
if choice == "4":
editClient = int(input("Please enter the index number of the client you wish to edit: "))
print("Please enter the details for each of the following: ")
for i in range(len(existing_clients[0])):
newDetails = input("Enter new data for " + str(existing_clients[0][i]) + ": ")
existing_clients[editClient][i] = newDetails
changes = input("Are you sure you'd like to make these changes? Enter Yes or No")
if changes == ("Yes"):
# Newline fixed the spacing issue I was having
with open("mock_data.csv", "w+", newline="") as file:
reader = csv.writer(file)
for i in range(len(existing_clients)):
reader.writerow(existing_clients[i])
if changes == ("No"):
exit()
I've tried changing
for i in range(len(existing_clients[0])):
to
for i in range(len(existing_clients[0:6])):
and I thought this worked until I tried editing a client later the row 6.
I've also messed around a lot with
newDetails = input("Enter new data for " + str(existing_clients[0][i]) + ": ")
to no avail.
Edit the row with slicing and exclude the last two columns:
with open("mock_data.csv", "w", newline="") as file:
writer = csv.writer(file)
for client in existing_clients:
writer.writerow(client[:-2]) # exclude last two columns
Working example with data:
input.csv
first_name,last_name,title,pronouns,dob,occupation,account_balance,overdraft_limit
Garner,Coupman,Ms,Male,14/04/2022,General Manager,2200.76,2.28
Jens,Eldrid,Honorable,Male,13/11/2021,Research Associate,967.64,79.15
test.py
import csv
with open('input.csv', newline='') as f:
reader = csv.reader(f)
data = list(reader)
with open('output.csv', 'w', newline='') as f:
writer = csv.writer(f)
for line in data:
writer.writerow(line[:-2])
output.csv
first_name,last_name,title,pronouns,dob,occupation
Garner,Coupman,Ms,Male,14/04/2022,General Manager
Jens,Eldrid,Honorable,Male,13/11/2021,Research Associate
To select specific columns, you could concatenate different slices:
writer.writerow(line[:2] + line[5:6]) # column indexes 0, 1, and 5
Or use DictReader/DictWriter:
import csv
with open('input.csv', newline='') as f:
reader = csv.DictReader(f)
data = list(reader)
with open('output.csv', 'w', newline='') as f:
writer = csv.DictWriter(f, fieldnames=['last_name', 'occupation'], extrasaction='ignore')
writer.writeheader()
for line in data:
writer.writerow(line)
output.csv
last_name,occupation
Coupman,General Manager
Eldrid,Research Associate
I'm having some trouble looping through the code below., Essentially, I am attempting to loop through a folder of files ("filename_x.csv") and remove all commas out within the fields, then I would like for the output to be saved with the "out". The code works like a charm if I input the counter individually, but it will not move to the next count. I know this is very close to functional status, but the 'with' statements may allow the other files to close, so it can't move to the next count or iteration (file path is just an example). Please, help!
import pandas as pd
n = 1
x = str(n)
while n < 9:
import csv
with open("C:\\Desktop\\server_"+x+".csv",'r', newline='') as infile, open("C:\\Desktop\\server_"+x+"_out.csv",'w', newline='') as outfile:
reader = csv.reader(infile)
writer = csv.writer(outfile)
for row in reader:
writer.writerow(item.replace(",", "") for item in row)
n += 1
The line:
x = str(n)
is in the wrong place it needs to be in the loop at the top.
You only set at to the initial value of n and then it stays that way.
You do not actually even need it. Just put the str(n) directly in the string creation like this:
"C:\\Desktop\\server_" + str(n) + ".csv"
I would also recommend you use a different loop structure. The while format you are using does not make sense for the way you are using it. You should be using a for loop, like this:
import pandas as pd
import csv
for n in range(1, 9):
with open("C:\\Desktop\\server_" + str(n) + ".csv",'r', newline='') as infile, open("C:\\Desktop\\server_" + str(n) + "_out.csv",'w', newline='') as outfile:
reader = csv.reader(infile)
writer = csv.writer(outfile)
for row in reader:
writer.writerow(item.replace(",", "") for item in row)
This fixes a few things. You simplify the code, but also there were some other issues. You had the increment of n in the wrong place which this corrects. Also you were doing the import cvs inside the loop which would probably not break anything, but would at the very least be inefficient since it would attempt reload the cvs module every pass through the outer loop.
Asssuming you have files server_1.csv, server_2.csv ...
Two things:
you don't need to keep track of both x and n
you need to increment n in the right place. You are currently incrementing it once per row.
import pandas as pd
import csv # <<< no need to repeat the import statement though it's harmless
n = 1
while n < 9:
with open("C:\\Desktop\\server_" + str(n) + ".csv",'r', newline='') as infile, open("C:\\Desktop\\server_" + str(n) + "_out.csv",'w', newline='') as outfile:
reader = csv.reader(infile)
writer = csv.writer(outfile)
for row in reader:
writer.writerow(item.replace(",", "") for item in row)
# finished with this file, increment n
n += 1
I am trying to read a csv and then transpose one column into a row.
I tried following a tutorial for reading a csv and then one for writing but the data doesnt stay saved to the list when I try to write the row.
import csv
f = open('bond-dist-rep.csv')
csv_f = csv.reader(f)
bondlength = []
with open("bond-dist-rep.csv") as f:
for row in csv_f:
bondlength.append(row[1])
print (bondlength)
print (len(bondlength))
with open('joined.csv', 'w', newline='') as csvfile:
csv_a = csv.writer (csvfile, delimiter=',',quotechar='"',
quoting=csv.QUOTE_ALL)
csv_a.writerow(['bondlength'])
with open('joined.csv') as csvfile:
readCSV = csv.reader(csvfile, delimiter=',')
for row in readCSV:
print(row)
print(row[0])
f.close()
The matter is that you only read the first value of each line and write only a string in the new file.
In order to transpose the read lines, you can use the zip function.
I also delete the first open function which is useless because of the good use of with for opening the file.
Here the final code:
import csv
bondlength = []
with open("bond-dist-rep.csv") as csv_f:
read_csv = csv.reader(csv_f)
for row in read_csv:
bondlength.append(row)
# delete the header if you have one
bondlength.pop(0)
with open('joined.csv', 'w') as csvfile:
csv_a = csv.writer (csvfile, delimiter=',')
for transpose_row in zip(*bondlength):
csv_a.writerow(transpose_row)
I'm trying to print certain values into a CSV from a list I'm passing through. Here is my function:
current = datetime.datetime.now() #defining datetime for use in function
def write_csv(custody_parts):
with open((current.strftime("%m_%d_%y_%H_%M_%S")) + '.csv', 'w', newline='') as csvfile:
csvfile = io.StringIO()
fieldnames = ['serial', 'user', 'time']
writer = csv.DictWriter(csvfile, extrasaction='ignore', fieldnames=fieldnames)
writer.writeheader()
writer.writerows(custody_parts)
csvfile.getvalue()
print(csvfile.getvalue())
return(csvfile.getvalue())```
Then I call it with the list I'm trying to pass through:
write_csv(parts)
and it creates the file: 06_06_18_12_13_53.csv
and prints to the screen:
serial,user,time
serial1,user1,date1
serial2,user2,date2
serial3,user3,date3
but the file it creates is empty, so it isn't writing to the file it is creating.
Can someone point me in the right direction?
Thanks ~
EDIT:
I ended up going with this instead:
def write_csv(custody_parts):
current = datetime.datetime.now()
with open((current.strftime("%m_%d_%y_%H_%M_%S")) + '.csv', 'w', newline='') as csvfile:
custodywriter = csv.writer(csvfile, delimiter=',', quoting=csv.QUOTE_MINIMAL)
custodywriter.writerow(['serial', 'user', 'time'])
for i in custody_parts:
x = [i["serial"],i["user"],i["time"]]
custodywriter.writerow(x)
custodywriter.writerow(["End of Report"])
I have a date column in a CSV file which I am trying to format from dd/mm/yyyy to ddmmyyyy. Some of the days and months are single digit which leave them as dmyyyy. When I run a print statement all of the rows output correctly.
import csv
with open(r'input file path,'r') as csvfile:
with open(r'outputfilepath,'w') as output:
w = csv.writer(output)
r = csv.reader(csvfile)
for row in r:
#this takes care of incomplete rows at the end
if len(row[6])>1:
dt = row[6].split("/")
n = 0
for n in range(len(dt)):
if len(dt[n])<2:
dt[n] = '0'+dt[n]
else:
dt[n]
row[6] = dt[0]+dt[1]+dt[2]
print(row)
else:
break
Print Output:
['a', '', 'Tom', 'Smith', 'J ', '', '12201956']
['b', '', 'Rick ', 'JOHNSON ', ' ', '', '08121922']
['c', '', 'Morty', 'Harvey', ' ', '', '06031940']
When I change the print to write rows:
import csv
with open(r'input file path,'r') as csvfile:
with open(r'outputfilepath,'w') as output:
w = csv.writer(output)
r = csv.reader(csvfile)
for row in r:
#this takes care of incomplete rows at the end
if len(row[6])>1:
dt = row[6].split("/")
n = 0
for n in range(len(dt)):
if len(dt[n])<2:
dt[n] = '0'+dt[n]
else:
dt[n]
row[6] = dt[0]+dt[1]+dt[2]
w.writerows(row)
else:
break
I get the output below. I've tried moving the writerows function around with no luck. Looking at the CSV module documentation it should delimit on the commas. I'm relatively new to python.
To fix your problem change w.writerows(row) to w.writerow(row). The difference is between the singular and the plural is that the plural version thinks its getting a collection of rows to write. It treats each item in the row you gave as a single row.
Also newline='' to your open because the csv module interacts poorly with universal newline mode on windows. (It tries to write '\r\n'. Universal newline translates that to '\r\r\n'.)
Finally, use datetime to fix your dates.
import csv
from datetime import datetime
with open(inpath, 'r', newline='') as fin:
with open(outpath, 'w', newline='') as fout:
reader = csv.reader(fin)
writer = csv.writer(fout)
for row in reader:
row[6] = datetime.strptime(row[6], '%m/%d/%Y').strftime('%m%d%Y')
writer.writerow(row)