I have a CSV file where the first row in the first column is blank with some numbers in the second and third row. This whole column is useless and I need to remove it so I can convert the data into a JSON file. I just need to know how to remove the first column of data so I can parse it. Any help is greatly appreciated!
My script is as follows
#!/usr/bin/python3
import pandas as pd
import csv, json
xls = pd.ExcelFile(r'C:\Users\Andy-\Desktop\Lab2Data.xlsx')
df = xls.parse(sheetname="Sheet1", index_col=None, na_values=['NA'])
df.to_csv('file.csv')
file = open('file.csv', 'r')
lines = file.readlines()
file.close()
data = {}
with open('file.csv') as csvFile:
csvReader = csv.DictReader(csvFile)
for rows in csvReader:
id = rows['Id']
data[id] = rows
with open('Lab2.json', 'w') as jsonFile:
jsonFile.write(json.dumps(data, indent=4))
I don't know much about json files but this will remove the first column from your csv file.
with open ('new_file.csv', 'w') as out_file :
with open ('file.csv') as in_file :
for line in in_file :
test_string = line.strip ('\n').split (',')
out_file.write (','.join (test_string [1:]) + '\n')
Related
I'm using pandas to open a CSV file that contains data from spotify, meanwhile, I have a txt file that contains various artists names from that CSV file. What I'm trying to do is get the value from each row of the txt and automatically search them in the function I've done.
import pandas as pd
import time
df = pd.read_csv("data.csv")
df = df[['artists', 'name', 'year']]
def buscarA():
start = time.time()
newdf = (df.loc[df['artists'].str.contains(art)])
stop = time.time()
tempo = (stop - start)
print (newdf)
e = ('{:.2f}'.format(tempo))
print (e)
with open("teste3.txt", "r") as f:
for row in f:
art = row
buscarA()
but the output is always the same:
Empty DataFrame
Columns: [artists, name, year]
Index: []
The problem here is that when you read the lines of your file in Python, it also gets the line break per row so that you have to strip it off.
Let's suppose that the first line of your teste3.txt file is "James Brown". It'd be read as "James Brown\n" and not recognized in the search.
Changing the last chunk of your code to:
with open("teste3.txt", "r") as f:
for row in f:
art = row.strip()
buscarA()
should work.
I have the following in a .txt file
1.['LG','Samsung','Asus','HP','Apple','HTC']
2.['covid','vaccine','infection','cure','chloroquine']
3.['p2p','crypto','bitcoin','litecoin','blockchain']
How do I convert the above into a csv file under different columns?
My current code is this
import csv
with open('Full_txt_results.txt', 'r') as in_file:
stripped = (line.strip() for line in in_file)
lines = (line.split(",") for line in stripped if line)
with open('textlabels.csv', 'w') as out_file:
writer = csv.writer(out_file)
writer.writerows(lines)
the code currently gives the result in the following format in csv
Column 1 Column2 column 3 column 4 Column 5 column 6
['LG' 'Samsung' 'Asus' 'HP' 'Apple' 'HTC']
['covid' 'vaccine' 'infection' 'cure' 'chloroquine']
['p2p' 'crypto' 'bitcoin' 'litecoin' 'blockchain']
The texts are spilled in to different columns.
Ideal output required is in the below format
Column 1 Column2 column 3
LG Covid p2p
Samsung Vaccine crypto
Asus Infection bitcoin
HP cure litecoin
Apple chloroquine blockchain
HTC
Use ast module to convert string to list object and then write to csv using writerow method
Ex:
import csv
import ast
with open('Full_txt_results.txt') as in_file, open('textlabels.csv', 'w', newline="") as out_file:
writer = csv.writer(out_file)
data = [ast.literal_eval(line.strip().split(".")[1]) for line in in_file] #If you do not have column number(1.,2.,...) Use [ast.literal_eval(line.strip()) for line in in_file]
for row in zip(*data):
writer.writerow(row)
Demo:
import csv
import ast
with open(filename) as in_file, open(outfile, 'w', newline="") as out_file:
writer = csv.writer(out_file)
data = [ast.literal_eval(line.strip()) for line in in_file]
for row in zip(*data):
writer.writerow(row)
SRC txt file
['LG','Samsung','Asus','HP','Apple','HTC']
['covid','vaccine','infection','cure','chloroquine']
['p2p','crypto','bitcoin','litecoin','blockchain']
Output:
LG,covid,p2p
Samsung,vaccine,crypto
Asus,infection,bitcoin
HP,cure,litecoin
Apple,chloroquine,blockchain
I am trying to read a csv and then transpose one column into a row.
I tried following a tutorial for reading a csv and then one for writing but the data doesnt stay saved to the list when I try to write the row.
import csv
f = open('bond-dist-rep.csv')
csv_f = csv.reader(f)
bondlength = []
with open("bond-dist-rep.csv") as f:
for row in csv_f:
bondlength.append(row[1])
print (bondlength)
print (len(bondlength))
with open('joined.csv', 'w', newline='') as csvfile:
csv_a = csv.writer (csvfile, delimiter=',',quotechar='"',
quoting=csv.QUOTE_ALL)
csv_a.writerow(['bondlength'])
with open('joined.csv') as csvfile:
readCSV = csv.reader(csvfile, delimiter=',')
for row in readCSV:
print(row)
print(row[0])
f.close()
The matter is that you only read the first value of each line and write only a string in the new file.
In order to transpose the read lines, you can use the zip function.
I also delete the first open function which is useless because of the good use of with for opening the file.
Here the final code:
import csv
bondlength = []
with open("bond-dist-rep.csv") as csv_f:
read_csv = csv.reader(csv_f)
for row in read_csv:
bondlength.append(row)
# delete the header if you have one
bondlength.pop(0)
with open('joined.csv', 'w') as csvfile:
csv_a = csv.writer (csvfile, delimiter=',')
for transpose_row in zip(*bondlength):
csv_a.writerow(transpose_row)
Not to over write csv, I tried 'a' but I get no separators.
Tried to replaced 'w' with 'a', but get no separator just mismash
import csv
with open('csv/1.n04', 'r') as csvFile:
reader = csv.reader(csvFile)
s = "".join(row[0] for row in reader)
a = s.count('0')
b = s.count('1')
csvFile.close()
import csv
f = open('csv/sum1.n04', 'w')
if b>=1:
f.write(str((a+b)/b))
else:
f.write(str(a))
f.close
import os
os.remove("csv/1.n04")
print ('Klar')
No error messages, it just over write csv file
I tried to convert text file content into a .csv format by reading each and every line using python csv module and converting that to a list. But i couldn't get the expected output and it stores the first line in a row but second line will be stored in 3rd row and 5th so on. Since I am new to python i don't know how to skip the line and store it in the right order.
def FileConversion():
try:
with open('TextToCSV.txt', 'r') as textFile:
LineStripped = (eachLine.strip() for eachLine in textFile)
lines = (eachLine.split(" ") for eachLine in LineStripped if eachLine)
with open('finalReport.csv', 'w') as CSVFile:
writer = csv.writer(CSVFile)
writer.writerow(('firstName', 'secondName', 'designation', "age"))
writer.writerows(lines)
Why don't you try doing something more simple:
import pandas as pd
aux = pd.read_csv("TextToCSV.txt", sep=" ")
aux.columns=['firstName', 'secondName', 'designation', "age"]
aux.to_csv("result.csv")