I want to append data into multiple columns of a csv file at a time.
I have defined a function "writeCSV" for it which is taking col_name and col_value to append in csv file.
def writeCSV(col_name,col_data):
with open('COMPONENT_DETAILS.csv',"a") as COMPONENT_DETAILS_file:
fieldnames = ['ID','module_name','module_id','tech_name','component_name','connecion','counter','connection_type']
csv_writer = csv.DictWriter(COMPONENT_DETAILS_file,fieldnames=fieldnames)
csv_writer.writerow({col_name:col_data})
Now, suppose I want to append data into columns- tech_name, component_name,connection_type, how should I pass the argument to writeCSV() function? Would dictionary be helpful here?
Right now, I can only append to only one column by calling function with below syntax,
writeCSV('tech_name': 'Python')
def writeCSV(dictionary):
##Your code
csv_writer.writerow(dictionary)
writeCSV({'tech_name': 'Python2','component_name': 'Python3'})
This should do your work.As you have defined a DictWriter for writing to csv, you will have no problem in implementing this method.
Related
I have this script that reads a CSV and saves the second column to a list, I'm trying to get it to write the contents of the list to a new CSV. The problem is every entry should have its own row but the new file sets everything into the same row.
I've tried moving the second with open code to within the first with open and I've tried adding a for loop to the second with open but no matter what I try I don't get the right results.
Here is the code:
import csv
col_store=[]
with open('test-data.csv', 'r') as rf:
reader = csv.reader(rf)
for row in reader:
col_store.append(row[1])
with open('meow.csv', 'wt') as f:
csv_writer = csv.writer(f)
csv_writer.writerows([col_store])
In your case if you have a column of single letters/numbers then Y.R answer will work.
To have a code that works in all cases, use this.
with open('meow.csv', 'wt') as f:
csv_writer = csv.writer(f)
csv_writer.writerows(([_] for _ in col_store))
From here it is mentioned that writerows expect an an iterable of row objects. Every row object should be an iterable of strings or numbers for Writer objects
The problem is that you are using 'writerows' treating 'col_store' as a list with one item.
The simplest approach to fixing this is calling
csv_writer.writerows(col_store)
# instead of
csv_writer.writerows([col_store])
However, this will lead to a probably unwanted result - having blank lines between the lines.
To solve this, use:
with open('meow.csv', 'wt', newline='') as f:
csv_writer = csv.writer(f)
csv_writer.writerows(col_store)
For more about this, see CSV file written with Python has blank lines between each row
Note: writerows expects 'an iterable of row objects' and 'row objects must be an interable of strings or numbers'.
(https://docs.python.org/3/library/csv.html)
Therefore, in the generic case (trying to write integers for examlpe), you should use Sam's solution.
Brand new to Python and programming. I have a function that extracts a file creation date from .csv files (the date is included the file naming convention):
def get_filename_dates(self):
"""Extract date from filename and place it into a list"""
for filename in self.file_list:
try:
date = re.search("([0-9]{2}[0-9]{2}[0-9]{2})",
filename).group(0)
self.file_dates.append(date)
self.file_dates.sort()
except AttributeError:
print("The following files have naming issues that prevented "
"date extraction:")
print(f"\t{filename}")
return self.file_dates
The data within these files are brought into a DataFrame:
def create_df(self):
"""Create DataFrame from list of files"""
for i in range(0, len(self.file_dates)):
self.agg_data = pd.read_csv(self.file_list[i])
self.agg_data.insert(9, 'trade_date', self.file_dates[i],
allow_duplicates=False)
return self.agg_data
As each file in file_list is worked with, I need to insert its corresponding date into a new column (trade_date).
As written here, the value of the last index in the list returned by get_filename_dates() is duplicated into every row of the trade_date column. -- presumably because read_csv() opens and closes each file before the next line.
My questions:
Is there an advantage to inserting data into the csv file using with open() vs. trying to match each file and corresponding date while iterating through files to create the DataFrame?
If there is no advantage to with open(), is there a different Pandas method that would allow me to manipulate the data as the DataFrame is created? In addition to the data insertion, there's other clean-up that I need to do. As it stands, I wrote a separate function for the clean-up; it's not complex and would be great to run everything in this one function, if possible.
Hope this makes sense -- thank you
You could grab each csv as an intermediate dataframe, do whatever cleaning you need to do, and use pd.concat() to concatenate them all together as you go. Something like this:
def create_df(self):
self.agg_data = pd.DataFrame()
"""Create DataFrame from list of files"""
for i, date in enumerate(self.file_dates):
df_part = pd.read_csv(self.file_list[i])
df_part['trade_date'] = date
# --- Any other individual file level cleanup here ---
self.agg_data = pd.concat([self.agg_data, df_part], axis=0)
# --- Any aggregate-level cleanup here
return self.agg_data
It makes sense to do as much of the preprocessing/cleanup as possible on the aggregated level as you can.
I also went to the liberty of converting the for-loop to use the more pythonic enumerate
i am searching everywhere for a method to write a list of floats into csv but must be in column format.
My code for writing csv as follow:
csvfile=open('Test.csv','w', newline='')
obj=csv.writer(csvfile)
obj.writerow(list_dis_B1_avg)
csvfile.close()
It turn out that the floats are written in rows.
I have a list of floats stored under "list_dis_B1_avg"
How can i just write it in column?
You dont need any csv module to do that:
with open("Test.csv", "w") as f: # use with to close the file in any case
f.write("\n".join(list_dis_B1_avg)) # newline between the elements
More about the with keyword: https://www.geeksforgeeks.org/with-statement-in-python/
More about str.join(): https://www.programiz.com/python-programming/methods/string/join
so, me and a friend are working on a game built in python, and we're working on a leaderboard system, but having trouble getting the CSV file to order itself when called.
we were using a dedicated python file to attempt to work on opening the external file, but we cannot work out how to make it order.
import csv
import operator
with open('leaderboard.csv', 'r') as f:
csv1 = csv.reader(f, delimiter=',', quotechar='"')
for row in csv1:
if row:
some=row[1]
print(row)
our code looks like that.
Any help?
thanks.
Sort things in Python with the built-in sorted function or the list.sort method. csv.reader(source, ...) returns an iterator of lists of strings, which can be passed to sorted.
import csv
data = ('c cee', 'a aaa', 'z zee')
for row in sorted(csv.reader(data)):
print(row)
prints
['a aaa']
['c cee']
['z zee']
Pass a key function to sorted to sort the lists on a different field. See the sorted doc.
I have this in csv file:
Titre,a,b,c,d,e
01,jean,paul,,
01,,,jack,
02,jeanne,jack,,
02,,,jean
and i want :
Titre,a,b,c,d,e
01,jean,paul,jack,
02,jeanne,jack,,jean
can you help me ?
In general, a good approach is to read the csv file and iterate through the rows using Python's CSV module.
CSV will create an iterator that will let you loop through your file like this:
import csv
with open('your filename.csv', 'r') as infile:
reader = csv.reader(infile)
for line in reader:
for value in line:
# Do your thing
You're going to need to construct a new data set that has different properties. The requirements you described:
Ignore any empty cells
Any time you encounter a row that has a new index number, add a new row to your new data set
Any time you encounter a row that has an index number you've seen before, add it to the row that you already created (except for that index number value itself)
I'm not writing that part of the code for you because you need to learn and grow. It's a good task for a beginner.
Once you've constructed that data set, it will look like this:
example_processed_data = [["Titre","a","b","c","d","e"],
["01","jean","paul","jack"],
["02","jeanne","jack","","jean"]]
You can then create a CSV writer, and create your outfile by iterating over that data, similarly to how you iterated over the infile:
with open('outfile.csv', 'w') as outfile:
writer = csv.writer(outfile)
for line in example_processed_data:
writer.writerow(line)
print("Done! Wrote", len(example_processed_data), "lines to outfile.csv.")