First Pandas Project
Starting to learn pandas and wanted to test with a dataset of my weightlifting which I exported as a CSV format. The purpose of this was to analyze my progression, but I have unfortunately run into an issue where my data rows are all stored in the same column and not splitting the data into the different columns which looks correct based on the imported header.
I've tried to add the seperator function while importing the csv, but looking at the data it needs to be "," that seperates the values (I guesss CSV always takes comma as default).
I am using the following code:
import pandas as pd
data = pd.read_csv("strong.csv")
Data from CSV looks like this:
Date,Workout Name,Exercise Name,Set Order,Weight,Reps,Distance,Seconds,Notes,Workout Notes
2018-05-08 19:27:54,"1: Back, Biceps & Abs","Deadlift (Barbell)",1,50,12,0,0,"",""
2018-05-08 19:27:54,"1: Back, Biceps & Abs","Deadlift (Barbell)",2,50,10,0,0,"",""
2018-05-08 19:27:54,"1: Back, Biceps & Abs","Deadlift (Barbell)",3,110,1,0,0,"",""
See image to see data.head() result:
( https://i.imgur.com/qQtw66S.png )
EDIT: See link to CSV file with first columns.
https://github.com/Trools/StrongProject
It seems there is an error in the CSV export.
I just tried creating a new file with the same data and suddently I dot an error about 11 lines instead of 10 existing in row 134. When going through the file, I found that one of the last data entries (weight) was stored as 72,5 instead of 72.5 which resulted in the issues of having an additional seperated value.
I am however a little confused why Pandas didnt give this error when trying to load the data in Jupyter notebook?
How would one go about an issue where a CSV export is not correctly formated?
Related
I've been struggling a while trying to solve the following issue.
I have some reports in .txt format that in the first 3 rows the metadata is placed, then, the report is displayed as a normal table.
Report Layout viewed in excel
Because of the layout of the report i have to use the following code to read the file
import pandas as pd
df=pd.read_csv('R1.txt', sep='delimiter',header=None)
I use sep='delimiter' because my .txt file is tabular separated file. getting the following:dataframe after reading
how can I take only the "dates" value and have it repeated along into a new column? having this i can erase the metadata rows so i can have the dataframe layout as a table. see example
desired dataframe layout
if your separator is a tabular you should use the following
import pandas as pd
df=pd.read_csv('R1.txt', sep='\t',header=None)
I´m struggling to open a text file using pandas.
I have this dataset from an experiment and it should be a 87x249 table.
However, when I use the pandas df.read_csv() command I get all the time a table which is 87x1. I tried to change the delimiters but I always get different tables of 87x1.
I tried to open the ascii file in excel and then save it as a csv. Then it worked and I got a nice table. But the point for me is to use the txt file directly.
Pandas DataFrames - how do I export list 'X' to a CSV so it appears as a string? The problem is when I open the CSV using Excel it appears in date format.
X=['1-4', '1-5', '2-3', '4-8']
ie. when list 'X' is exported to a CSV and opened with Excel it appears as a date:
I would like list 'X' to appear in Excel as is - that is, not converted it to date format.
Desired output for Excel is:
I have tried the following code - but it throws an error:
import pandas as pd
X=['1-4', '1-5', '2-3', '4-8']
Y=[1,4,3,5]
df=pd.DataFrame(list(zip(X,Y)))
column_names=['A','B']
df.columns=[column_names]
df.A.to_string()
df.to_csv('yyy.csv', mode='a', header=True)
Thankyou
worked fine with me...
maybe the excel or whatever program u use to open the file is casting it... try open it as text file...
Even if Excel reads in date format, when you open in pandas it will come in original format (at least in my case). If someone only wants to save data in csv and work in pandas again, it should be fine.
I also tried doing the 2nd option here (https://www.winhelponline.com/blog/stop-excel-convert-text-to-number-date-format-csv-file/) which transform the data as text. And then saving again. It worked for me.
My CSV file contains "" which ruins the file, when I import using Pandas, it considers that all columns as one value.
what I want to make is to change the following value in the column
4.7,3.2,1.3,.2,"Setosa"
to
4.7,3.2,1.3,.2,'Setosa'
Can't you use something like below?
string.replace('"', "'")
I have been trying to import an excel (xlsx) file into phpMyAdmin.
I have tried as both excel and csv file. I have tried csv and csv using load data.
I have replaced the default field termination value from ; to ,.
Most times I was getting an variety of error messages, so I deleted my field names column and then was able to import a single row of data only.
The data was off by a column, and I guess that has something to do with the structure of my table, which has a field for ID# as a primary auto-incrementing field which is not in my csv file.
I tried adding a column for that before importing with no success. I would have thought that I could import right from the xlsx file as that is one of the choices in phpMyAdmin but everything I read or watch online converts to csv.
I could use some help here.
I had a similar problem that I solved it by changing the 'fields enclosed by' option from " (double quote) to ' (single quote) and doing the same to the first line of the file which contains the field names. Worked like a charm. Hope this helps.
This is hopelessly late, but I'm replying in the hope that this might help a future viewer.
The reason that the CSV data is off by one is the very fact that you don't have the ID# field in it! The way to get around this is to import the file into a temporary table, then run
INSERT INTO `table`
SELECT NULL, <field1>, <field2>...
FROM `temp table`;
Adding NULL to the list of fields means that MySQL will autogenerate the ID# field (assuming you've set it to AUTO_INCREMENT).