I have one Python file that helps me pull and organize datetime data from a server into a pandas DataFrame, and then export out into a nice and usable CSV. My second Python script needs to read the CSV and analyze the data. If I do not touch the exported CSV, the analysis can read the CSV and runs smoothly. However, as soon as I try to merge a few CSV files together with Excel/other spreadsheet software, I get a datetime error
ValueError("time data %r does not match format %r" %
ValueError: time data '2019-12-26 23:00' does not match format '%Y-%m-%d %H:%M:%S'
Even though, it is a direct copy/paste and still saved as a CSV. Any guru can provide some insight on this matter?
Pretty sure this is an Excel issue, not a Python problem.
If you load a .csv that has timestamp strings into Excel, Excel recognizes the datetimes - and formats them. This format seems to default to MM.DD.YYYY hh:mm (the date component might be different depending on your locale):
If you save the file in Excel, the seconds are removed in the .csv!
The only procedure that seems to reliably prevent this behavior is to set a specific date/time format for the respective column, e.g. DD.MM.YYYY hh:mm:ss. AFAIK, You'll have to do this manually for each workbook
Or perhaps write a macro. In older Excel versions, I had a PERSONAL.XLSB for that; should still work with newer versions, you'll have to put it in C:\Users\[username]\AppData\Roaming\Microsoft\Excel\XLSTART
you can use .xlsx format instead of .csv, presumably you won't loose format information there
I see from the error that not all your CSV files have the same time format.
Some on them are in [hour:minute] and some in [hour:minute:second].
ValueError: time data '2019-12-26 23:00' does not match format '%Y-%m-%d %H:%M:%S'
Make sure that datetime fields in all of your CSV matched the same format.
Related
Pandas DataFrames - how do I export list 'X' to a CSV so it appears as a string? The problem is when I open the CSV using Excel it appears in date format.
X=['1-4', '1-5', '2-3', '4-8']
ie. when list 'X' is exported to a CSV and opened with Excel it appears as a date:
I would like list 'X' to appear in Excel as is - that is, not converted it to date format.
Desired output for Excel is:
I have tried the following code - but it throws an error:
import pandas as pd
X=['1-4', '1-5', '2-3', '4-8']
Y=[1,4,3,5]
df=pd.DataFrame(list(zip(X,Y)))
column_names=['A','B']
df.columns=[column_names]
df.A.to_string()
df.to_csv('yyy.csv', mode='a', header=True)
Thankyou
worked fine with me...
maybe the excel or whatever program u use to open the file is casting it... try open it as text file...
Even if Excel reads in date format, when you open in pandas it will come in original format (at least in my case). If someone only wants to save data in csv and work in pandas again, it should be fine.
I also tried doing the 2nd option here (https://www.winhelponline.com/blog/stop-excel-convert-text-to-number-date-format-csv-file/) which transform the data as text. And then saving again. It worked for me.
experts!
I've got two similar Excel files (xlsm) as templates. Both have sheets with Date column.
Visible format for both files when use Excel is "10-Aug-20".
But when I read these files with SSIS process with Script Component Source using Microsoft.ACE.OLEDB.12.0 with "IMEX=1"... ta-da... some I see as expected, but some are 10.08.2020 00:00:00
This causes me a lot of pain because I will process files from both US (MM/dd/yyyy) and German (dd.MM.yyyy) date formats and would like to have locale-independent date format to process dates same way.
How can I force excel to give or ssis to read a correct date format.
Any suggestion how to see both files same programmaticly is most wanted and highly appreciated!
You could try to use the script component (as Transformation) to transform the data by using DateTime.TryParseExact
string dateString = "10-Aug-20";
string format = "dd-MMM-yy";
DateTime dateTime;
if (DateTime.TryParseExact(dateString, format, "en-US",DateTimeStyles.None, out dateTime))//if (DateTime.TryParseExact(dateString, format, "de-DE",DateTimeStyles.None, out dateTime))
{
Console.WriteLine(dateTime);
}
having trouble with date formats in CSV file format(Excel)
I want the date in "YYYY-MM-DD hh:mm " format and I need it in .CSV format.
When reopening the file, the date format gets changed and it looks like "MM-DD-YYYY hh:mm".
Does anyboy know how I can retain my desired date format when saving the file as .CSV?
Thank you!
Use the format YYYYMMDD in all CSV files, which doesn't convert to date in Excel or you can do one of the below things
Use a Custom format, rather than one of the pre-selected Date formats, the export to CSV will keep your selected format. Otherwise it defaults back to the US format
Place an apostrophe in front of the date and it should export in the correct format. Just found it out for myself, I found this thread searching for an answer.
Change the date and time settings for your computer in the "short date" format under calendar settings. This will change the format for everything yyyy-mm-dd or however you want it to display; but remember it will look like that even for files saved on your computer.
Surely someone before me has needed to produce a year, month, day in a single field for a CSV that "just works" in popular versions of Microsoft Excel? I want only a date, no timestamp, though I suppose I could include 00:00 or something like that if I absolutely had to.
Panagiotis Kanavos points out that "Excel can only import it and try to guess whether the text values correspond to a certain type, using the user's locale settings." My question is about what format will cause Excel to guess correctly in the US and Europe, and ideally everywhere else.
If it's impossible or unreliable to do this in CSV, I will accept a link to using some zipped XML format or something that Excel and other spreadsheets accept universally instead of CSV.
This is NOT a duplicate of of the following:
Best timestamp format for CSV/Excel? because I want it without a timestamp.
What are the "standard unambiguous date" formats? because I need CSV specifically for Excel to read.
Excel CSV date format because I need Excel import from CSV, not export.
Read Date Format in PHP EXCEL because I need Microsoft Excel, not PHP Excel.
Excel will recognize YYYY-MM-DD as a global standard.
Cartoon from: https://www.xkcd.com/1179/
The difficulty with answering your question is that to test the proposed answer, the format must be tested in "all popular versions" of Excel
I have several versions of Excel and in my testing this:
worked in all my versions (English-US Locale)
All,
I have a script that runs and when it does it saves the dates in a csv file by appending the file every time it runs.
The code shown below
with open('historyWeekly.csv','a',newline='') as f:
writer=csv.writer(f)
writer.writerow([todayScript])
the name of the csv file is 'historyWeekly.csv' and the format of todayScript is '06-05-2018'
The date is saved succesfully. However when I open the csv file I do find that the format has changed to 06/05/2018.
Using python, how can I force the format to stay the same?
or how can I read the csv with the correct format ('06-05-2018' instead of 06/05/2018)
Many Thanks
proposed solution to my question is to read the csv in python and convert the data back to the desired format.
pd.read_csv("path/historyWeekly.csv")
initial data:
Dates
0 06/05/2018
history['Dates']=pd.to_datetime(history['Dates'], format="%m/%d/%Y")
history['Dates']=history['Dates'].dt.strftime('%m-%d-%Y')
Dates
0 06-05-2018