When importing data from a text file (csv) into MS Access, I get an error "Type conversion failure" for 1 field. The field has data with date format "yyyy-mm-dd hh:nn:ss" and Access simply refuses to recognise it and places #Num! or simply blank data. The csv file is huge with 8m rows and cannot be opened in Excel to edit the date format. Facing no problems with any other fields.Anyway to avoid this error?
Use the Advanced... button at the field specification step of the import and try these settings:
I don't have the exact date format in the picture above, but it is just to show how to import that specific date.
Date Order should be YMD because in your dates, you have the years coming first, followed by the month and the date.
The date delimiter for your csv will be a dash -, while the time delimiter should be the default colon :. Make sure the 4 digit years checkbox is checked, and I would also check the Leading Zeros in Dates checkbox since your month and dates are in mm and dd formats respectively (i.e. they will begin with 0 if it is a single digit).
If there are problematic dates from your csv now, then this is another problem that won't be easy to tackle. You will maybe have to correct the date manually from the csv before importing it, or import the date as text and then create a new column to manipulate the text dates to date fields (and fix any problematic dates there).
Nothing wrong with the date format, but some records may be empty or have invalid entries.
Or you miss at the import to specify the separators and format for the date field.
If still no luck, link the file and specify text for the field. Then create a select query that uses the linked file as source and use CDate to convert the text date to true date values.
When done, change the query to an append or create table query to import your data.
Related
I had a data set of 8 millon rows in a txt file with tab delimited format without quotes.
I had 5 of the 14 columns with date values in dd.MM.yyyy format.
Problem 1
I am trying to import the file. In "Format your colums" step, if I choose the type of that colums as "date", it gives errors and all cells in columns turns "?"
So I selected "polynomial" and planed to convert attribute type to date later.
Problem 2 (the real one)
I imported the data and put "nominal to date" operator. When I run I got error in line 14.899:
Cannot parse date: Unparseable date: "0"
I find the line and I see that columns separated wrong. There was a tab character in a string in the a prior cell. So values moved one cell right. And this row was not the only one that moved.
I want to split the rows that has the values in wrong data type for spesified attributes. So I cant correct them manually.
How can I do that in Rapidminer?
Or any other ideas to figure theese problems out?
so most likely you need to adjust the date formatting in this pull-down menu:
To be honest, I usually just import as polynominal and then convert to date in my process. It's easier and reproducable.
You appear to have a broken input file.
The best solution, obviously, is to fix the process that generates the data. Espace or replace tab characters and format the date in a non-ambiguous format such as the ISO date format.
Assuming that you can't fix the date, you should probably write a robust parser program yourself. A generic parser such as rapidminer's won't be able to fix every problem.
I am trying to extract the date from a date timestamp in excel. I currently have a data file with a mixture of date formats including date only and date timestamps. This is causing me problems as I am importing the data into SAS and it cannot read both the date only and date timestamps under the same column.
I have tried in Excel converting the timestamp to a date using the following formula:
=DATEVALUE(DAY(E32) & "/" & MONTH(E32) & "/" & YEAR(E32))
This works in excel and converts the date so that they are all formatted the same and therefore gets around the issue of the timestamp. However when I import the data into SAS, I get null values if the day is greater than 12, i.e. it is reading the date as mm/dd/yyyy. For example:
Excel Date SAS Import Date
09/12/2016 09/12/2016
15/12/2016 #VALUE!
I tried to reformat this in excel using the following to see if it would get around the issue:
=DATEVALUE(MONTH(E32) & "/" & DAY(E32) & "/" & YEAR(E32))
But I then get the same SAS error in excel.
Can anyone help suggest a formula to use in excel that will get around this issue or advice on importing the data into SAS?
It sounds like your Excel data is in DMY format, but SAS is using MDY. You can check SAS by running the following code :
proc options option=datestyle;
run;
If it is MDY, then change it (and if you're in the UK ask your SAS admin to change the default setting)
option datestyle='DMY';
You can also check the locale value, which in the UK will be EN_GB. This value determines the datestyle value used when working with dates.
proc options option=locale;
run;
If you asked SAS to import from an XLSX file then it should be able to tell that the column contains dates, independent of which display format you have attached to the cells. Make sure that all of the cells in a single column are the same type of data and use the same display format.
CSV files are not Excel files and so there is no place to put a formula or any metadata about what type of data is in each column. If you use PROC IMPORT to read the CSV file then SAS will have to guess at what type of data each column in the CSV contains. If you are saving an Excel files as a CSV file for later reading into SAS or other software then you should format your date columns using yyyy/mm/dd format in Excel to prevent confusion that can be caused by different defaults for month and day order. Nobody uses YDM order.
Since a CSV file is just a text file if you want complete control over how SAS reads the date strings then just write the data step to read it yourself. You could run PROC IMPORT and then recall the code that it generates and modify it to read your data. You could read the string into a character variable and then write your own statements to convert it using say the INPUT() function.
If the column has some date values and some date time values then you could try using the ANYDTDTE informat to pull just the date part. That informat should properly handle 15/10/2016 even if your LOCALE settings are for US or other locations where dates are normally represented in MDY order and not DMY order.
If your dates are consistently in the DMY order then use DDMMYY informat to prevent the LOCALE setting from causing PROC IMPORT or ANYDTDTE informat to convert 12/10/2016 to December 10th instead of October 12th. But if your text file actually has some rows with dates in month first order and others in day first order then you will really need some extra information to properly tell the difference between December 10th and October 12th.
Program: Excel 2010
Issue: External report gives me a text string, I want to convert it to Date & 24 hour time.
141221205535 needs to be:
14/12/21 20:55:35
I have tried text to columns making the result show: YMD
I have then tried using the format cells option: (custom) yy/mm/dd hh:mm:ss
* I receive a string of ######## in the Sample field in the formatting box.
I have tried also just using the date/time version, no seconds
1412212055 needs to be:
14/12/21 20:55
ideally, if possible:
20141421 2055 or 2014/14/21 20:55 I will accept
I can not alter the report as it comes through a 3rd party and I am pulling the date from a 'reference' field. The report, when downloaded, is in CSV and there is no existing formatting on the data.
I need to convert this as my sales report then needs to be sorted by date order.
Note: If I do use the seconds in the string, I get the weird looking scientific number. So i've had to reduce the digits, this isn't ideal, is there a way to avoid the scientific number whilst the process of changing 'text to date' is happening?
Thanks in advance.
I used this formula to convert:
=TEXT(A1,"2000-00-00 00\:00\:00")+0
format result cell in required format, e.g. yyyy/mm/dd hh:mm:ss
If you don't want the seconds in the underlying value you can use this version
=TEXT(LEFT(A1,10),"2000-00-00 00\:00")+0
These formulas work because TEXT function converts your value 141221205535 to a text value 2014-12-21 20:55:35, then, because that is a valid date/time format in Excel the +0 "co-erces" the text value to a serial number representing the correct date/time in Excel.....so you can then simply format the result to display any way you want.
Note that I'm assuming all dates will be in the current (21st) century.....
A formula like this would work -
=TEXT(DATE(LEFT(A1,2),MID(A1,3,2),MID(A1,5,2))+TIME(MID(A1,7,2),MID(A1,9,2),MID(A1,11,2)),"yy/mm/dd hh:mm:ss")
If your format always has 12 digits
You'll need to add the datevalue with the timevalue to get also the HMS
for example (suppose the text is in D5 cell):
=DATEVALUE(LEFT(D5,2)&"-"&MID(D5,3,2)&-MID(D5,5,2))+TIMEVALUE(MID(D5,7,2)&":"&MID(D5,9,2)&":"&RIGHT(D5,2))`
I have an excel file, with a date column, but I want to convert the date column to
YY/MM/DD/Time
Ive been searching for 2 hours and no result yet.
This is my data:
Source Data: http://i.stack.imgur.com/75zbS.jpg
Expected Output: YY/MM/DD/Time
Can someone help me how I can do it? I want to insert it into postgresql and I want to change everything to compatible date format.
EDIT: I have tried Right Click -> Format cells -> date but it does not change anything!
Thanks
You could use this method and split the date and time into separate cells:
=DATE((LEFT(A1,4)),(MID(A1,5,2)),MID(A1,7,2))
=TIME(MID(A1,10,2),(MID(A1,12,2)),0)
Once your date value is in a format Excel can recognize, you can then change the formatting to whatever you'd like.
Or if you don't care to have the value in a recognizable date format, you can just get your desired formatting like this (will give you a string that looks like this: YY/MM/DD/Time):
=MID(A1,3,2)&"/"&MID(A1,5,2)&"/"&MID(A1,7,2)&"/"&MID(A1,10,4)
ISO 8601 format would be YYYY-MM-DD H24:MI:SS.
But you can set Postgres to accept various date styles by setting the datestyle setting. You can do that globally in postgresql.conf or temporarily for your session.
SET datestyle = SQL, DMY
For more exotic formats, you can create a temporary staging table, COPY to it and INSERT into your target table from there. Among others, you can use to_timestamp():
SELECT to_timestamp('13/10/14/17:33', 'YY/MM/DD/hh24:mi')
More info and example code in related answers like these:
Replacing whitespace with sed in a CSV (to use w/ postgres copy command)
How to bulk insert only new rows in PostreSQL
Your going to have to parse the date into four columns using fixed parsing.
Then reassemble the columns any way you want.
Just Google with excel parse columns fixed.
When I am importing the excel file in date format(dd/mm/yyyy) to access it is showing non date value what is the problem.
eg:27/3/2012 to 33765 here 33765 is a non date value
The reason is because all dates within the MS Office world are actually numeric values, e.g the date you gave above 27/03/2012 equates to 40995.
When using the import wizard to create a new table, Access will automatically read this as it's true numeric value and set the column type as so.
Fortunately it's easy to fix, open up the design view of your table, amend the column in question to a date format and save the table, this should now present all the values correctly.
Alternatively, setup a blank table with the column already set to the date data type and then import your spreadsheet into the existing table.
Access by default thinks that dates coming from Excel are in American date format (mm/dd/yyyy) not European (dd/mm/yyyy).
The number you are seeing is how excel is storing the dates. (every day since 1/1/1990 is +1).
When the dates are in european format Access doesnt recognize them as dates during import and may load their raw serial date numbers.
Additionally be sure that the Access column that these are importing to is set as 'date'. I figured that you may have already done that, but I wanted to check.