I am trying to convert date from '2019-12-12' to '2019/12/12' in my mapping dataflow.
But i cannot find dataflow expression which can convert to this format.
I want a function similar to formatDateTime() which is available in datafactory expression and not in dataflow expression.
Tried toDate() ->doesnt take yyyy/MM/dd
Tried toTimestamp() -> doesnt take yyyy/MM/dd
Your first conversion results in a timestamp which doesn't have a format. To output with your desired format use the below which wraps your code with an additional toString() to format as desired.
toString(toTimestamp(toString(byName('PostedTransactionDate')), 'yyyy-MM-dd') , 'yyyy/MM/dd')
You could using this expression:
toString(toDate('2019-12-12'), 'yyyy/MM/dd')
Result:
Hope this helps
I used this approach to get a date column format, Used derived column and used this the expression.
coalesce(toDate(YourDate,'MM/dd/yyyy'),toDate(YourDate,'yyyy/MM/dd'),toDate(YourDate,'dd/MM/yyyy'),toDate(YourDate,'MMddyyyy'),toDate(YourDate,'yyyyddMM'),toDate(YourDate,'MMddyyyy'),toDate(YourDate,'yyyyMMdd'))
I hope it works for you as well, Cheers
Related
I need to convert the YYYYMM to YYYY-MM-01 using azure data factory pipeline expression.
I tried the below expression but it giving me error that date value should follow the ISO 8601 format.
#formatDateTime(concat('202301','01'),'YYYY-MM-DD')
It should return in this format '2023-01-01'.
Thanks
I used multiple substring to get this as shown below, if any better answer please let me know.
#concat(substring(concat('202301','01'),0,4 ),'-',substring(concat('202301','01'),4,2),'-',substring(concat('202301','01'),6,2))
You can supply a format string including numbers to formatDateTime, for example:
#formatDateTime(utcnow(), 'yyyy-MM-01')
NB You have the case of your argument wrong, it should be lower-case y for Year, upper-case M for month and lower-case d for Day.
Instead of concatenating 202301 with 01 (three times) and then taking substring, you can use the below expression to achieve the same.
#concat(substring('202301',0,4 ),'-',substring('202301',4,2),'-','01')
I have a requirement to fetch the quarter from a string field using hive version less than <1.3 so cant use quarter feature
sample data format in string field say abc--20210415
I tried doing it by using
CONCAT('T',(INT((MONTH(from_unixtime(unix_timestamp(abc,'yyyymmdd')))-1)/3)+1))
but its showing T1 instead it should show T2 (since april comes in second quarter 20210415) but this expression does not convert string to date before calculating quarter
So, i converted it to date using casting
cast(to_date(from_unixtime(unix_timestamp(date, 'yyyymmdd'))) as date)
it converted but output was 2021-01-15 and the same was converted to quarter using
CONCAT('T',(INT((MONTH(cast(to_date(from_unixtime(unix_timestamp(qc_creation_date, 'yyyymmdd'))) as date))-1)/3)+1))
Is there a way around to resolve this issue where in for ex: string 20210415 should give the output as T2 .Kindly help
Regards
i got the solution
CONCAT('Q',(INT((MONTH(cast(to_date(from_unixtime(unix_timestamp(abc, 'yyyyMMdd'))) as date))-1)/3)+1))
Thanks
Your date format yyyyMMdd can be efficiently converted to Hive format yyyy-MM-dd using regexp_replace function instead of unix_timestamp+from_unixtime functions which are using SimpleDateFormat. Bug in your query is wrong format template, it should be 'yyyyMMdd', see SimpleDateFormat for reference. But regexp_replace is much simpler, use unix_timestamp+from_unixtime only if not possible to do the same using regexp_replace.
Do not need to cast it to Date, string works fine.
For Hive < 1.3.0 extract month and use ceil(month/3.0):
select ceil(month(regexp_replace('20210415','(\\d{4})(\\d{2})(\\d{2})','$1-$2-$3'))/3.0) -- returns 2
Even simpler method:
select ceil(regexp_extract('20210415','^(\\d{4})(\\d{2})',2)/3) --returns 2
And since Hive 1.3.0 there is quarter(date/timestamp/string) function.
Demo:
quarter(regexp_replace('20210415','(\\d{4})(\\d{2})(\\d{2})','$1-$2-$3')); --Returns 2
I have a text value that looks like this: 2019-03-25T06:05:00-07:00. The general format is yyyy-mm-ddThh:mm:ss-GMT. I don't care about the GMT part. I am trying to use this text field to make time series scatter plots in excel.
I want to convert it to a timestamp as simply as possible. I currently do this using a bunch of formulas:
Input: 2019-03-25T06:05:00-07:00
Extract parts of time individually: =value(mid(input_cell,12,2))
Use date() and time() to get timestamp types
Add them together per this answer: https://stackoverflow.com/a/41164517/11163122
Use custom formatting to get a timestamp value
Output: 3/25/2019 6:05:00 AM
In total this took me 8 cells and custom formatting. This is too complicated. What is a simpler/more elegant way to do this?
You can use:
=--REPLACE(LEFT(A1,19),11,1," ")
and format as desired
Turns out the timestamp type is ISO 8601: https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations
This led me to this answer: https://stackoverflow.com/a/26315881/11163122
Using the formula there, I found this to be a sufficient solution. If anyone out there has a better method, please speak up! :)
Date stored as a string in Access. I'm not able to apply the suggested solutions such as DateValue and Cdate. I tried to reformat to add a delimiter but it gives me a mismatched error. The data is stored as such: "201704100705"
You need to cast it to a datetime value first:
DateSerial(Left(MyField,4), Mid(MyField,5,2), Mid(MyField,8,2)) + TimeSerial(Mid(MyField,10,2), Right(MyField,2))
After that, you can use the usual date/time functions.
I have a column which is a date in the following format : 2018-04-28.
I try to convert it in Impala but it seems there is no way it lets me operate on it at all. I know it doesn't support date formats but it won't even let me convert it.
I've tried to use things like the following but nothing works :
SELECT unix_timestamp(cast(t1.`date` as string),'yyyy-MM-dd') FROM example
Even cast(t1.dateas string) won't work. I always get the same error :
AnalysisException: Unsupported type 'DATE' in 't1.`date`'.
Would anyone know a way to convert this ?
Thanks,
First use DATE_FORMAT to convert datetime
if you use impala please set your date-column type to timestamp impala support timestamp much well than date . the time stamp String format should be like yyyy-mm-dd hh:mm:ss
I tried your SQL it works well when change date to timeStamp
the date type might be a bug in impala.