I need to convert the YYYYMM to YYYY-MM-01 using azure data factory pipeline expression.
I tried the below expression but it giving me error that date value should follow the ISO 8601 format.
#formatDateTime(concat('202301','01'),'YYYY-MM-DD')
It should return in this format '2023-01-01'.
Thanks
I used multiple substring to get this as shown below, if any better answer please let me know.
#concat(substring(concat('202301','01'),0,4 ),'-',substring(concat('202301','01'),4,2),'-',substring(concat('202301','01'),6,2))
You can supply a format string including numbers to formatDateTime, for example:
#formatDateTime(utcnow(), 'yyyy-MM-01')
NB You have the case of your argument wrong, it should be lower-case y for Year, upper-case M for month and lower-case d for Day.
Instead of concatenating 202301 with 01 (three times) and then taking substring, you can use the below expression to achieve the same.
#concat(substring('202301',0,4 ),'-',substring('202301',4,2),'-','01')
Related
I have a requirement to fetch the quarter from a string field using hive version less than <1.3 so cant use quarter feature
sample data format in string field say abc--20210415
I tried doing it by using
CONCAT('T',(INT((MONTH(from_unixtime(unix_timestamp(abc,'yyyymmdd')))-1)/3)+1))
but its showing T1 instead it should show T2 (since april comes in second quarter 20210415) but this expression does not convert string to date before calculating quarter
So, i converted it to date using casting
cast(to_date(from_unixtime(unix_timestamp(date, 'yyyymmdd'))) as date)
it converted but output was 2021-01-15 and the same was converted to quarter using
CONCAT('T',(INT((MONTH(cast(to_date(from_unixtime(unix_timestamp(qc_creation_date, 'yyyymmdd'))) as date))-1)/3)+1))
Is there a way around to resolve this issue where in for ex: string 20210415 should give the output as T2 .Kindly help
Regards
i got the solution
CONCAT('Q',(INT((MONTH(cast(to_date(from_unixtime(unix_timestamp(abc, 'yyyyMMdd'))) as date))-1)/3)+1))
Thanks
Your date format yyyyMMdd can be efficiently converted to Hive format yyyy-MM-dd using regexp_replace function instead of unix_timestamp+from_unixtime functions which are using SimpleDateFormat. Bug in your query is wrong format template, it should be 'yyyyMMdd', see SimpleDateFormat for reference. But regexp_replace is much simpler, use unix_timestamp+from_unixtime only if not possible to do the same using regexp_replace.
Do not need to cast it to Date, string works fine.
For Hive < 1.3.0 extract month and use ceil(month/3.0):
select ceil(month(regexp_replace('20210415','(\\d{4})(\\d{2})(\\d{2})','$1-$2-$3'))/3.0) -- returns 2
Even simpler method:
select ceil(regexp_extract('20210415','^(\\d{4})(\\d{2})',2)/3) --returns 2
And since Hive 1.3.0 there is quarter(date/timestamp/string) function.
Demo:
quarter(regexp_replace('20210415','(\\d{4})(\\d{2})(\\d{2})','$1-$2-$3')); --Returns 2
I have a column in a database which is all postcodes. I want to use that column to get statistical data about specific regions. To do this, I want to extract just the first non numeric characters of the postcode (B for Birmingham, BT for Belfast).
I can see solutions in other SQL formats using a CASE WHEN with ISNUMERIC but that function doesn't work in Presto. Are there any solutions to this?
As always, any advice would be greatly appreciated.
Many thanks,
Barry
I think you'll want to look at using regular expression functions to either extract the non-numeric characters using regexp_extract or replace all non-numeric characters using regex_replace.
See https://prestodb.io/docs/current/functions/regexp.html
I managed to get around this using
select concat(substr(postcode,1,1), case when substr(postcode,2,1) in ('1','2','3','4','5','6','7','8','9','0') then '' else substr(postcode,2,1) end)
I am trying to convert date from '2019-12-12' to '2019/12/12' in my mapping dataflow.
But i cannot find dataflow expression which can convert to this format.
I want a function similar to formatDateTime() which is available in datafactory expression and not in dataflow expression.
Tried toDate() ->doesnt take yyyy/MM/dd
Tried toTimestamp() -> doesnt take yyyy/MM/dd
Your first conversion results in a timestamp which doesn't have a format. To output with your desired format use the below which wraps your code with an additional toString() to format as desired.
toString(toTimestamp(toString(byName('PostedTransactionDate')), 'yyyy-MM-dd') , 'yyyy/MM/dd')
You could using this expression:
toString(toDate('2019-12-12'), 'yyyy/MM/dd')
Result:
Hope this helps
I used this approach to get a date column format, Used derived column and used this the expression.
coalesce(toDate(YourDate,'MM/dd/yyyy'),toDate(YourDate,'yyyy/MM/dd'),toDate(YourDate,'dd/MM/yyyy'),toDate(YourDate,'MMddyyyy'),toDate(YourDate,'yyyyddMM'),toDate(YourDate,'MMddyyyy'),toDate(YourDate,'yyyyMMdd'))
I hope it works for you as well, Cheers
I have a text value that looks like this: 2019-03-25T06:05:00-07:00. The general format is yyyy-mm-ddThh:mm:ss-GMT. I don't care about the GMT part. I am trying to use this text field to make time series scatter plots in excel.
I want to convert it to a timestamp as simply as possible. I currently do this using a bunch of formulas:
Input: 2019-03-25T06:05:00-07:00
Extract parts of time individually: =value(mid(input_cell,12,2))
Use date() and time() to get timestamp types
Add them together per this answer: https://stackoverflow.com/a/41164517/11163122
Use custom formatting to get a timestamp value
Output: 3/25/2019 6:05:00 AM
In total this took me 8 cells and custom formatting. This is too complicated. What is a simpler/more elegant way to do this?
You can use:
=--REPLACE(LEFT(A1,19),11,1," ")
and format as desired
Turns out the timestamp type is ISO 8601: https://en.wikipedia.org/wiki/ISO_8601#Combined_date_and_time_representations
This led me to this answer: https://stackoverflow.com/a/26315881/11163122
Using the formula there, I found this to be a sufficient solution. If anyone out there has a better method, please speak up! :)
I am currently trying to convert yyyymmdd type of date to a ddmmyyy format.
I've tried using DATE function and something like this:
=DATE(LEFT(A3;4);MID(A3;5;3);RIGHT(A3;7))
Original date for this function is 20120401 but the formula returns: 16.12.2104.
I've tried using functions YEAR, MONTH and DAY instead of LEFT, MID and RIGHT but that's not the right way.
I also tried using DATEVALUE but since Excel probably doesn't recognize yyyymmdd as a date, it gives me a #VALUE! error.
I've found a couple of solutions for SQL but it isn't my strong side yet - and this should be achievable easily in Excel.
Is there a way to do this in Excel?
Applying =DATE(LEFT(A3;4);MID(A3;5;3);RIGHT(A3;7)) to 20120401 in A3 is effectively to reassemble the component characters as below on the left:
These effectively become converted as on the right.
The solution was simply to exclude the digits highlighted red with:
=DATE(LEFT(A3;4);MID(A3;5;2);RIGHT(A3;2))
but the locale might have been a factor and when dealing with date serial numbers and/or what appears to be a date but is a string various other issues might have been involved.
Consider:
=DATE(LEFT(A1,4),MID(A1,5,2),RIGHT(A1,2))