I'm able to get the current timestamp using now() in NIFI. But I want to get the yesterday's timestamp in the below format and could not achieve it. what expression should be given for getting yesterday's timestamp at NIFI processor.
Egg: "superDate":"2021-03-07 23:59:00"
Thanks in advance!
${now():toNumber():minus(86400000):format('yyyy-MM-dd HH:mm:ss')}
Related
Currently having a field with the below date format.
3/2/2021 18:48
I need to convert it to 2021-03-02. I tried taking a substring and converting to date format. But it is not providing the desired output. Any suggestions will be helpful
Below if you are using spark SQL:
from_unixtime(unix_timestamp('3/2/2021 18:48', 'M/d/yyyy'), 'yyyy-MM-dd')
Same functions are available in Dataframe API as well:
https://spark.apache.org/docs/2.4.0/api/sql/index.html#from_unixtime
https://spark.apache.org/docs/2.4.0/api/sql/index.html#unix_timestamp
Does anyone know this tool (DataStax Bulk Loader)? I'd like to change date format in some rows from 2020-05-18T14:18:45.878Z to 1593402243336 (like Instant Java type/epochMilliseconds) because of the error in code.
The date in this "column" in cassandra is of type text. Is it possible? I tried to create a proper script but without any success
No, all available options for time/date conversion are applied to the columns with time, date, or timestamp types. As I suggested in the previous answer, you need to create a column with timestamp type, unload from the text column, and load into the new column, and then unload into milliseconds format if it's necessary...
P.S. Although, can you add more information why you need so complex approach?
I am usig azure data factory and basically I need to add a dynamic content (date function) to do this:
SELECT DATEADD(DAY, -2, GETDATE())
any idea?
I guess you can do the below using Logical Function
#equals(formatDateTime(addDays(utcnow(),-2),'yyyy-MM-dd'),formatDateTime(activity('Your Metadata Activity Name').output.lastmodifieddate,'yyyy-MM-dd'))
You can use just addDays() function to get date differences. Detailed document on Azure Data Factory expressions and function is available : here
I used #{formatDateTime(convertfromutc(addDays(utcNow(),-6),'Eastern Standard Time'), 'yyyy-MM-dd' )} Here I wanted to get 6 days old date (from current date).
If Date contains timestamp then you should try below code.
#string(addhours('2022-08-03 06:20:51',-2, 'yyyy-MM-dd HH:mm'))
I'm trying to access Oracle table using cx_oracle module and convert that as a dataframe, every thing is fine except couple of date columns has date format like "01-JAN-01" Python considering it as datetime.datetime(1,1,1,0,0) and after creating dataframe it's showing as 0001-01-01 00:00:00. I am expecting output as 2001-01-01 00:00:00. Please help me on this. Thanks in advance.
You have a couple of choices. You could
* Retrieve it from the Oracle database with [read_sql](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_sql.html specifying the date in a format (TO_CHAR) more appropriate for the default date format of pandas
* Retrieve it from the database as a string (as above) and then convert it into a date in the pandas framework.
I've written a program that reads a file containing the date (in yyyy/MM/dd format) and uses the Datastax Java Driver to read the date and add it to a cassandra table.
So for instance, if my record contains a date value of '2010/06/01', then this date value gets converted into a date object (using the SimpleDateFormat class).
However, when I view the data (containing the date) in the database, I see that the date (which in the cassandra table is a timestamp type) shows the following:
2010-06-01 00:00:00+0100
The issue here is that I don't want the timestamp to have "+0100" (to indicate that this is british summer time), rather I'd want to store the date just as "2010-06-01 00:00:00+0000".
I've done the following to my program to try and 'ignore' the timezone by doing the following:
SimpleTimeZone tz = new SimpleTimeZone(0, "Out Timezone");
TimeZone.setDefault(tz);
String dateStringFromFile = "2010/06/01";
SimpleDateFormat sdf = new SimpleDateFormat("yyyy/MM/dd");
Date theDate = sdf.parse(dateStringFromFile);
...Now when I add debug statements to my program, I can see that the date shows "2010-06-01 00:00:00+0000" on my log file (this is right for me). However when i see the date stored in Cassandra, i still see that the date shows as
"2010-06-01 00:00:00+0100" and not "2010-06-01 00:00:00+0000".
Is there anything on the cassandra side that I would have to change or update to ignore the timezone (i.e. not put +0100 on the date and to put +0000), so that the timestamp shows as "2010-06-01 00:00:00+0000"?
Please note that I am running Cassandra 3.0.5 on a Docker VM (Centos linux), Java 8.
Any advice is appreciated.
Thanks.
This solution has been sorted out. Nothing to do with cqlshrc. Its to do with forcing the timezone as 0 (like the code in the original post) and getting the time in milliseconds and writing that to the database - (which is a timestamp type column)