I'm trying to compare just the date of a column but because the query below is also comparing the time I get no result set back.
How could I write this same LINQ query if I'm only interested in the actual date value matching?
The column "ImportDate" has a value that looks similar to this 2009-08-30 12:26:00
from t in UnitsOfWork _
where t.ImportDate = "08/30/2009" _
select t
You can compare it as a string or you can use just the Date property on ImportDate
where t.ImportDate.ToString("yyyyMMdd") == "20090830"
or
where t.ImportDate.Date == new DateTime(2009,8,30)
Related
Source date fields coming to my flow looks like a regular date in YYYY-MM-DD format. I'd like to use Conditional split operator to validate if a particular column can be converted to date column.
I was trying to use something like this:
isNull(toDate($column_list[3])) == false()
But it does not work as expected -> it rejects every single row....
Can you help?
Per my experience, if the coming data(we consider it's a string data in default) is not a valid date string, the result of toDate() function will return NULL.
Ref this example:
As you said , the coming data are regular date in YYYY-MM-DD format, it's a valid date data(string) and that will cause the expression isNull(toDate($column_list[3])) always return false(not null), then false()==false() also will always return true.
The expression will always return true, that why it doesn't work ad expected.
I essentially just want to create a date column that represents the date of the filename.
My table filesInDir is just a single column and 4 rows called filepath:
":..\..\code\products\Q\ExtData\CIBC\availability\Global\EquityOnly\daily\bnyMellon_inventory\push_list_20190314_040253_Equity.csv"
":..\..\code\products\Q\ExtData\CIBC\availability\Global\EquityOnly\daily\bnyMellon_inventory\push_list_20190314_040306_Equity.csv"
":..\..\code\products\Q\ExtData\CIBC\availability\Global\EquityOnly\daily\bnyMellon_inventory\push_list_20190311_040321_Bond.csv"
":..\..\code\products\Q\ExtData\CIBC\availability\Global\EquityOnly\daily\bnyMellon_inventory\push_list_20190312_999999_Cash.csv"
I also have a list of possible dates, 2019.03.12 2019.03.11 2019.03.14. How can match the list of dates with the rows of the table above so that I can get a new column that specifies that date value that matched with the filepath string.
If all your file paths follow the same format as your example ones,you can create a date column pretty easily doing:
update date:"D"$8#'103_'filePaths from filesInDir
Then match with your dates using this column.
This is how you parse the filePath to get the date, Please note as "\" is the escape sign so you need to write as "\\" instead that or directly retrieve the string from system command.
Created table
filesInDir:([]filePaths:(":..\\..\\code\\products\\Q\\ExtData\\CIBC\\availability\\Global\\EquityOnly\\daily\\bnyMellon_inventory\\push_list_20190314_040253_Equity.csv";
":..\\..\\code\\products\\Q\\ExtData\\CIBC\\availability\\Global\\EquityOnly\\daily\\bnyMellon_inventory\\push_list_20190314_040306_Equity.csv";
":..\\..\\code\\products\\Q\\ExtData\\CIBC\\availability\\Global\\EquityOnly\\daily\\bnyMellon_inventory\\push_list_20190311_040321_Bond.csv";
":..\\..\\code\\products\\Q\\ExtData\\CIBC\\availability\\Global\\EquityOnly\\daily\\bnyMellon_inventory\\push_list_20190312_999999_Cash.csv"))
Create the date column
update date:{"D"$("_"vs last "\\" vs x)[2]} each filePaths from `filesInDir
I also have a list of possible dates, 2019.03.12 2019.03.11 2019.03.14. How can match the list of dates with the rows of the table above so that I can get a new column that specifies that date value that matched with the filepath string.
Can you give some example illustration on that? Not clear about what you intend and expect to see
If you just want let say a "Flag" column indicating whether the date in the records match with the dateRange, you can simply use a in to match that.
dateRange:2019.03.12 2019.03.11 2019.03.14
update match:date in dateRange from `filesInDir
This will be the output:
Another approach using 0:
update date:raze(" D";"_")0:filePaths from filesInDir
It is dependent on the occurrences of _ in your filepaths
I have a Dataset with one column lastModified of type string with format "yyyy-MM-ddThh:mm:ss.SSS+0000" (sample data: 2018-08-17T19:58:46.000+0000).
I have to add a new column lastModif_mapped of type Timestamp by converting the lastModified's value to format "yyyy-MM-dd hh:mm:ss.SSS".
I tried the code below, but the new column is getting the value null in it:
Dataset<Row> filtered = null;
filtered = ds1.select(ds1.col("id"),ds1.col("lastmodified"))
.withColumn("lastModif_mapped", functions.unix_timestamp(ds1.col("lastmodified"), "yyyy-MM-dd HH:mm:ss.SSS").cast("timestamp")).alias("lastModif_mapped");
Where am I going wrong?
As I have answered in your original question, your input data String field didn't correspond to allowed formats of the unix_timestamp(Column s, String p):
If a string, the data must be in a format that can be cast to a timestamp, such as yyyy-MM-dd or yyyy-MM-dd HH:mm:ss.SSSS
For you case, you need to use to_timestamp(Column s, String fmt)
import static org.apache.spark.sql.functions.to_timestamp;
...
to_timestamp(ds1.col("lastmodified"), "yyyy-MM-dd'T'HH:mm:ss.SSSXXX")
And you don't need to cast explicitly to Timestamp since to_timestamp returns already Timestamp.
When you use withColumn("lastModif_mapped",...) you don't need to add alias("lastModif_mapped"), because withColumn would create a new column with the provided name.
I have the below set of records in Google Sheets. I would like to filter the rows with specific name and date range. Once I have the filtered data, I would like to fetch the last row's final amount cell data.
Ex: I would like to fetch final amount as 300 if my date(dd/mm/yyyy) range is 01/01/206 to 11/06/2016 and Name selection is 'Sandeep'.
As I have experience SQLite db, I have inserted the same records in DB and got the expected result using the below query.
select Final from MyTable where Date in (select max(Date) from MyTable WHERE Date BETWEEN '01/01/2016' AND '11/06/2016' and name = "Sandeep")
But I am not getting idea how to use multiple select statements in google sheets. It is ok for me to get result using any other way. So please help me to get the result as explained above.
= QUERY (A1:E50,"Select F where A > date '2016-1-1' and A < date '2016-6-11' and B ='Sandeep' order by A desc limit 1")
Use Column IDs A,B,C instead of name, income. Multiple columns can be given in a single Select clause separated by a ,
Dates in where clause should be written in yyyy-mm-dd format only(regardless of the format of dates in actual column)
See if this works
=index(E:E, max(filter(row(A:A), A:A>date(2016, 1, 1), A:A<date(2016, 6, 11), B:B="Sandeep")))
If you want to include start and end date, change > to >= and < to <=.
I am a beginner in cognos 10. I want to know how to get the value of a prompt from a Query1 and use it on Query2. My requirement is I want a prompt to ask the year for which I want the data in Query1 and in Query2 I want the data relevant to the previous year to that entered in the prompt. How to do that?
You can use the same parameter (tied to the prompt) in filters in both queries. If your parameter is Parameter1 and contains a four-digit year, and your data item in your filter is [Year] then your Query1 filter might look like this:
[Year] = ?Parameter1?
Your Query2 filter would be:
[Year] = ?Parameter1? - 1
Depending on your data source you may have to cast the string parameter to an integer before doing the subtraction though most SQL implementations will implicitly convert the string parameter to an integer for you.