Following query in SQLite Studio for a date in 2021 or 2022 returns all rows correspondingly:
select * FROM daily_reports
WHERE strftime('%Y', daily_reports.reporting_date) in ('2021','2022');
Same query in Python returns no rows and the where clause
created is ... where 1 = 0:
q = ...
q = q.filter((func.strftime('%Y', DailyReportItem.reporting_date) in ['2021','2022']))
How to make it work?
The problem is that the statement you passed into .filter() does not get converted into SQL, it is executed as false in Python right away and then passed down to .filter() as argument.
The way you do the IN operator in SQLAlchemy is via DailyReportItem.reporting_date.in_(['2021', '2022']).
So in your case it should be something like this: q.filter(func.strftime('%Y', DailyReportItem.reporting_date).in_(['2021', '2022']))
I would rather just check if date is bigger than the beginning of the year 2021 and lower than the end of 2022. This way you would also get a better use of indexes.
Related
I wanted to change time at date type which returning from mongodb with custom time like below
"2021-05-26T00:00:00.000Z"
to
"2021-05-26T10:20:00.000Z"
I wanted to change time from a variable at the date, so my technique was split this date with "T" then get time part and change it with custom time
let splitedTime = timev[0].validFrom.toString().split()[0];
let customTime = "10:20:00.000Z";
let finalTime = splitedTime + customTime;
but this split not working this giving me date like this "Wed May 26 2021 06:00:00 GM". Can you please help me for this?
Working with Date
Whilst I understand your logic of converting it to a string and then using string methods to convert it to your desired output, I believe a simpler approach is to use the Date object
function dateAdd(original, hours, minutes) {
const date = new Date(original);
date.setHours(original.getHours() + hours);
date.setMinutes(original.getMinutes() + minutes);
return date.toISOString();
}
When original = "2021-05-26T00:00:00.000Z" then the return value is "2021-05-26T10:20:00.000Z".
If you want a fixed time:
const date = new Date('2021-05-26T00:00:00.000Z');
date.setUTCHours(10);
date.setUTCMinutes(20);
date.setUTCSeconds(0);
date.setUTCMilliseconds(0);
// a cleaner approach:
date.setUTCHours(10, 20, 0); // hoursValue, minutesValue, secondsValue
console.log(date.toISOString());
Which produces the following:
"2021-05-26T10:20:00.000Z"
Another Solution
Your actual problem is being caused by the fact you call toString which returns a date string in the format of "Tue Aug 19 1975 23:15:30 GMT+0200 (CEST)" so when you're splitting by "T", that's way down at the end. toISOString will return the correct format.
Explanation
As you can see above, we avoid using string methods and use the methods that exist on Date. This approach is safer as you avoid issues with the difference between toISOString and toString. You may also find moment useful if you're using dynamic methods of changing dates regularly.
Note
In all honesty, I'm not entirely sure I understand the why behind what you're doing, so if I'm wrong please correct me so I can update my answer to be more relevant for you.
Learn More
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toString
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date
I'm using python 3.6 to do this. Below are just a few important columns that I'm interested to query out.
Auto-Gen Index : Product Container : Ship Date :.......
0 : Large Box : 2017-01-09:.......
1 : Large Box : 2012-07-15:.......
2 : Small Box : 2012-07-18:.......
3 : Large Box : 2012-07-31:.......
I would like to query rows that indicate Large Box as their product container and the shipping date must be in the period of July in the year of 2012.
file_name = r'''Sample-Superstore-Subset-Excel.xlsx'''
df = read_excel(file_name, sheet_name = my_sheet)
lb = df.loc[df['Product Container'] == 'Large Box'] //Get large box
july = lb[(lb['Ship Date'] > '2012-07-01') & (lb['Ship Date'] < '2012-07-31')]
I just wonder how to use query and where condition by python(pd.query())?
If your question is when to use loc vs where, see my answer here:
Think of loc as a filter - give me only the parts of the df that
conform to a condition.
where originally comes from numpy. It runs over an array and checks if
each element fits a condition. So it gives you back the entire array,
with a result or NaN. A nice feature of where is that you can also get
back something different, e.g. df2 = df.where(df['Goals']>10,
other='0'), to replace values that don't meet the condition with 0.
If you are asking when to use query, AFAIK there is no real reason to do besides performance. If you have a very large dataset, query is expected to be faster. More on high-level performance here.
I am using 2.22.1 to format dates.
When i put in a date that comes from a date selector, moment will put the date back one day. For example, when I try to format a date in the following way:
Example 1:
const day = new Date(value).getDate();
const month = new Date(value).getMonth();
const fullYear = new Date(value).getFullYear();
console.log('day', day); // 7
console.log('month', month); // 5
console.log('fullYear', fullYear); //2018
Formatting function:
moment.utc(new Date(month + ' ' + day + ' ' + fullYear), 'MM-DD-YYYY').format('DD-MM-YY')
Output: 06-05-18
Expected: 07-05-18
Example 2:
Input: Thu May 31 2018 00:00:00 GMT+0100 (GMT Summer Time)
Formatting function:
moment.utc(new Date(value)).format('DD-MM-YY')
Output: 30-05-18
Expected: 31-05-18
moment.utc will convert the input to universal time.
You are running new Date(...) with an input that's based on GMT +1.
If you think about this, it makes total sense.
Your output is 30-05-18, because it's 11 PM / 23:00 o'clock on the previous day.
This (how ever) would in fact work:
moment('05-06-2018', 'MM-DD-YYYY').format('DD-MM-YY')
// alternatively (and preferrably) this:
moment.utc('05-06-2018', 'MM-DD-YYYY').format('DD-MM-YY')
and output: "06-05-18"
Because the non utc version does not take a time input in this example.
One of the reasons moment.js exists, is to get rid of Date in your code all together.
(Keep in mind tho, that Date has drastically improved now. DateTimeFormat is a game changer)
Please just read the momentjs documentation on how to properly use moment.
edit:
If you want to process 400000 dates with this, I'd advise using RegExp, .split, .exec, .slice or Date instead.
(I can relate since I wrote a client sided Log parser with javascript generators and Service Workers for a statistical anti-cheat analysis)
I truly recommend playing around with such things to raise your knowledge.
I just ran into this issue and a quick fix I found for the time being processed in "Zulu time" (due to the Z at the end of the string) is to add .LocaleString() after the date variable.
I find that for data consistency, it's easier to store data in UTC time and then convert it to the locale string when displaying the data to the user.
For example, I'm using:
moment.utc(dateVariable.toLocaleString()).format("MM/DD/YYYY")
Please Help! This has been driving me nuts. YES I have looked at dozens of Stack Overflow answers for similar questions, and they all say "Your dates should be in a consistent accepted format".
All of my SQLite dates are stored in the following string format:
'yyyy-MM-dd HH:mm:ss'
I'm trying to do date comparison but NOTHING works. For instance:
SELECT transactiondate
FROM transactions
WHERE transactiondate >= '2010-02-19 00:00:00'
Doesn't work.. I get 0 records returned.
Even though when I run it without the WHERE clause I get returned date strings like:
'2017-02-25 00:00:00'
In fact, simply doing:
SELECT transactiondate
FROM transactions
WHERE '2017-02-25 00:00:00' >= '2010-02-19 00:00:00'
Does work, i.e. it compares the dates successfully in its raw form but not when referring to the table column value.
PLEASE.. WTF am I doing wrong?
Also, I've tried every permutation of datetime(...), date(...) in the SQL to no avail. I have also tried using ... BETWEEN ... AND ..., it doesn't help.
Thanks in advance if anyone can figure this out.
You got the ' in the front and end included in your data. In column transactiondate.
Then you also need to include ' chars in your WHERE filter too. Two ways to do that in SQLite:
select * from transactions where transactiondate >= "'2010-02-19 00:00:00'";
select * from transactions where transactiondate >= '''2010-02-19 00:00:00''';
However, I suggest you clean up your data instead:
update transactions
set transactiondate=substr(transactiondate,2,length(transactiondate)-2)
where transactiondate like "'%'";
Now you can use your original WHERE without the extra ' chars.
OK, This is not an answer. But in SO you can't do this kind of thing in a comment.
I can't replicate the condition you have. The code, in particular the SELECT, works well enough executed from Python.
>>> import sqlite3
>>> conn = sqlite3.connect(':memory:')
>>> cur = conn.cursor()
>>> cur.execute('CREATE TABLE transactions (transactionsdate string)')
<sqlite3.Cursor object at 0x000000000531CCE0>
>>> cur.execute('''INSERT INTO transactions VALUES ('2010-02-10 00:00:00')''')
<sqlite3.Cursor object at 0x000000000531CCE0>
>>> cur.execute('''INSERT INTO transactions VALUES ('2017-02-10 00:00:00')''')
<sqlite3.Cursor object at 0x000000000531CCE0>
>>> list(cur.execute("SELECT * from transactions where transactionsdate >= '2010-02-19 00:00:00'"))
[('2017-02-10 00:00:00',)]
>>> list(cur.execute("SELECT * from transactions"))
[('2010-02-10 00:00:00',), ('2017-02-10 00:00:00',)]
Stab in the dark: Can you dump the contents of the field into a text file and examine it using (say) Notepad++, which would enable to to see non-printing characters that are interfering?
I have 2 year-month prompts. If I don't select any year-month in 1st prompt, report should by default, run from January of the same year, selected in 2nd prompt. My prompts are value prompts and have string values. Please help me materialise the requirement. I have already tried # prompt macro, ?prompt?, case when etc. I am nto sure, If javascript would help.
I'm going to assume your underlying date fields are not stored as DATE value types since you're using strings. This may be easier split into 4 prompts: from month, from year, to month, to year.
The filter would then be an implied if:
(
(?FROM_YEAR? = '' or ?FROM_MONTH? = '') and
[database_from_month] = '01' and
[database_from_year] = ?TO_YEAR? and
[database_to_month] = ?TO_MONTH? and
[database_to_year] = ?TO_YEAR?
)
OR
(
(?FROM_YEAR? <> '' or ?FROM_MONTH? <> '') and
[database_from_month] = ?FROM_MONTH? and
[database_from_year] = ?FROM_YEAR? and
[database_to_month] = ?TO_MONTH? and
[database_to_year] = ?TO_YEAR?
)
The above style filter is superior for many reasons:
More likely to be sargeable
Easy to understand
Uses simple built-in Cognos functions; more likely to be cross-version compliant
No issues with cross-browser support you would get with Javascript
Code snippet would work in other Cognos studios (Business Insight, etc)
You've likely seen CASE statements in filters throws an error. The CASE statement is passed to SQL, not compiled into a SQL statement via Cognos. Hence it's not seen as proper syntax.