I'm writing a code from python to query data from my postgres database, here is the code:
cur.execute(
"""SELECT id FROM reddit_tesla_title WHERE created_time like '{"2016-08%' """)
In my query, I will gradually increase the month ascendingly as to 2016-09, 2016-10...2017-01..2020-11..
I wonder if is there a way to insert a variable, say like:
year = 2016
month = 09
and in the for loop every time the year and the month increase by 1, but since this code is inside the cur.execute select statement, I'm not sure how to do it...any ideas? Thanks!
also, if is possible, can someone please let me know how to do the same thing for the output csv file's name.
df.to_csv('201610_tesla_cooments.csv', index=False)
Here is one way. It is a little tricky because you have curly braces in the base part of the query, so I just split it into two variables so that it was easy to use the Python format function:
https://docs.python.org/3.4/library/functions.html#format
Code:
query_base = """SELECT id FROM reddit_tesla_title WHERE created_time like '{"""
query_date_form = "{year}-{month:02}%'"
start_year = 2017
start_month = 11
end_year = 2019
end_month = 5
year = start_year
month = start_month
while True:
if year == end_year and month > end_month:
break
if month > 12:
year = year + 1
month = 1
continue
query_date = query_date_form.format(year=year, month=month)
query = query_base + query_date
print(f"Execute this: {query}")
month = month + 1
Execution:
Just update this with your code.
(venv) [ttucker#zim stackoverflow]$ python sql.py
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2017-11%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2017-12%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-01%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-02%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-03%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-04%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-05%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-06%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-07%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-08%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-09%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-10%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-11%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2018-12%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2019-01%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2019-02%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2019-03%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2019-04%'
Execute this: SELECT id FROM reddit_tesla_title WHERE created_time like '{2019-05%'
Related
I want to update the Delta table based on the condition it matches with another table.
below is my sql , and I want to convert it into delta update.
update emp set empid = (select emp_id from dept where empname="xyz")
where emp.deptid in (select dept_id from emp where emp_dept="IT")
I tried something like this with merge statement:
emp.merge(dept, "emp.dept_id = dept.merge_id")
.whenMatched()
.updateExpr(Map("emp_id" -> "dept.emp_id"))
.execute()
but now I'm stuck where I need to check if that emp_id having name as "xyz" and also wanted to check dept_id in where clause as well.
Any help would be appreciated
Given Table A with columns: ColA1, ColA2, ColA3
And a Table B with columns: ColB1
I want to restrict the data that can be returned from Table A based on data in Table B, like:
ColA1 not in ColB1
Ideally, some way incorporate SQL queries in the filter with select statements
What you want is
SELECT a.ColA1
, a.ColA2
, a.ColA3
FROM TableA a
LEFT OUTER JOIN TableB b on b.ColB1 = a.ColA1
WHERE b.ColB1 IS NULL
So...
Query1 contains ColA1, ColA2, and ColA3 from TableA.
Query2 contains ColB1 from TableB.
Query3
joins Query1 and Query2 on ColA1 1..1 = 0..1 ColB1
Data Items: ColA1, ColA2, ColA3
Filter: ColB1 IS NOT NULL
not exists is probably what you are looking for
Try something like this
select * from TableA as T1
where not exists
(select * from TableB as T2
where t1.key1 = t2.key1 and T1.key2 = t2.key2)
I got two tables below:
And I want to create a query to combine them like below:
But unsuccessfully I got something like this:
Some data for "value1" duplicated
How can I solve this?
Is there any function that can have "value1" for the first "no." only?
Thank you.
You can do this by using a subquery to prepare the data.
It seems you want only the rows with the lowest Sub no to join, so we'll first select that:
SELECT [No], Value2
FROM Table2 m
WHERE
EXISTS(
SELECT 1
FROM Table2 s
WHERE s.[No] = m.[No]
HAVING MIN(s.sub_no) = m.sub_no
)
Then, integrate this into your main query:
SELECT *
FROM Table1
INNER JOIN (
SELECT [No], Value2
FROM Table2 m
WHERE
EXISTS(
SELECT 1
FROM Table2 s
WHERE s.[No] = m.[No]
HAVING MIN(s.sub_no) = m.sub_no
)
) AS T2 ON T1.[No] = T2.[No]
I have a variable with column names in it, like below:
VAR=Fullname,Address,DOB
I need to pass this variable in my spark sql instead of column names like below:
spark.sql("""
Select
VAR
From mytable
where 1=1
#some additional filters
""")
so it is treated as if I was giving the column names explicitly like below:
spark.sql("""
Select
Fullname
,Address
,DOB
From mytable
where 1=1
#some additional filters
""")
how can this be implemented using pyspark/sparksql
Below is code sample in pyspark for same,
var = 'Fullname,Address,DOB'
query = "SELECT {} FROM From mytable WHERE 1=1 #additional filters".format(var)
spark.sql(query)
I'm working on a WinForms application with an Access database.
Please correct me.
Select *
from EMP
where Empid in
(Select id
from Test1
where pin=4
UNION
Select id
from Test2
where pin=4)
When I try to execute this on the Access database, it gave me this error:
Operations are not allowed in subquery
You can also use two standard SQL where conditions, as per below. I have tested this approach and it appears to work as expected
SELECT x, y, z
WHERE x NOT IN (SELECT x FROM b)
AND NOT IN (SELECT x FROM c)
I tried as below and its worked.
Select *
from EMP where Empid in (Select id from Test1 where pin=4 ) UNION
Select *
from EMP where Empid in(Select id from Test2 where pin=4)
Really late but this works
Select *
from EMP
where Empid in
(SELECT id FROM (
Select id from Test1 where pin=4
UNION Select id from Test2 where pin=4 ) )
How about something like
Select e.*
from EMP e INNER JOIN
(
Select id
from Test1
where pin=4
UNION
Select id
from Test2
where pin=4
) subSelect ON e.Empid = subSelect.id
as far as i know union operations are not allowed in subqueries,instead you can create a named union query and include in the subquery expression