I currently have the following query:
query = self.session.query(Student, School).join(
Person.student, aliased=True).join(
Student.school, aliased=True).filter(
Person.id == 1)
Which compiles to this SQL.
SELECT student.id AS student_id, student.school_id AS student_school_id, student.person_id AS student_person_id, school.id AS school_id, school.name AS school_name
FROM student, school, person JOIN student AS student_1 ON person.id = student_1.person_id JOIN school AS school_1 ON school_1.id = student_1.school_id
WHERE person.id = :id_1
I want the query to remain exactly as it is but I want the from statement to be exclusively from the Person model. So something like
SELECT * FROM person JOIN ... WHERE person.id = :id_1
I think the aliased kwarg is messing up the from condition. Removing the aliased kwarg fixes the behavior but I need the aliased kwarg for special use cases. How can I remove the student and school tables from the "FROM" statement.
The aliased argument to .join uses anonymous aliasing, meaning the Student and School you pass to session.query are different 'instances' of the table.
from sqlalchemy.orm import aliased
aliased_student = aliased(Student)
aliased_school = aliased(School)
query = (
session.query(aliased_student, aliased_school)
.select_from(Person)
.join(aliased_student, Person.student)
.join(aliased_school, Student.school)
.filter(Person.id == 1))
Here you can see that you can tell .join which alias to use when joining to a relationship.
Related
What is the difference between filter with multiple arguments and chain filter in django?
As you can see in the generated SQL statements the difference is not the "OR" as some may suspect. It is how the WHERE and JOIN is placed.
Example1 (same joined table): from https://docs.djangoproject.com/en/dev/topics/db/queries/#spanning-multi-valued-relationships
Blog.objects.filter(
entry__headline__contains='Lennon',
entry__pub_date__year=2008)
This will give you all the Blogs that have one entry with both (entry__headline__contains='Lennon') AND (entry__pub_date__year=2008), which is what you would expect from this query.
Result:
Blog with {entry.headline: 'Life of Lennon', entry.pub_date: '2008'}
Example 2 (chained)
Blog.objects.filter(
entry__headline__contains='Lennon'
).filter(
entry__pub_date__year=2008)
This will cover all the results from Example 1, but it will generate slightly more result. Because it first filters all the blogs with (entry__headline__contains='Lennon') and then from the result filters (entry__pub_date__year=2008).
The difference is that it will also give you results like:
A single Blog with multiple entries
{entry.headline: '**Lennon**', entry.pub_date: 2000},
{entry.headline: 'Bill', entry.pub_date: **2008**}
When the first filter was evaluated the book is included because of the first entry (even though it has other entries that don't match). When the second filter is evaluated the book is included because of the second entry.
One table: But if the query doesn't involve joined tables like the example from Yuji and DTing. The result is same.
The case in which results of "multiple arguments filter-query" is different than "chained-filter-query", following:
Selecting referenced objects on the basis of referencing objects and relationship is one-to-many (or many-to-many).
Multiple filters:
Referenced.filter(referencing1_a=x, referencing1_b=y)
# same referencing model ^^ ^^
Chained filters:
Referenced.filter(referencing1_a=x).filter(referencing1_b=y)
Both queries can output different result:
If more then one
rows in referencing-modelReferencing1can refer to same row in
referenced-modelReferenced. This can be the case in Referenced:
Referencing1 have either 1:N (one to many) or N:M (many to many)
relation-ship.
Example:
Consider my application my_company has two models Employee and Dependent. An employee in my_company can have more than dependents(in other-words a dependent can be son/daughter of a single employee, while a employee can have more than one son/daughter).
Ehh, assuming like husband-wife both can't work in a my_company. I took 1:m example
So, Employee is referenced-model that can be referenced by more then Dependent that is referencing-model. Now consider relation-state as follows:
Employee: Dependent:
+------+ +------+--------+-------------+--------------+
| name | | name | E-name | school_mark | college_mark |
+------+ +------+--------+-------------+--------------+
| A | | a1 | A | 79 | 81 |
| B | | b1 | B | 80 | 60 |
+------+ | b2 | B | 68 | 86 |
+------+--------+-------------+--------------+
Dependenta1refers to employeeA, and dependentb1, b2references to employeeB.
Now my query is:
Find all employees those having son/daughter has distinction marks (say >= 75%) in both college and school?
>>> Employee.objects.filter(dependent__school_mark__gte=75,
... dependent__college_mark__gte=75)
[<Employee: A>]
Output is 'A' dependent 'a1' has distinction marks in both college and school is dependent on employee 'A'. Note 'B' is not selected because nether of 'B''s child has distinction marks in both college and school. Relational algebra:
Employee ⋈(school_mark >=75 AND college_mark>=75)Dependent
In Second, case I need a query:
Find all employees whose some of dependents has distinction marks in college and school?
>>> Employee.objects.filter(
... dependent__school_mark__gte=75
... ).filter(
... dependent__college_mark__gte=75)
[<Employee: A>, <Employee: B>]
This time 'B' also selected because 'B' has two children (more than one!), one has distinction mark in school 'b1' and other is has distinction mark in college 'b2'.
Order of filter doesn't matter we can also write above query as:
>>> Employee.objects.filter(
... dependent__college_mark__gte=75
... ).filter(
... dependent__school_mark__gte=75)
[<Employee: A>, <Employee: B>]
result is same! Relational algebra can be:
(Employee ⋈(school_mark >=75)Dependent) ⋈(college_mark>=75)Dependent
Note following:
dq1 = Dependent.objects.filter(college_mark__gte=75, school_mark__gte=75)
dq2 = Dependent.objects.filter(college_mark__gte=75).filter(school_mark__gte=75)
Outputs same result: [<Dependent: a1>]
I check target SQL query generated by Django using print qd1.query and print qd2.query both are same(Django 1.6).
But semantically both are different to me. first looks like simple section σ[school_mark >= 75 AND college_mark >= 75](Dependent) and second like slow nested query: σ[school_mark >= 75](σ[college_mark >= 75](Dependent)).
If one need Code #codepad
btw, it is given in documentation #Spanning multi-valued relationships I have just added an example, I think it will be helpful for someone new.
Most of the time, there is only one possible set of results for a query.
The use for chaining filters comes when you are dealing with m2m:
Consider this:
# will return all Model with m2m field 1
Model.objects.filter(m2m_field=1)
# will return Model with both 1 AND 2
Model.objects.filter(m2m_field=1).filter(m2m_field=2)
# this will NOT work
Model.objects.filter(Q(m2m_field=1) & Q(m2m_field=2))
Other examples are welcome.
This answer is based on Django 3.1.
Environment
Models
class Blog(models.Model):
blog_id = models.CharField()
class Post(models.Model):
blog_id = models.ForeignKeyField(Blog)
title = models.CharField()
pub_year = models.CharField() # Don't use CharField for date in production =]
Database tables
Filters call
Blog.objects.filter(post__title="Title A", post__pub_year="2020")
# Result: <QuerySet [<Blog: 1>]>
Blog.objects.filter(post__title="Title A").filter(post_pub_date="2020")
# Result: <QuerySet [<Blog: 1>, [<Blog: 2>]>
Explanation
Before I start anything further, I have to notice that this answer is based on the situation that uses "ManyToManyField" or a reverse "ForeignKey" to filter objects.
If you are using the same table or an "OneToOneField" to filter objects, then there will be no difference between using a "Multiple Arguments Filter" or "Filter-chain". They both will work like a "AND" condition filter.
The straightforward way to understand how to use "Multiple Arguments Filter" and "Filter-chain" is to remember in a "ManyToManyField" or a reverse "ForeignKey" filter, "Multiple Arguments Filter" is an "AND" condition and "Filter-chain" is an "OR" condition.
The reason that makes "Multiple Arguments Filter" and "Filter-chain" so different is that they fetch results from different join tables and use different conditions in the query statement.
"Multiple Arguments Filter" use "Post"."Public_Year" = '2020' to identify the public year
SELECT *
FROM "Book"
INNER JOIN ("Post" ON "Book"."id" = "Post"."book_id")
WHERE "Post"."Title" = 'Title A'
AND "Post"."Public_Year" = '2020'
"Filter-chain" database query use "T1"."Public_Year" = '2020' to identify the public year
SELECT *
FROM "Book"
INNER JOIN "Post" ON ("Book"."id" = "Post"."book_id")
INNER JOIN "Post" T1 ON ("Book"."id" = "T1"."book_id")
WHERE "Post"."Title" = 'Title A'
AND "T1"."Public_Year" = '2020'
But why do different conditions impact the result?
I believe most of us who come to this page, including me =], have the same assumption while using "Multiple Arguments Filter" and "Filter-chain" at first.
Which we believe the result should be fetched from a table like following one which is correct for "Multiple Arguments Filter". So if you are using the "Multiple Arguments Filter", you will get a result as your expectation.
But while dealing with the "Filter-chain", Django creates a different query statement which changes the above table to the following one. Also, the "Public Year" is identified under the "T1" section instead of the "Post" section because of the query statement change.
But where does this weird "Filter-chain" join table diagram come from?
I'm not a database expert. The explanation below is what I understand so far after I created the exact structure of the database and made a test with the same query statement.
The following diagram will show where this weird "Filter-chain" join table diagram comes from.
The database will first create a join table by matching the row of the "Blog" and "Post" tables one by one.
After that, the database now does the same matching process again but uses the step 1 result table to match the "T1" table which is just the same "Post" table.
And this is where this weird "Filter-chain" join table diagram comes from.
Conclusion
So two things make "Multiple Arguments Filter" and "Filter-chain" different.
Django create different query statements for "Multiple Arguments Filter" and "Filter-chain" which make "Multiple Arguments Filter" and "Filter-chain" result come from other tables.
"Filter-chain" query statement identifies a condition from a different place than "Multiple Arguments Filter".
The dirty way to remember how to use it is "Multiple Arguments Filter" is an "AND" condition and "Filter-chain" is an "OR" condition while in a "ManyToManyField" or a reverse "ForeignKey" filter.
The performance difference is huge. Try it and see.
Model.objects.filter(condition_a).filter(condition_b).filter(condition_c)
is surprisingly slow compared to
Model.objects.filter(condition_a, condition_b, condition_c)
As mentioned in Effective Django ORM,
QuerySets maintain state in memory
Chaining triggers cloning, duplicating that state
Unfortunately, QuerySets maintain a lot of state
If possible, don’t chain more than one filter
You can use the connection module to see the raw sql queries to compare. As explained by Yuji's, for the most part they are equivalent as shown here:
>>> from django.db import connection
>>> samples1 = Unit.objects.filter(color="orange", volume=None)
>>> samples2 = Unit.objects.filter(color="orange").filter(volume=None)
>>> list(samples1)
[]
>>> list(samples2)
[]
>>> for q in connection.queries:
... print q['sql']
...
SELECT `samples_unit`.`id`, `samples_unit`.`color`, `samples_unit`.`volume` FROM `samples_unit` WHERE (`samples_unit`.`color` = orange AND `samples_unit`.`volume` IS NULL)
SELECT `samples_unit`.`id`, `samples_unit`.`color`, `samples_unit`.`volume` FROM `samples_unit` WHERE (`samples_unit`.`color` = orange AND `samples_unit`.`volume` IS NULL)
>>>
If you end up on this page looking for how to dynamically build up a django queryset with multiple chaining filters, but you need the filters to be of the AND type instead of OR, consider using Q objects.
An example:
# First filter by type.
filters = None
if param in CARS:
objects = app.models.Car.objects
filters = Q(tire=param)
elif param in PLANES:
objects = app.models.Plane.objects
filters = Q(wing=param)
# Now filter by location.
if location == 'France':
filters = filters & Q(quay=location)
elif location == 'England':
filters = filters & Q(harbor=location)
# Finally, generate the actual queryset
queryset = objects.filter(filters)
If requires a and b then
and_query_set = Model.objects.filter(a=a, b=b)
if requires a as well as b then
chaied_query_set = Model.objects.filter(a=a).filter(b=b)
Official Documents:
https://docs.djangoproject.com/en/dev/topics/db/queries/#spanning-multi-valued-relationships
Related Post: Chaining multiple filter() in Django, is this a bug?
There is a difference when you have request to your related object,
for example
class Book(models.Model):
author = models.ForeignKey(Author)
name = models.ForeignKey(Region)
class Author(models.Model):
name = models.ForeignKey(Region)
request
Author.objects.filter(book_name='name1',book_name='name2')
returns empty set
and request
Author.objects.filter(book_name='name1').filter(book_name='name2')
returns authors that have books with both 'name1' and 'name2'
for details look at
https://docs.djangoproject.com/en/dev/topics/db/queries/#s-spanning-multi-valued-relationships
I'm trying query a many_to_many relationship with a prefix as follows:
Student
|> join(:left, [s], t in assoc(s, :teachers))
|> Repo.all(prefix: "my_prefix")
which results in a PostgreSQL query:
SELECT s0."id", s0."name", s0."inserted_at", s0."updated_at"
FROM "my_prefix"."students" AS s0
LEFT OUTER JOIN "teachers_students" AS t2 ON t2."student_id" = s0."id"
LEFT OUTER JOIN "my_prefix"."teachers" AS t1 ON t2."teacher_id" = t1."id"
I would expect the prefix to get added to the join_through table teacher_students, but it doesn't get added. Is this a bug in Ecto? Or is there a workaround for this?
Looks like I was able to fix the problem by changing the join_through value in the many_to_many definition from a string to a module name:
schema "students" do
# many_to_many :teachers, Teacher, join_through: "teachers_students", on_replace: :delete
many_to_many :teachers, Teacher, join_through: TeachersStudents, on_replace: :delete
...
end
defmodule MyApp.TeachersStudents do
use Ecto.Schema
alias MyApp.Teachers.Teacher
alias MyApp.Students.Student
schema "teachers_students" do
belongs_to :teacher, Teacher
belongs_to :student, Student
end
end
Ecto ignoring the prefix for the string value may be a bug.
Ok
i have this class in my model :
i want to get the agencys value which is a many to many on this class and store them in a list or array . Agency which store agency_id with the id of my class on a seprate table.
Agency has it's own tabel as well
class GPSpecial(BaseModel):
hotel = models.ForeignKey('Hotel')
rooms = models.ManyToManyField('Room')
agencys = models.ManyToManyField('Agency')
You can make it a bit more compact by using the flat=True parameter:
agencys_spe = list(GPSpecial.objects.values_list('agencys', flat=True))
The list(..) part is not necessary: without it, you have a QuerySet that contains the ids, and the query is postponed. By using list(..) we force the data into a list (and the query is executed).
It is possible that multiple GPSpecial objects have a common Agency, in that case it will be repeated. We can use the .distinct() function to prevent that:
agencys_spe = list(GPSpecial.objects.values_list('agencys', flat=True).distinct())
If you are however interested in the Agency objects, for example of GPSpecials that satisfy a certain predicate, you better query the Agency objects directly, like for example:
agencies = Agency.objects.filter(gpspecial__is_active=True).distinct()
will produce all Agency objects for which a GPSpecial object exists where is_active is set to True.
I think i found the answer to my question:
agencys_sp = GPSpecial.objects.filter(agencys=32,is_active=True).values_list('agencys')
agencys_spe = [i[0] for i in agencys_sp]
I am creating BOOK_Issue table which will contain id of person to whom the book is issued.
i have a column name user_id witch will contain ids from tbl_student as well as tbl_faculty. so how to set user_id field of book_issue table with reference to two primary key columns.
Your database schema is not correct.
If you expect unique IDs then they should be in one table.
You can create a table with all the users, and have a column to set their type (student, faculty). Then create 2 different tables for each type that has the proper information for each user based on their type.
Create a "person" superclass that can be either of type "student" or type "faculty". Reference this from the BOOK_Issue table instead.
Basically to create this relationship, you'll need one unique ID that spans both "student" and "faculty". Put this in a table (tbl_person?) and have each row in tbl_student and tbl_faculty reference this new table. It's probably also best to then pull out the fields present in both tbl_student and tbl_faculty and put them in this new supertable instead.
You can solve the problem by either having an extra column in BOOK_Issue table, next to user_id, which indicates if this is a Student ID or a Faculty ID.
Alternatively, the IDs themselves may readily include some pattern which indicate their nature (for example no all faculty Ids may start with say "UC", and none of the student Id are so).
The two solutions above then allow using queries similar to the following
SELECT B.*,
CASE B.BorrowerType -- could be LEFT(user_id, 2) etc...
WHEN 'S' THEN S.Name
WHEN 'F' Then F.Name
END As Name,
CASE B.BorrowerType
WHEN 'S' THEN S.PhoneNumber
WHEN 'F' Then F.Phone -- Note that these constructs allow
-- mapping distinct columns names etc.
END As PhoneNr
FROM BOOK_Issue B
LEFT JOIN tbl_student S ON B.BorrowerType = 'S' AND B.user_id = S.id
LEFT JOIN tbl_faculty F ON B.BorrowerType = 'F' AND B.user_id = F.id
WHERE B.DueDate < '11/23/2009' -- or some other condition
This can get a bit heavy when we need to get multiple columns from the student/faculty tables. A possible alternative is a UNION, but this would then cause the repeating of the search clause.
Finally, the best solution but not avaible on all DBMS is a sub-query driven by an "IF B.BorrowerType = 'S' " condition.
This should be your table design:
FacultyTable (FacultyID, FacultyName)
StudentsTable (StudentID, StudentName, FacultlyID, ...)
BookTable (BookID, BookName, ...)
UsersTable(UserID, UserName, UserPassword, StudentID, LastLogin, ...)
Now this is the main thing:
BookIssedTable(BookIssedID, BookID, UserID)
//This table tells me that a book of "BookID was issued to a user of "UserID"
//this can be better for this is certainly a great improvement from the initial design.
i've a trouble with linq, i'll explain on example :
i have a database table called Employee which got FirstName and LastName columns,
and a method to search employees which gets a nameList list as argument, the elements in this list are names formatted like this one "Fred Burn", or this1 "Grim Reaper",
already tryed these approaches with no luck =[
//just all employees
var allData = from emp in Context.Employee select emp;
var test1 = from emp in allData
where(emp.FirstName + " " + emp.LastName).Contains
("" + ((from n in nameList select n).FirstOrDefault()))
select emp;
var test2 = (from emp in allData
where (emp.FirstName + " " + emp.LastName)
== ((from n in nameList select n).FirstOrDefault())
select emp);
var test3 = from emp in allData
where (from n in nameList select n).Contains
(emp.FirstName + " " + emp.LastName)
select emp;
first and second queries give : {"Unable to create a constant value of type 'Closure type'. Only primitive types ('such as Int32, String, and Guid') are supported in this context."} exceptionand third : {"LINQ to Entities does not recognize the method 'Boolean Contains[String](System.Collections.Generic.IEnumerable`1[System.String], System.String)' method, and this method cannot be translated into a store expression."}
would be glad to hear your suggestions :)
Thank You!
p.s.
yea i know it's possible to split names in list and compare them separately, but still curious why wont these queries work.
I assume nameList in this case is an in memory collection and that you are trying to use the LINQ to SQL trick creating a SQL "IN" clause inside of the Entity Framework. Unfortunately, EF doesn't support this syntax (yet). Depending on the number of records you are trying to match, you may be able to run separate queries for each child you are desiring. Alternatively, you could build an entitysql query using concatenation to append the multiple items from the nameList as separate OR clauses in the WHERE operation.