Django Aggregate Between Timezone Aware Fields - python-3.x

I want to find the average time between two dates - 'created_date' and 'modified_date'. Both are DateTimeField and values are timezone aware.
Color Model:
name = models.CharField(max_length=150)
is_published = models.BooleanField(default=False)
created_date = models.DateTimeField(null=True, blank=True)
modified_date = models.DateTimeField(null=True, blank=True)
To Save:
example.created_date = datetime.datetime.now(tz=timezone.utc)
example.save()
I ran the below query:
avg_change = Color.objects.filter(is_published=True, created_date__isnull=False, modified_date__isnull=False).aggregate(avg_score=Avg(F('modified_date') - F('created_date')))
I got the error below.
if dt.tzinfo is not None:
AttributeError: 'decimal.Decimal' object has no attribute 'tzinfo'
I'm on Python3.6/Django2.2/mysql 5.7.
I'm not sure if I need to change the values to naive and then aggregate. What am I missing? Any viable route to approach this as I am not good with raw SQL.

I later did this:
from django.db.models import F, ExpressionWrapper
from django.db import models
avg_duration = ExpressionWrapper(F('modified_date') - F('created_date'), output_field=models.DurationField())
avg_change = Color.objects.filter(is_published=True, created_date__isnull=False, modified_date__isnull=False).aggregate(avg_score=Avg(avg_duration))
If you know of a more better option kindly share.

Related

Django query fetch foreignkey association without N+1 queries in database

I have two models, Product and Price. I have used the ForeignKey association of the Django models to define the association between product and price. the scenario is, one product can have multiple prices according to size. On the home page, I have to fetch all the products with their prices and need to show their price(probably base price).
Following is the code that I tried.
class Product(BaseModel):
name = models.CharField(max_length=50)
category = models.ForeignKey(Category, null=True, on_delete=models.SET_NULL, help_text='Please add new category if not given.')
image = models.ImageField(upload_to='images/')
tag = models.ManyToManyField(Tag, help_text='You can add multiple tags')
slug = models.SlugField(unique=True, null=True)
time = models.TimeField(verbose_name='Time Required')
class Price(BaseModel):
product = models.ForeignKey(Product, on_delete=models.CASCADE)
size = models.FloatField(validators=[MinValueValidator(0)])
amount = models.FloatField(validators=[MinValueValidator(0)])
Then in the view.py file
class ProductListView(ListView):
model = Product
context_object_name = 'products'
paginate_by = 32
def get_context_data(self,*args, **kwargs):
object = super(ProductListView, self).get_context_data(*args, **kwargs)
object['categories'] = Category.objects.order_by('name')
return object
def get_queryset(self):
return Product.objects.order_by('name')
In the template, I am able to get and loop through the categories and products but I am not able to access the related prices of each product.
If I tried something in the get_context_data, will it cause N+1 queries to fetch prices for every product?
In the template I tried to use something like {{ product.price_set }} but it returns order.Price.None.
use {{ product.price_set.all }}.
To avoid N+1 queries in your filter, use prefetch_related so it looks something like.
Product.objects.all().prefetch_related('price_set')
See prefetch_related in the Django documentation.
See also select related vs prefetch related

Django ORM query with user defined fields

I'm trying to create an Django ORM query to replace a really messy raw SQL query i've written in the past but i'm not sure if Django ORM can let me do it. I have three tables:
contacts
custom_fields
custom_field_data
What I'm hoping to be able to do with the ORM is create an output as if i've queryied one single table like this:
Is such a thing possible with the ORM?
EDIT:
The models are:
class Contact(models.Model):
company = models.ForeignKey(Company, on_delete=models.PROTECT, null=True, blank=True)
class CustomField(models.Model):
name = models.CharField(max_length=100)
company = models.ForeignKey(Company, on_delete=models.PROTECT, null=False, blank=False)
class ContactCustomFieldValue(models.Model):
custom_field = models.ForeignKey(CustomField, on_delete=models.PROTECT, related_name='contact_values')
contact = models.ForeignKey(Contact, on_delete=models.PROTECT, related_name='custom_field_values', null=True)
value = models.TextField(null=True)
Solved this one in large part thanks to Ken from the Django forums.
The solution he provided looked like this:
subquery1 = Subquery(ContactCustomFieldValue.objects.filter(contact=OuterRef('id'), custom_field_id=1).values_list('value'))
subquery2 = ...
subquery3 = ...
subquery4 = ...
contact_list = Contact.objects.filter(...).annotate(field1=subquery1, ...)
I've built on it to fit my needs but as a starting point this was perfect

Django Foreign key to not loaded object

Hello I have a problem that I want to link foreign key to model's related object in other words I want to link foreign key to not loaded object's fields.
class Category(models.Model):
...
filter_option_content_type = models.ForeignKey(ContentType, on_delete=models.SET_NULL, limit_choices_to=(
models.Q(app_label='catalog', model='FrameSize') |
models.Q(app_label='catalog', model='ShoeSize') |
models.Q(app_label='catalog', model='ClothSize')
), null=True)
...
class Product(models.Model):
...
category = models.ForeignKey(Category, on_delete=models.SET_NULL, null=True)
...
class ProductItem(models.Model):
...
model = models.ForeignKey(Product, verbose_name='Модель', on_delete=models.CASCADE, related_name='productitem')
size = models.ForeignKey('model.category.filter_option_content_type', on_delete=models.SET_NULL, null=True)
...
And sure i got this error:
ValueError: Invalid model reference 'model.category.filter_option_content_type'. String model references must be of the form 'app_label.ModelName'
is it possible to do relation like this wtihout using GenericForeignKey instead of ForeignKey?
I think that you don't really need to add an additional relation since you can access the field you want via Product:
item = ProductItem()
size = item.model.category.filter_option_content_type
In case you'd like to query using this field, it would look like:
item = ProductItem.objects.filter(
model__in=Product.objects.filter(
category__in=Category.objects.filter(
filter_option_content_type='<your desired value of size>'
)
)
)
Since the relations are based on Foreign Keys which are indexed - such query shouldn't have a noticeable effect on performance (despite that it may seem a bit awkward)

Proper way to do a robust search in Django models via REST framework

I'm writing a web application (DRF + Vue.js) where frontend should have an ability to narrow down GET request results via different filters.
For example, I have a model like this:
class Human(models.Model):
first_name = models.CharField(_('first name'), max_length=50, null=True, blank=True)
last_name = models.CharField(_('last name'), max_length=50, null=True, blank=True)
birth_date = models.DateField(_('birth date'), blank=True, null=True)
city = models.ForeignKey('City', on_delete=models.SET_NULL, blank=True, null=True)
phone_number = models.ForeignKey('Contact' on_delete=models.SET_NULL, blank=True, null=True)
#property
def full_name(self):
# Last name + First name
return ' '.join(str(x) for x in (self.last_name, self.first_name) if x)
#property
def is_adult(self):
now = timezone.now()
if self.birth_date:
if now.year - self.birth_date.year - \
((now.month, now.day) < (self.birth_date.month, self.birth_date.day)) >= 18:
return True
return False
Now I have simple CRUD ViewSet where I can use a list of search_fields to search by all needed fields (in my case that's birth_date, city, phone_number and full_name/is_adult). But here next problems arise:
Using search_fields I can do a search only by all fields specified in the ViewSet's search_fields (frontend can't search only by city or other distinct fields if it wants to) - other way I'll need to create a separate ViewSet (and separate URL?) for every combination of fields to filter by. Terrific.
So it looks like the correct decision must be capable of filtering by several GET parameters at once. Ideally - with opportunity to choose exact/icontains/etc comparison method on each query.
That sounds like a work for django-filter but I'm not sure yet.
It's impossible to search/filter by full_name or is_adult because they are dynamic model properties, not usual fields.
So it looks like instead of using model properties I need to use separate QuerySets (Manager methods?) that will do the logic of fiddling with model fields and creating the filtered result.
But for now I didn't find a way to choose different QuerySets in a single ViewSet depending on GET parameters (or how to use search by these complex properties together with simple search from problem 1?).
And I have no understanding if it is possible to provide this kind of search by the same URL as a "simple" search - like site.com/api/people/?city=^New&full_name=John%20Doe (perfectly - with opportunity to document query parameters for OpenAPI schema / Swagger)
So maybe someone knows which is the most elegant way to provide a capability of such complex search with Django/DRF? In which direction should I look?
"full_name" alias needs to be part of your queryset.
You can achieve it by queryset annotation.
In your People view (api/people/ controller) you have to set your queryset to be:
from django.db.models.functions import Concat
from django.db.models import Value
queryset = Human.objects.annotate(fullname=Concat('first_name', Value(' '), 'last_name'))
Also, customize your "filter_backed" to act the way you need.
class PeopleFilterBackend(filters.BaseFilterBackend):
def filter_queryset(self, request, queryset, view):
params_serializer = PeopleFilterSerializer(data=request.params)
params_serializer.is_valid(raise_exception=True)
valid_params = params_serializer.data
if full_name := valid_params.get("full_name")
queryset = queryset.filter(full_name__icountains=full_name)
# insert here all of other filters logic
return queryset
PeopleFilterSerializer is your custom params serializer,
You have to specify there all of your accepted params:
from rest_framework import serializers
class PeopleFilterSerializer(serializers.Serializer):
full_name = serializers.CharField() # set required=False if not required
city = serializer.CharField()

PostGIS geography does not support the "~=" function/operator

I am trying to save point field in the database via update_or_create method but it is giving me this error.
My function to save data:
for city in cities_data:
obj, created = models.City.objects.update_or_create(
name = city["name"],
country = city["country"],
state = city["state"],
point = Point(
float(city["longitude"]),
float(city["latitude"])
),
radius = city["radius"],
is_curated = city["is_curated"],
is_metro_city = city["is_metro_city"]
)
obj.save()
Model:
class City(models.Model):
name = models.CharField(max_length=50)
country = models.CharField(max_length=50)
state = models.CharField(max_length=50)
point = gis_models.PointField(geography=True, null=True)
is_metro_city = models.BooleanField(default=False)
is_curated = models.BooleanField(default=False)
radius = models.IntegerField(null=True)
when I try to run this I get this error:
ValueError: PostGIS geography does not support the "~=" function/operator.
I want to know why I am getting this error, I did not find any useful information related to this. Thanks in Advance.
If you want to use this you can use it like this:
obj, created = models.City.objects.update_or_create(
**city_data,
defaults={'point': Point(
float(city["longitude"]),
float(city["latitude"])
)}
)
I do not know why it does not work that way, but I made it work this way, if you find it out why it does not work you can improve the answer.
I have the same error and the problem was that I have a unique together constraint in model's class Meta eg:
class Meta:
unique_together = 'name', 'geom'
After deleting it, the error disappears although you must check by yourself if you need such constraint.

Resources