Merge two redundant models in Django - python-3.x

I'm working on an online shop at the moment. The shop is written in Django, and was programmed by another person, so I spent days trying to make sense of what he did. At the moment the shop sells only two articles, on two different pages (meaning they have to be bought separately), so the shop is heavily oriented towards this situation. The problem is, the shop's owner expressed his interest in selling everything in one page in the near future, and in adding more products. And that's where my problems start.
The model structure was something like this (extremely simplified):
class Category1(BaseModel):
name = models.CharField(max_length=32)
class Category2(BaseModel):
name = models.CharField(max_length=64)
class Price1(BaseModel):
category = models.ForeignKey(Category1, on_delete=models.CASCADE)
price = models.DecimalField(max_digits=16, decimal_places=2)
currency = models.CharField(max_length=3)
class Price2(BaseModel):
category = models.ForeignKey(Category2, on_delete=models.CASCADE)
price = models.DecimalField(max_digits=16, decimal_places=2)
currency = models.CharField(max_length=3)
class Order1(BaseModel):
[personal information fields]
class Order2(BaseModel):
[personal information fields]
class Article1(BaseModel):
price = models.ForeignKey(Price1, on_delete=models.CASCADE)
order = models.ForeignKey(Order1, on_delete=models.CASCADE, related_name='articles')
class Article2(BaseModel):
price = models.ForeignKey(Price2, on_delete=models.CASCADE)
order = models.ForeignKey(Order2, on_delete=models.CASCADE, related_name='articles')
There is much more than this, of course, but this should be enough to show the relationships between the models. The complete structure of course makes more sense than this one. BaseModel is a class that contains an ID, creation time and last edit.
I managed to put all the common elements into abstract classes BaseCategory, BasePrice, BaseOrder and BaseArticle, but this is not enough if I want to really expand the shop. Finishing this work is just a matter of time and patience, but how should I proceed once I'm in this situation?
class BaseCategory(BaseModel):
name = models.CharField(max_length=64)
class Meta:
abstract = True
class Category1(BaseCategory):
pass
class Category2(BaseCategory):
pass
class BasePrice(BaseModel):
price = models.DecimalField(max_digits=16, decimal_places=2)
currency = models.CharField(max_length=3)
class Meta:
abstract = True
class Price1(BasePrice):
category = models.ForeignKey(Category1, on_delete=models.CASCADE)
class Price2(BasePrice):
category = models.ForeignKey(Category2, on_delete=models.CASCADE)
class BaseOrder(BaseModel):
[personal information fields]
class Meta:
abstract = True
class Order1(BaseOrder):
pass
class Order2(BaseOrder):
pass
class BaseArticle(BaseModel):
class Meta:
abstract = True
class Article1(BaseArticle):
price = models.ForeignKey(Price1, on_delete=models.CASCADE)
order = models.ForeignKey(Order1, on_delete=models.CASCADE, related_name='articles')
class Article2(BaseArticle):
price = models.ForeignKey(Price2, on_delete=models.CASCADE)
order = models.ForeignKey(Order2, on_delete=models.CASCADE, related_name='articles')
I need to get rid of the specific classes completely, otherwise when I will add new articles, I will have to create new classes, and this is not a scalable solution.
My problems are the following:
How do I get rid of the empty specific classes like Price1 or Order1 without losing any information? I know I will have to get rid of the abstract variable, but I don't know what to do next.
How do I manage the foreign keys in the remaining classes? I'm experimenting a bit with GenericForeignKey at the moment, and this would probably let me move the declarations into the base classes, but I'm not sure if changing a definition will reset all fields.
Just to be clear, the shop is already up and running. We can't stop it, and we can't lose data. We sell services, so the customers have to be able to access their products even long after the purchase.
Thanks in advance for your interest and your time.

To keep this answer short we will only discuss one model here. In this instance Category. Firstly add a new model Category (keep your other models for now):
class Category(BaseModel):
name = models.CharField(max_length=64)
Next run makemigrations this would generate a migration file to make this new table in the database. After this you need to make a Data Migration [Django docs] to copy the data from the other two tables that you have.
To do this first run:
python manage.py makemigrations --empty yourappname
This will generate a migration file that does nothing for now. We will edit this migration file and add some code to copy the data from your other tables to this new table. In the end your migration file would look something like:
from django.db import migrations
def populate_category(apps, schema_editor):
Category1 = apps.get_model('yourappname', 'Category')
Category2 = apps.get_model('yourappname', 'Category')
Category = apps.get_model('yourappname', 'Category')
# add all fields except the pk in values(), i.e. values('field1', 'field2')
for category in Category1.objects.values('name'):
Category.objects.create(**category) # Add some field indicating this object is of Category1 if needed
for category in Category2.objects.values('name'):
Category.objects.create(**category) # Add some field indicating this object is of Category2 if needed
class Migration(migrations.Migration):
dependencies = [
('yourappname', '0001_initial'),
]
operations = [
migrations.RunPython(populate_category, reverse_code=migrations.RunPython.noop),
]
Now you can simply run python manage.py migrate and you would have a new table Category which has all the data from Category1 and Category2. (This might take some time if there are many rows). After this you can remove the models Category1 and Category2 and migrate again to remove those tables.
Note: Perform these operations carefully, and make sure you have got the data properly in the new table before deleting the old ones.
Refer the documentation linked above for more information on
migrations. (Test this on a local development server before doing it
on production to be safe)

Related

Django query fetch foreignkey association without N+1 queries in database

I have two models, Product and Price. I have used the ForeignKey association of the Django models to define the association between product and price. the scenario is, one product can have multiple prices according to size. On the home page, I have to fetch all the products with their prices and need to show their price(probably base price).
Following is the code that I tried.
class Product(BaseModel):
name = models.CharField(max_length=50)
category = models.ForeignKey(Category, null=True, on_delete=models.SET_NULL, help_text='Please add new category if not given.')
image = models.ImageField(upload_to='images/')
tag = models.ManyToManyField(Tag, help_text='You can add multiple tags')
slug = models.SlugField(unique=True, null=True)
time = models.TimeField(verbose_name='Time Required')
class Price(BaseModel):
product = models.ForeignKey(Product, on_delete=models.CASCADE)
size = models.FloatField(validators=[MinValueValidator(0)])
amount = models.FloatField(validators=[MinValueValidator(0)])
Then in the view.py file
class ProductListView(ListView):
model = Product
context_object_name = 'products'
paginate_by = 32
def get_context_data(self,*args, **kwargs):
object = super(ProductListView, self).get_context_data(*args, **kwargs)
object['categories'] = Category.objects.order_by('name')
return object
def get_queryset(self):
return Product.objects.order_by('name')
In the template, I am able to get and loop through the categories and products but I am not able to access the related prices of each product.
If I tried something in the get_context_data, will it cause N+1 queries to fetch prices for every product?
In the template I tried to use something like {{ product.price_set }} but it returns order.Price.None.
use {{ product.price_set.all }}.
To avoid N+1 queries in your filter, use prefetch_related so it looks something like.
Product.objects.all().prefetch_related('price_set')
See prefetch_related in the Django documentation.
See also select related vs prefetch related

Django ORM query with user defined fields

I'm trying to create an Django ORM query to replace a really messy raw SQL query i've written in the past but i'm not sure if Django ORM can let me do it. I have three tables:
contacts
custom_fields
custom_field_data
What I'm hoping to be able to do with the ORM is create an output as if i've queryied one single table like this:
Is such a thing possible with the ORM?
EDIT:
The models are:
class Contact(models.Model):
company = models.ForeignKey(Company, on_delete=models.PROTECT, null=True, blank=True)
class CustomField(models.Model):
name = models.CharField(max_length=100)
company = models.ForeignKey(Company, on_delete=models.PROTECT, null=False, blank=False)
class ContactCustomFieldValue(models.Model):
custom_field = models.ForeignKey(CustomField, on_delete=models.PROTECT, related_name='contact_values')
contact = models.ForeignKey(Contact, on_delete=models.PROTECT, related_name='custom_field_values', null=True)
value = models.TextField(null=True)
Solved this one in large part thanks to Ken from the Django forums.
The solution he provided looked like this:
subquery1 = Subquery(ContactCustomFieldValue.objects.filter(contact=OuterRef('id'), custom_field_id=1).values_list('value'))
subquery2 = ...
subquery3 = ...
subquery4 = ...
contact_list = Contact.objects.filter(...).annotate(field1=subquery1, ...)
I've built on it to fit my needs but as a starting point this was perfect

ManytoManyField Django : how to call models in methods?

I need help for something, I want to call models with ManyToManyField.
I want to have method to get Class A from Class B, and another in Class B to get Class A.
here's my (shortened) code :
class Licence(models.Model):
name = models.CharField(max_length=64)
picture = models.ImageField(upload_to='finder/static/finder/img/licence/',null=True, blank=True)
description = models.TextField(null=True, blank=True)
#returns a list of games from this license
def getGamesOnThisLicence(self):
#i don't know how to proceed
class Game(models.Model):
name = models.CharField(max_length=64)
description = models.TextField()
release_date = models.DateField(null=True, blank=True)
licence = models.ManyToManyField(Licence, blank=True, null=True)
#return name of licence to which the game belongs
def getLicenceName(self):
return self.licence.name
You can access the Games with:
my_license.game_set.all()
so you can use self in the getGamesOnThisLicense, but probably there is not much gain to define a function since this makes accessing the Games already quite convenient.
Perhaps you however want to transform the ManyToManyField into a ForeignKey to License since self.license.name makes not much sense: for a ManyToManyField, self.license is a Manager over License objects that can manage zero, one or more Licenses, so you can not use self.license.name.

Django: Abstract base class for managing db_table

I'm trying to build my second django (and python for that matter, my first project being the django tutorial :)) project. since this is supposed to be something real i'd like to be thorough and build a good code structure before i go into the meat of the project.
I have a couple of simple models like this
class Task(models.Model):
name = models.CharField(max_length=50, null=False)
description = models.CharField(max_length=255, null=True)
dueDate = models.DateTimeField()
I'm using PostgreSQL and i set up my models to use the app label as a database schema by defining the meta class of each model like this
class Meta:
managed = True
db_table = 'app_name\".\"modelname'
This works well. But i have to do this for every single model.
I want to keep it DRY though. So what i'm trying to do now is to have an abstract base class that does this automatically
so i tried the following:
class SchemaModel(models.Model):
class Meta():
abstract = True
managed = True
db_table = AppConfig.label+'\".\"'+self.__class__.lower()+'s'
(the base class was then inherited of course and i took the nested Meta class out of the normal models)
this didn't work though because self isn't accessible in Meta
after consulting the documentation i tried this:
class SchemaModel(models.Model):
class Meta():
abstract = True
managed = True
db_table = '%(app_label)\".\"%(class)s'
which lead to the property db_table of every model being "%(app_label)\".\"%(class)s"
>>> t = Task()
>>> t._meta.db_table
'%(app_label)"."%(class)s'
>>>
I didn't find anything similar on the internet. Am i trying to do something impossible or "forbidden"?
Solution
The solution is as shown in elyas answer to set the db_table property at the end of models.py by looping through all __subclasses__()
for model in SchemaModel.__subclasses__():
db_table = '{}s'.format(model._meta.label_lower.replace('.','\".\"'))
model._meta.original_attrs['db_table'] = db_table
model._meta.db_table = db_table
I wouldn't say it's forbidden. But I can't think of any way to do this declaratively. There is a way to do it however.
Firstly, on your existing attempts:
Accessing 'self'
db_table = AppConfig.label+'\".\"'+self.__class__.lower()+'s'
An instance object is never created from the Meta class when models are loaded, and so there is no self to reference. But even if an instance object were created, db_table is an attribute of the class object, so it is evaluated when the class object is created, which is before any instance object is created, so self cannot be accessed when defining a class attribute in this way.
Edit: And as you mentioned, the app_label cannot be accessed through AppConfig.label.
String formatting
db_table = '%(app_label)\".\"%(class)s'
These placeholders are only used in a very specific situation when defining the related_name and related_query_name attributes of ForeignKey or OneToOneField's fields in an abstract base class.
A solution
As I said, I can't think of any declarative way to achieve this. For example, trying to use __qualname__ won't work because you would just end up with SchemaModel.Meta every time.
But you could put a for loop at the bottom of your models.py like this:
for model in SchemaModel.__subclasses__():
# Name your db_table here
db_table = model._meta.app_label + '\".\"' + model._meta.model_name.lower() + 's'
# Set db_table here
model._meta.original_attrs['db_table'] = db_table
model._meta.db_table = db_table
All of SchemaModel's children can be found using the built-in __subclasses__() method.
The db_table attribute needs to be updated in two places. Firstly in _meta, which is (partly) created by copying attributes from the Meta class, and secondly in _meta.original_attrs where the original Meta attributes are stored and are read by Django during migrations.
Alternative solution
Personally I would define the db_table names manually and simply have a unit test that checks that all models adhere to whatever naming convention I've come up. I prefer this so if another developer eyeballs a model they have never seen before they can get the full picture based on the declarations in the model (and the abstract base class) and don't get confused about an operation modifying them elsewhere.

Django Foreign key to not loaded object

Hello I have a problem that I want to link foreign key to model's related object in other words I want to link foreign key to not loaded object's fields.
class Category(models.Model):
...
filter_option_content_type = models.ForeignKey(ContentType, on_delete=models.SET_NULL, limit_choices_to=(
models.Q(app_label='catalog', model='FrameSize') |
models.Q(app_label='catalog', model='ShoeSize') |
models.Q(app_label='catalog', model='ClothSize')
), null=True)
...
class Product(models.Model):
...
category = models.ForeignKey(Category, on_delete=models.SET_NULL, null=True)
...
class ProductItem(models.Model):
...
model = models.ForeignKey(Product, verbose_name='Модель', on_delete=models.CASCADE, related_name='productitem')
size = models.ForeignKey('model.category.filter_option_content_type', on_delete=models.SET_NULL, null=True)
...
And sure i got this error:
ValueError: Invalid model reference 'model.category.filter_option_content_type'. String model references must be of the form 'app_label.ModelName'
is it possible to do relation like this wtihout using GenericForeignKey instead of ForeignKey?
I think that you don't really need to add an additional relation since you can access the field you want via Product:
item = ProductItem()
size = item.model.category.filter_option_content_type
In case you'd like to query using this field, it would look like:
item = ProductItem.objects.filter(
model__in=Product.objects.filter(
category__in=Category.objects.filter(
filter_option_content_type='<your desired value of size>'
)
)
)
Since the relations are based on Foreign Keys which are indexed - such query shouldn't have a noticeable effect on performance (despite that it may seem a bit awkward)

Resources