Right now I have double the classes for the same data. The first are the Bill and the Expense class, used locally to exchange data within the program. I then I have Bill_Table and Expense_Table, used to exchange data between the program and database. This makes my program needlessly complicated, when I just want one of each.
Bill has a member variable that is a list of Expenses, like so:
class Bill:
vendor = None # type: str
expenses = None # type: list[Expense]
# plenty more variables here
def __init__(self, vendor=None,):
self.vendor = vendor
self.expenses = list()
class Expense:
account = None # type: str
amount = None # type: int
# etc...
My Bill_Table and Expense_Table are set up pretty much identical. I use some functions to convert a Bill into a Bill_table, or Expense into an Expense_Table, or visa versa.
from sqlalchemy import Column, Integer, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Bill_Table(Base):
__tablename__ = 'bills'
id = Column(Integer, primary_key=True)
expenses = relationship("Expense_Table")
# etc...
class Expense_Table(Base):
__tablename__ = 'expenses'
id = Column(Integer, primary_key=True)
bill_id = Column(Integer, ForeignKey('bills.id'))
# etc...
How would I map some Expense_Table objects to a Bill_Table object, without connecting to a database? So I could have the same functionality, but also when I insert a Bill_Table into the database, it will also import it's Expense_Table objects with it too?
Related
I declared the following models in sqlalchemy (python). My loading strategy is to use deferred as a default, and to modify my query when I need to avoid lazy loading.
What I'm trying to achieve is to get all legal_provision records, and have the related case_law items loaded at the same time, overruling the lazy loading (deferred) for the url column, but not for the other columns of case_law.
I don't seem to be able to find a convenient solution for this. Any help much appreciated!
ORM:
from sqlalchemy import Integer, String, Column, Table, ForeignKey
from sqlalchemy.orm import deferred, relationship, declarative_base
Base = declarative_base()
case_law_legal_provisions = Table(
'case_law_legal_provisions',
Column('case_law_id', ForeignKey('case_law.id'), primary_key=True),
Column('legal_provision_id', ForeignKey('legal_provision.id'), primary_key=True)
)
class LegalProvision(Base):
__tablename__ = 'legal_provision'
id = Column(Integer, primary_key=True)
reference = deferred(Column(String, unique=True))
case_law = relationship('CaseLaw', secondary='case_law_legal_provisions', backpopulates='legal_provisions')
class CaseLaw(Base):
__tablename__ = 'case_law'
id = Column(Integer, primary_key=True)
url = deferred(Column(String))
contents = deferred(Column(String))
legal_provisions = relationship('LegalProvision', secondary='case_law_legal_provisions', backpopulates='case_law')
class Category(models.Model):
name = models.CharField(max_length=100)
date = models.DateTimeField(auto_now=True)
class Hero(models.Model):
name = models.CharField(max_length=100)
category = models.ForeignKey(Category, on_delete=models.CASCADE)
I want Categoty model name, data, id
In cookbook , I wrote the code as above.
hero_qs = Hero.objects.filter(
category=OuterRef("pk")
).order_by("-benevolence_factor")
Category.objects.all().annotate(
most_benevolent_hero=Subquery(
hero_qs.values('name')[:1]
)
)
It seems that only one value can be entered in hero_qs.values('name')
Is it possible to get name, data, id with one annotate?
You can try Concatenating the fields if you really want to use a single annotation
from django.db.models import Subquery, OuterRef, CharField, Value as V
from django.db.models.functions import Concat
hero_qs = Hero.objects.filter(
category=OuterRef("pk")
).order_by("-benevolence_factor").annotate(
details=Concat('name', V(','), 'id', output_field=CharField())
)
Category.objects.all().annotate(
most_benevolent_hero=Subquery(
hero_qs.values('details')[:1]
)
)
Then you can use string interpolation to separate that data out which is a relatively inexpensive operation
name, id = category.most_benevolent_hero.split(',')
I am trying to create a model class Document which optionally contains 2 pointers to 2 other elements of the same class (table).
original_id which should hold the index of another member of the table which is "the" original version of the document (can be null).
first_id which should hold the index of another member of the table which is the "first" document that was stored (can be null).
I defined my class as
class Document(Base):
__tablename__ = 'documents'
id = Column(Integer, primary_key=True)
name = Column(String)
original_id = Column(Integer, ForeignKey('documents.id'))
original = relationship('Document', foreign_keys= original_id)
first_id = Column(Integer, ForeignKey('documents.id'))
first = relationship('Document', foreign_keys= first_id)
def __repr__(self):
return f'{self.name} ({self.id})'
But when I test it as follows:
d1 = Document(name='Dee one')
d2 = Document(name='Dee two')
d1.original.append(d2) # d1 has for original document d2
I am surprised by the implementation in terms of ids stored in what row (after committing)
>>> d1.original_id
None
>>> d2.original_id
1
>>> d1.original
[Dee two (2)]
>>> d2.original
[]
I wanted d1.original_id to hold 2 (the index for Dee two).
Obviously I am doing something wrong, but I am struggling to understand what. It seems I am ending up with a many to one relation but in the wrong direction.
I am using SQLAlchemy 1.3.5
Originally, I wanted to have a class that contains 2 nullable pointers to other Nodes. To make the relationship clearer, let's call the class Person, and the links Father and Mother. Every person has at most a known Father and a known Mother.
So following the docs, I created the class as
class Person(Base):
__tablename__ = 'persons'
id = Column(Integer, primary_key=True)
name = Column(String)
father_id = Column(Integer, ForeignKey('persons.id'))
father = relationship('Person')
When doing so, the father relationship is 1-to-many, i.e. a Person can have many fathers, and a father can be only the father of 1. Which is not what is expected.
Following Many to One relationship with SQLAlchemy in the same table I found a way to make that a 1 to many relationship. Enter the remote_side parameter:
class Person(Base):
__tablename__ = 'persons'
id = Column(Integer, primary_key=True)
name = Column(String)
father_id = Column(Integer, ForeignKey('persons.id'))
father = relationship('Person', remote_side=id)
^^^^^^^^^^^^^^
Now the relationship is as expected. However with multiple links (father and mother), this fails again. I.e. if I refine the class as:
class Person(Base):
__tablename__ = 'persons'
id = Column(Integer, primary_key=True)
name = Column(String)
father_id = Column(Integer, ForeignKey('persons.id'))
father = relationship('Person', remote_side= id)
mother_id = Column(Integer, ForeignKey('persons.id'))
mother = relationship('Person', remote_side= id)
SQLAlchemy fails with an "AmbiguousForeignKeysError". The solution is then to specify what field is used to link to father and mother, as follows:
class Person(Base):
__tablename__ = 'persons'
id = Column(Integer, primary_key=True)
name = Column(String)
father_id = Column(Integer, ForeignKey('persons.id'))
father = relationship('Person', remote_side= id, foreign_keys=father_id)
^^^^^^^^^^^^^^^^^^^^^^
mother_id = Column(Integer, ForeignKey('persons.id'))
mother = relationship('Person', remote_side= id, foreign_keys=mother_id)
^^^^^^^^^^^^^^^^^^^^^^
Now that works as expected.
So in some way, this is an extension of the solution provided in Many to One relationship with SQLAlchemy in the same table
Many thanks Gord Thomson for your remarks.
I'm trying to understand why I'm getting sql errors for the following objects.
I'm using a PostGresSql database on heroku
class Member(BASE):
__tablename__ = "members"
name = Column(String)
discord_id = Column(BigInteger, primary_key=True,nullable=False)
role = Column(String, ForeignKey('roles.name'))
nick = Column(String)
rs_runs = relationship("RSRun",secondary='member_on_rs_run')
def __init__(self,name,discord_id,nick="" ):
self.name = name
self.discord_id = discord_id
self.nick = nick
class RSRun(BASE):
__tablename__ = "rs_runs"
id = Column(Integer, primary_key=True, nullable=False)
level = Column(String)
dtg = Column(DateTime)
members = relationship("Member",secondary='member_on_rs_run')
class MemberOnRSRun(BASE):
__tablename__ = "member_on_rs_run"
member_id = Column(BigInteger, ForeignKey('members.discord_id'),primary_key = True)
run_id = Column(Integer,ForeignKey('rs_runs.id'),primary_key=True)
The errors I'm getting are here
sqlalchemy.exc.DataError: (psycopg2.errors.NumericValueOutOfRange) integer out of range
[SQL: INSERT INTO members (name, discord_id, role, nick) VALUES (%(name)s, %(discord_id)s, %(role)s, %(nick)s)]
[parameters: {'name': '', 'discord_id': 126793648221192192, 'role': None, 'nick': ''}]
I've checked and the discord_id is within legal range of a PostGres BigInt which is what I'm assuming I'm getting with an sqlalchemy Biginteger Column. But it keeps telling me it's out of range for integer. Obviously I want to use the same ID that Discord is using since that is the identifier for a member on Discord.
Making changes to the python objects that you are using in the sqlalchemy declarative form will not FORCE heroku to update the database. You must manually blow away the database for these changes to take affect.
I am using SQLAlchemy to handle requests from an API endpoint; my database tables (I have hundreds) are differentiated via a unique string (e.g. test_table_123)...
In the code below, __tablename__ is static. If possible, I would like that to change based on the specific table I would like to retrieve, as it would be tedious to write several hundred unique classes.
from config import db, ma # SQLAlchemy is init'd and tied to Flask in this config module
class specific_table(db.Model):
__tablename__ = 'test_table_123'
var1 = db.Column(db.Integer, primary_key=True)
var2 = db.Column(db.String, index=True)
var3 = db.Column(db.String)
class whole_table_schema(ma.ModelSchema):
class Meta:
model = specific_table
sqla_session = db.session
def single_table(table_name):
# collect the data from the unique table
my_data = specific_table().query.order_by(specific_table.level_0).all()
Thank you very much for your time in advance.
You can use reflect feature of SQLAlchemy.
engine = db.engine
metadata = MetaData()
metadata.reflect(bind=engine)
and finally
db.session.query(metadata.tables[table_name])
If you want smoother experience with querying, as previous solution cannot offer one, you might declare and map your tables: tables = {table_name: create_table(table_name) for table_name in table_names}, where create_table constructs models with different __tablename__. Instead of creating all tables at once, you can create them on demand.