I am trying to wrap my head around SQLAlchemy in combination with Marshmallow. I had a Flask API that contains some Assets and Trading Pairs. I want bidirectional One-to-Many relationships between these models. I have the following code:
class Asset(db.Model):
__tablename__ = 'asset'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(255), unique=True, nullable=False)
abbreviation = db.Column(db.String(20), unique=True, nullable=True)
trading_bases = relationship("TradingPair", back_populates="base_asset", foreign_keys="TradingPair.base_id")
trading_quotes = relationship("TradingPair", back_populates="quote_asset", foreign_keys="TradingPair.quote_id")
class TradingPair(db.Model):
__tablename__ = 'trading_pair'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(255), unique=True, nullable=False)
# One to many (a pair can have only one base, but 1 asset can be the base of many pairs)
base_id = db.Column(db.Integer, db.ForeignKey("asset.id"), nullable=False)
base_asset = relationship("Asset", foreign_keys=[base_id], uselist=False, back_populates="trading_bases")
# One to many (same reasoning)
quote_id = db.Column(db.Integer, db.ForeignKey("asset.id"), nullable=False)
quote_asset = relationship("Asset", foreign_keys=[quote_id], uselist=False, back_populates="trading_quotes")
With the following resource for trading pair POST:
def post(self, name):
pair = TradingPair.query.filter_by(name=name).first()
if pair:
return {"Error": "This resource already exists"}, 409
data = request.get_json()
schema = TradingPairSchema()
try:
pair = schema.load(data, session=db.session)
if not pair.name == name:
return {"Error": f"{name} does not correspond to name in data"}
db.session.add(pair)
db.session.commit()
return {"Success":f"Added: {pair.name}"}
except ValidationError as e:
return {"Error": e.messages}, 400
except:
return {"Error":"Database error"}, 500
I expect SQLAlchemy to add new Assets that are POSTed as part of a new trading pair. However, if I want to post new pairs via the API using the following JSON:
{'name': 'DASHUSDT',
'base_asset': {
'name': 'Dash',
'abbreviation': 'DASH'},
'quote_asset': {
'name': 'Tether',
'abbreviation':
'USDT'}}
This works properly and the pair gets added to the DB as expected. The problem occurs when I try to add another pair that contains Dash or Tether. The pair is added again to the DB and my uniqueness constraint on the Asset table is violated. How can I ensure that a new instance is not created but the existing asset is used?
I ended up with checking whether the assets exist and adding them to the database if they do not yet exist. The code I used in the POST function of the trading pair is:
loaded = schema.load(data, session=db.session)
if not loaded.name == name:
return {"Error": f"{name} does not correspond to name in data"}
base = Asset.query.filter_by(abbreviation=loaded.base_asset.abbreviation).first()
if not base:
base = Asset(name=loaded.base_asset.name , abbreviation=loaded.base_asset.abbreviation)
db.session.add(base)
quote = Asset.query.filter_by(abbreviation=loaded.quote_asset.abbreviation).first()
if not quote:
quote = Asset(name=loaded.quote_asset.name, abbreviation=loaded.quote_asset.abbreviation)
db.session.add(quote)
pair = TradingPair(name=name, base_asset=base, quote_asset=quote)
db.session.add(pair)
db.session.commit()
This seems to work properly when the Asset already exist, but also does not crash when a new Asset is inserted via the POST of a trading pair. I could not find any documentation in either SQLAlchemy, Flask-SQLAlchemy or Marshmallow-SQLAlchemy on how this should be handled properly but for now, this works.
Related
I'm trying to understand the nature and usage of .relationship and .ForeignKey in SQLalchemy.
In every example I see they seem to be tied to a variable that is not often not referenced anywhere making it a dead variable and the .relationship backref value is also often not referenced so the whole thing seems arbitrary and difficult to understand.
Here is a random example I pulled from online.
The best that I can see is that
excuses = db.relationship('Excuse', backref='student',
lazy='dynamic')
provides a one-to-many link between the Student model and the Excuse model with 'Excuse' being the 'many' and backref='student' being the 'one. However the excuses variable that is is connected to is not referenced in the foreign key or anywhere else so I don't know how it comes into play. I would be able to understand better with a visual diagram on how they interact but I haven't been able to find such a thing.
student_id = db.Column(db.Integer, db.ForeignKey('students.id'))
Creates a variable with the student ID from the Student model but it seems to do so without the need for the .relationship statement in the Student model.
There is no reference here to the .relationship syntax in the Student model. It does provide a link from Excuse to Student but I don't understand the point of the
excuses = db.relationship('Excuse', backref='student',
lazy='dynamic')
clause as it doesn't seem to do anything.
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
class Student(db.Model):
__tablename__ = "students" # table name will default to name of the model
# Create the three columns for our table
id = db.Column(db.Integer, primary_key=True)
first_name = db.Column(db.Text)
last_name = db.Column(db.Text)
excuses = db.relationship('Excuse', backref='student',
lazy='dynamic')
# define what each instance or row in the DB will have (id is taken care of for you)
def __init__(self, first_name, last_name):
self.first_name = first_name
self.last_name = last_name
# this is not essential, but a valuable method to overwrite as this is what we will see when we print out an instance in a REPL.
def __repr__(self):
return f"The student's name is {self.first_name} {self.last_name}"
class Excuse(db.Model):
__tablename__ = "excuses"
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.Text)
is_believable = db.Column(db.Boolean)
# remember - the name of our table is "students"
student_id = db.Column(db.Integer, db.ForeignKey('students.id'))
def __init__(self, name, is_believable, student_id):
self.name = name
self.is_believable = is_believable
self.student_id = student_id
elie = Student('Elie', 'Schoppik')
matt = Student('Matt', 'Lane')
michael = Student('Michael', 'Hueter')
db.session.add_all([elie, matt, michael])
db.session.commit()
len(Student.query.all()) # 3
elie = Student.query.get(1)
excuse1 = Excuse('My homework ate my dog', False, 1)
db.session.add(excuse1)
db.session.commit()
elie.excuses.all() # list of excuses
elie.excuses.first().is_believable # False
Excuse.query.get(1).student # The student's name is Elie Schoppik
excuse2 = Excuse('I overslept', True, 1)
db.session.add(excuse2)
db.session.commit()
len(elie.excuses.all()) # 2
Does anyone mind explaining the dynamic between .relationship and .ForeignKey and how they interact? As well as the point of the variables they are attached to and how the backref variable is used?
I'm using the python's framework pyramid, sqlalchemy for the data and marshmallow to serialize.
I'm want to send a sum of value to the client
I have Challenge and player whom use this challenge, each actions of the player is an event and each events has a duration.
For each challenge, i want to send the sum's duration for each player and also send this value only for one player. I try to use hybrid attribute or pre_dump function but i didn't succeeded
First method : with hybrid attribute
__tablename__ = "Challenge"
id = Column(Integer, primary_key=True)
name = Column(String(255), unique=True, nullable=False)
description = Column(TEXT(length=65535))
map_url = Column(String(255))
end_date = Column(DateTime(timezone=False))
alone_only = Column(Integer)
level = Column(String(255))
scalling = Column(Integer)
draft = Column(Boolean, server_default=text("0"))
admin_id = Column(Integer, ForeignKey("User.id"))
admin = relationship("User", backref="challenge_manager")
event_sum_user = relationship("Events")```
#hybrid_property
def event_sum(self):
return sum(Events.duration for Events in self.event_sum_user)
But i have sum for all user, not by user or for one user
Second method :with pre_dump method
id = fields.Int()
name = fields.Str()
description = fields.Str()
end_date = fields.DateTime()
alone_only = fields.Int()
level = fields.Str()
scalling = fields.Int()
draft = fields.Bool()
admin = fields.Nested(UserSchema)
admin_id = fields.Int(load_only=True)
event_sum = fields.Int(dump_only=True)
#pre_dump
def get_eventsum(self,data, **kwargs):
data["event_sum"] = DBSession.query(func.sum(Events.duration)).filter(Events.challenge_id==data["id"]).filter(Events.user_id==1).first()
return data```
With this method, i've have an error TypeError: 'Challenge' object is not subscriptable'
The purpose of this is to send with each challenge the total duration realise by a user or for each user on the challenge : id_user and total duration.
Thanks for your help
Marilyn
I have model User:
class User(db.Model):
__tablename__ = "users"
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
username = db.Column(db.String(50), unique=True, nullable=False)
token_life_time = db.Column(db.Integer, nullable=False)
and ModelView for it from flask-admin:
class UserModelView(ModelView):
column_list = ('id', 'username', 'token_life_time')
form_create_rules = ('username', 'token_life_span') # custom field, that doesn't exist in actual model
form_edit_rules = ('username', 'token_life_time', 'new_token_life_time')
form_extra_fields = {
'token_life_span': StringField(default='1-0-0-0', validators=[DataRequired()]),
'new_token_life_time': StringField() # how to put here actual value from model.token_life_time as default value in EDIT form ???
}
column_formatters = dict(
token_life_time=lambda v, c, m, p: UserModelView.convert_seconds_to_human_readable(m.token_life_time)
)
def on_model_change(self, form, model, is_created):
token_time = form.data.get('token_life_span', None)
if token_time:
model.token_life_time = self.convert_to_seconds(token_time) # my custom function that converts string of format (days-hours-minutes-seconds, example: 1-8-23-15) into seconds for storing in DB int type
I want to access model.token_life_time value before EDIT form presented to user and convert it from seconds to human readable string (with days, hours etc.). Then to put converted value as default value in custom field in edit form (new_token_life_time).
The question is - how to access actual value from model and then put converted value to custom field as default value in EDIT form ???
Override the view's edit_form method - see documentation.
Something like the following:
class UserModelView(ModelView):
def edit_form(self, obj=None):
form = super().edit_form(obj=obj)
# calculate token_life_span
_token_life_span = get_token_life_span(obj.token_life_time)
form.token_life_span.data = _token_life_span
return form
I have recently come to know about fastAPI and worked my way through the tutorial and other docs. Although fastAPI is pretty well documented, I couldn't find information about how to process a nested input when working with a database.
For testing, I wrote a very small family API with two models:
class Member(Base):
__tablename__ = 'members'
id = Column(Integer, primary_key=True, server_default=text("nextval('members_id_seq'::regclass)"))
name = Column(String(128), nullable=False)
age = Column(Integer, nullable=True)
family_id = Column(Integer, ForeignKey('families.id', deferrable=True, initially='DEFERRED'), nullable=False, index=True)
family = relationship("Family", back_populates="members")
class Family(Base):
__tablename__ = 'families'
id = Column(Integer, primary_key=True, server_default=text("nextval('families_id_seq'::regclass)"))
family_name = Column(String(128), nullable=False)
members = relationship("Member", back_populates="family")
and I created a Postgres database with two tables and the relations described here. With schema definitions and a crud file as in the fastAPI tutorial, I can create individual families and members and view them in a nested fashion with a get request. Here is the nested schema:
class Family(FamilyBase):
id: int
members: List[Member]
class Config:
orm_mode = True
So far, so good. Now, I would like to add a post view which accepts the nested structure as input and populates the database accordingly. The documentation at https://fastapi.tiangolo.com/tutorial/body-nested-models/ shows how to do this in principle, but it misses the database (i.e. crud) part.
As the input will not have id fields and obviously doesn't need to specify family_id, I have a MemberStub schema and the NestedFamilyCreate schema as follows:
class MemberStub(BaseModel):
name: str
age: int
class NestedFamilyCreate(BaseModel):
family_name: str
members: List[MemberStub]
In my routing routine families.py I have:
#app.post('/nested-families/', response_model=schemas.Family)
def create_family(family: schemas.NestedFamilyCreate, db: Session = Depends(get_db)):
# no check for previous existence as names can be duplicates
return crud.create_nested_family(db=db, family=family)
(the response_model points to the nested view of a family with all members including all ids; see above).
What I cannot figure out is how to write the crud.create_nested_family routine. Based on the simple create as in the tutorial, this looks like:
def create_nested_family(db: Session, family: schemas.NestedFamilyCreate):
# split information in family and members
members = family.members
core_family = None # ??? This is where I get stuck
db_family = models.Family(**family.dict()) # This fails
db.add(db_family)
db.commit()
db.refresh(db_family)
return db_family
So, I can extract the members and can loop through them, but I would first need to create a new db_family record which must not contain the members. Then, with db.refresh, I would get the new family_id back, which I could add to each record of members. But how can I do this? If I understand what is required here, I would need to achieve some mapping of my nested schema onto a plain schema for FamilyCreate (which works by itself) and a plain schema for MemberCreate (which also works by itself). But how can I do this?
I found a solution after re-reading about Pydantic models and their mapping to dict.
in crud.py:
def create_nested_family(db: Session, family: schemas.NestedFamilyCreate):
# split information in family and members
family_data = family.dict()
member_data = family_data.pop('members', None) # ToDo: handle error if no members
db_family = models.Family(**family_data)
db.add(db_family)
db.commit()
db.refresh(db_family)
# get family_id
family_id = db_family.id
# add members
for m in member_data:
m['family_id'] = family_id
db_member = models.Member(**m)
db.add(db_member)
db.commit()
db.refresh(db_member)
return db_family
Hope, this may be useful to someone else.
I'm building an app for rating beers at an event. The beers one can rate should be added to a table, as well should the event be added to another table and the beers and the event should be connected. Since at an event there is more than just one beer to be tasted and a beer can be tasted at multiple events, I want to make a m:n-relationship. I'm doing this with python3, I'm using flask and flasksqlalchemy. I'm using an sqlite-database.
The model I builded sofar looks like this:
#association table
event_beer = db.Table('event_beer',
db.Column('event_id', db.Integer, db.ForeignKey('event.id'), primary_key=True),
db.Column('beer_id', db.Integer, db.ForeignKey('beer.id'), primary_key=True))
class Event(db.Model):
__tablename__ = 'event'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(80), nullable=False)
def __str__(self):
return f'{name}'
class Beer(db.Model):
__tablename__ = 'beer'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(80), nullable=False)
event = db.relationship('Event', secondary=event_beer)
def __str__(self):
return f'{self.name}, {self.event}'
I omitted a few Columns which don't have any Foreign Keys or so for the sake of simplicity. The code which is executed when I want to save the recorded data is:
event = Event(name = 'event_name')
beer1 = Beer(name = 'beerone')
beer2 = Beer(name = 'beertwo')
beer1.event.append(event)
beer2.event.append(event)
db.session.commit()
The values to be saved aren't strings, but for the sake of simplicity I replaced them. The values are there though and in the database there aren't any empty rows.
I don't know whether I set up the model wrong or whether it's an issue while committing. Any help would be appreciated.
I'm deeply sorry, I just found the problem. Obviously I forgot to add the items to the session. All that missed were db.session.add(event). I was trying to figure this out for at least 6 hours now but I just found it after I posted the problem to stackoverflow.