I'm new to marshmallow and flask etc.
I'm trying to learn by creating an API that consumes a jsonified python dictionary. The dictionary contains nested dictionaries like this. It also contains a few Null items.
{
"TITLE": "XXX",
"NAME": "Mongoose",
"TIME": "0430",
"USED": null,
"DESCRIPTION": "",
"WEAPONS": {
"HEAT": "Israel",
"RADAR": "Flat",
"CONV": {
"S": true,
"N": false,
"A": false
},
},
}
I simply want to consume this back into a dictionary type. Something like this on the POST action
fields_schema = FieldSchema(many=True)
field_schema = FieldSchema()
json_data = request.get_json(force=True)
if not json_data:
return {'message': 'No input data provided'}, 400
# Validate and deserialize input
try:
data = field_schema.load(json_data)
except ValidationError as e:
return e.messages, 422
Where data would simply be a nested dictionary.
It is defining the schema class that is causing me problems.
From what I can tell, when defining the schema, marshmallow doesnt have a JSON type and when I use fields.Dict I get the following error:
{
"meta": [
"Missing data for required field."
],
"TITLE": [
"Unknown field."
etc...
I'm not sure whether I should be looking at using a nested Schema or whether I am totally over complicating things.
My fields_schema currently looks like this:
class FieldSchema(ma.Schema):
id = fields.Integer()
meta = fields.Dict(required=True)
Any pointers would be greatly appreciated
If you're going to validate the nested object, you can use Marshmallow's fields.Nested functionality.
Using their example
from marshmallow import Schema, fields, pprint
class UserSchema(Schema):
name = fields.String()
email = fields.Email()
created_at = fields.DateTime()
class BlogSchema(Schema):
title = fields.String()
author = fields.Nested(UserSchema)
user = User(name="Monty", email="monty#python.org")
blog = Blog(title="Something Completely Different", author=user)
result = BlogSchema().dump(blog)
pprint(result)
# {'title': u'Something Completely Different',
# 'author': {'name': u'Monty',
# 'email': u'monty#python.org',
# 'created_at': '2014-08-17T14:58:57.600623+00:00'}}
Your need to define a schema from the root document though. Something like
class Widget(Schema):
TITLE = fields.String()
NAME = fields.String()
# ...
WEAPONS = fields.Nested(Weapon)
class Weapon(Schema):
HEAT = fields.String()
# ...
might get you going.
Related
I am using django-rest for my back-end and want to overwrite default errors for fields.
My current code looks like this.
class DeckSerializer(serializers.ModelSerializer):
class Meta:
model = Product
fields = (
"id",
"title",
"get_absolute_url",
"description",
"price",
"image",
"category_id",
"category",
"title"
)
extra_kwargs = {
'title': {"error_messages": {"required": "Title cannot be empty"}},
'image': {"error_messages": {"required": "Image cannot be empty"},}
}
After writing these 2 kwargs i realised i would just be repeating something that could be solved by code.
By default the serializer validation returns this when the field is missing {title:"This field is required"}.
Is there any way that i can overwrite the current message so it can display directly the name_of_the_field + my_message . Example {title: Title is required}
I am not looking on how to write custom error message for a single field , im looking on how to write generic costum messages for every field that for example is missing or null.
We can achieve it by writing a custom exception handler.
Here is how a custom response might look like:
{
"status_code": 400,
"type": "ValidationError",
"message": "Bad request syntax or unsupported method",
"errors": [
"username: This field may not be null.",
"email: This field may not be null.",
"ticket number: This field may not be null."
]
}
We have to create a file: exception_handler.py in our project directory with the code that follows; I use utils for this kind of purposes. You can also put this code anywhere you like, but I prefer to have it in a separated file dedicated for this purpose.
from http import HTTPStatus
from rest_framework import exceptions
from rest_framework.views import Response, exception_handler
def api_exception_handler(exception: Exception, context: dict) -> Response:
"""Custom API exception handler."""
# Call REST framework's default exception handler first,
# to get the standard error response.
response = exception_handler(exception, context)
# Only alter the response when it's a validation error
if not isinstance(exception, exceptions.ValidationError):
return response
# It's a validation error, there should be a Serializer
view = context.get("view", None)
serializer = view.get_serializer_class()()
errors_list = []
for key, details in response.data.items():
if key in serializer.fields:
label = serializer.fields[key].label
help_text = serializer.fields[key].help_text
for message in details:
errors_list.append("{}: {}".format(label, message))
elif key == "non_field_errors":
for message in details:
errors_list.append(message)
else:
for message in details:
errors_list.append("{}: {}".format(key, message))
# Using the description's of the HTTPStatus class as error message.
http_code_to_message = {v.value: v.description for v in HTTPStatus}
error_payload = {
"status_code": 0,
"type": "ValidationError",
"message": "",
"errors": [],
}
# error = error_payload["error"]
status_code = response.status_code
error_payload["status_code"] = status_code
error_payload["message"] = http_code_to_message[status_code]
error_payload["errors"] = errors_list
# Overwrite default exception_handler response data
response.data = error_payload
return response
The main idea comes from here, but I changed it to my needs. change it as you see fit.
Don't forget to set it as your default exception handler in you settings.py file:
REST_FRAMEWORK["EXCEPTION_HANDLER"] = "utils.exception_handler.api_exception_handler";
I'm using Django 3 and Python 3.8. I have the below model, Note the "types" ManyToMany field, in which I set "blank" to False.
class Coop(models.Model):
objects = CoopManager()
name = models.CharField(max_length=250, null=False)
types = models.ManyToManyField(CoopType, blank=False)
addresses = models.ManyToManyField(Address)
enabled = models.BooleanField(default=True, null=False)
phone = models.ForeignKey(ContactMethod, on_delete=models.CASCADE, null=True, related_name='contact_phone')
email = models.ForeignKey(ContactMethod, on_delete=models.CASCADE, null=True, related_name='contact_email')
web_site = models.TextField()
I want verify a validation error occurs if I leave that field blank, so I have
#pytest.mark.django_db
def test_coop_create_with_no_types(self):
""" Verify can't create coop if no """
coop = CoopFactory.create(types=[])
self.assertIsNotNone(coop)
self.assertNone( coop.id )
and use the following factory (with FactoryBoy) to build the model
class CoopFactory(factory.DjangoModelFactory):
"""
Define Coop Factory
"""
class Meta:
model = Coop
name = "test model"
enabled = True
phone = factory.SubFactory(PhoneContactMethodFactory)
email = factory.SubFactory(EmailContactMethodFactory)
web_site = "http://www.hello.com"
#factory.post_generation
def addresses(self, create, extracted, **kwargs):
if not create:
# Simple build, do nothing.
return
if extracted:
# A list of types were passed in, use them
for address in extracted:
self.addresses.add(address)
else:
address = AddressFactory()
self.addresses.add( address )
#factory.post_generation
def types(self, create, extracted, **kwargs):
if not create:
# Simple build, do nothing.
return
if extracted:
# A list of types were passed in, use them
for _ in range(extracted):
self.types.add(CoopTypeFactory())
However, the "self.assertNone( coop.id )" assertion fails (an ID is generated). I would expect this not to happen, since I haven't specified any types. What else do I need to do to enforce my constraint, or should I be using a different constraint?
Edit: In response to #Melvyn's suggestion, tried modifying the test to the below
#pytest.mark.django_db
def test_coop_create_with_no_types(self):
""" Test customer model """ # create customer model instance
coop = CoopFactory.build(types=[])
coop.full_clean()
self.assertIsNotNone(coop)
self.assertIsNone( coop.id )
but not only did not get a validation error for the "types," field, got validation errors for the email and phone fields, which are clearly being populated in the factory.
File "/Users/davea/Documents/workspace/chicommons/maps/web/tests/test_models.py", line 76, in test_coop_create_with_no_types
coop.full_clean()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/django/db/models/base.py", line 1221, in full_clean
raise ValidationError(errors)
django.core.exceptions.ValidationError: {'phone': ['This field cannot be blank.'], 'email': ['This field cannot be blank.']}
Edit: Per the answer given #ArakkalAbu, I implemented the suggestion (https://github.com/chicommons/maps/blob/master/web/directory/serializers.py) but this test continues to pass
#pytest.mark.django_db
def test_coop_create_no_coop_types(self):
""" Test coop serizlizer model """
name = "Test 8899"
street = "222 W. Merchandise Mart Plaza, Suite 1212"
city = "Chicago"
postal_code = "60654"
enabled = True
postal_code = "60654"
email = "test#example.com"
phone = "7732441468"
web_site = "http://www.1871.com"
state = StateFactory()
serializer_data = {
"name": name,
"types": [
],
"addresses": [{
"formatted": street,
"locality": {
"name": city,
"postal_code": postal_code,
"state": state.id
}
}],
"enabled": enabled,
"phone": {
"phone": phone
},
"email": {
"email": email
},
"web_site": web_site
}
serializer = CoopSerializer(data=serializer_data)
assert serializer.is_valid(True), serializer.errors
You can't enforce this constrain, blank=False into neither database-level nor model-level. Because every m2m relation has a record with a foreign key to both sides of the m2m relation (the through--(Django Doc) relation).
Also in m2m relations, the m2m items are linked by a separate operation under the hood.
Create CoopFactory instance
Add CoopType to Coop.types by using .add()--(django doc) or .set()--(django doc) methods
That is, you can not create a M2M relation directly by
Coop.objects.create(name='foo', types=[1, 2, 3]) # 1,2 & 3 are PKs of `CoopType`
This statement trigger an exception by saying,
TypeError: Direct assignment to the forward side of a many-to-many set is prohibited. Use types.set() instead.
What is the best bet?
as per your this comment,
I'm not using this in a form though. I'm using the model by a serializer as part of the Django rest framework.
Since you are using DRF, you can validate the incoming payload.
class CoopSerializer(serializers.ModelSerializer):
class Meta:
model = Coop
fields = '__all__'
extra_kwargs = {
'types': {
'allow_empty': False
}
}
# execution
s = CoopSerializer(data={'name': 'foo coop', 'types': []})
s.is_valid(True)
s.save()
# excption
rest_framework.exceptions.ValidationError: {'types': [ErrorDetail(string='This list may not be empty.', code='empty')]}
This will help you to enforce to have a required M2M data.
blank=True on a ManyToManyField is not translated as a DBMS constraint, but will be (for exemple) checked on form validation.
On your unit-test, you use CoopFactory.create that seems to not check this logical (and non-dbms) constraint.
See https://docs.djangoproject.com/en/3.0/ref/models/fields/#blank
Note that this is different than null. null is purely database-related, whereas blank is validation-related. If a field has blank=True, form validation will allow entry of an empty value. If a field has blank=False, the field will be required.
Hello im trying to get this code to work and output nicknames from a json list, but some of them are null ('nickname' object doesnt exits on the 'profiles' array) but I still need them to be output as None and not simply supress. This code I wrote works but I have to do it 27 more times and it would become huge and is there a way to shorten it?
data is a JSON array, example
[{"nickname": "Ana", "id": 0}, {"nickname": "Sofia", "id": 1}, {"nickname": null, "id": 2}]
def userProfileList(self, data):
self.json = data
nicknames = []
for obj in data:
try: nickname = obj['nickname']
except TypeError: nickname = None
nicknames.append(nickname)
self.nickname = nicknames
As your JSON object contains 'null', which isn't a keyword in Python, I'll assume that you're getting it from an external source and so I'll also assume that you're parsing it with the json.loads() function, which will automatically turn null into None.
You can use list comprehension to generate the list of nicknames:
nicknames = [index['nickname'] for index in data]
Example
import json
raw = '[{"nickname": "Ana", "id": 0}, {"nickname": "Sofia", "id": 1}, {"nickname": null, "id": 2}]'
data = json.loads(raw)
nicknames = [index['nickname'] for index in data]
print(nicknames)
Returns ['Ana', 'Sofia', None]
I have a table like this:
class Mapping(db.Model):
map_id = db.Column(db.Integer, primary_key=True)
user_id = db.Column(db.Integer)
bike_id = db.Column(db.String(255))
is_active_data = db.Column(db.Boolean, default=True)
created_by = db.Column(db.String(150))
updated_by = db.Column(db.String(150))
My POST method:
def save_MapperM(adddata):
create_data = Mapping(**adddata)
db.session.add(create_data)
db.session.commit()
return dict(Successful="Successfully Created")
Route:
#test.route("/addmapper"))
class MapperV(Resource):
#staticmethod
def post():
if request.method == 'POST':
save_data = request.get_json()
try:
return jsonify(save_MapperM(save_data))
except Exception:
return jsonify({'Unsuccessful': 'Looks like you missed something !!'})
Current Code :
The current code will take only one bike_id for every request.
Requirements:
I want to take multiple bike_id's as for one user id and store it as multiple records in the table level.
Example data format coming from UI:
{ user_id: 1, bike_id: 1,2,3,4 }
The easiest solution is to modify your save_MapperM function with a cycle:
def save_MapperM(adddata):
for bike_id in adddata["bike_id"]:
item_data = adddata.copy()
item_data["bike_id"] = bike_id
create_data = Mapping(**item_data)
db.session.add(create_data)
db.session.commit()
return dict(Successful="Successfully Created")
But be careful with this function as it allows to create Mapping instances with any parameters received from the POST request. It looks like it is intended but it can lead to security issues if your Mapping class has some private attributes which should be filled only on the server side. For example the user "Mike" can send a request:
{ "user_id": 1, "bike_id": [1, 2], "created_by": "Alex", "updated_by": "Alex" }
It will cause the save_MapperM function to create instances with created_by and updated_by values set to "Alex" which may not be true. It is better to get such attributes from the session data.
So your post method may look like this (post and save_MapperM functionality combined):
def post():
request_data = request.get_json()
for bike_id in request_data.get("bike_id", []):
item = Mapping(
user_id=request_data.get("user_id"),
bike_id=bike_id,
created_by=session.get("username"),
updated_by=session.get("username"),
)
db.session.add(item)
try:
db.session.commit()
except Exception:
return jsonify({"success": False})
return jsonify({"success": True})
The next step may be implementing request JSON data validation. It is OK when you have a couple of JSON keys with a simple structure but when you need to pass lots of data you need to be sure it is correct. You can use some of the serialization/ODM libraries for this Marshmallow for example.
I'm trying to remove key pair values if they are not provided in my payload from SQLAlchemy query.
payload1 = {
"username": "James",
"ssn": "102-00-1911",
"state": "New York"
}
payload2 = {
"username": "James"
}
def search_database(body):
"""
Searches Database for user
"""
query = db_session.query(DatabaseModel).filter_by(
username=body['username'],
ssn=body.get('ssn', None), # kwarg should be omitted if it's not in body
state=body.get('state', None), # kwarg should be omitted if it's not in body
).all()
return query
search_database(payload1) # should not omit anything
search_database(payload2) # Should omit `ssn` and `state` from query
Currently I'm unable to find a way to dynamically remove a key pair from my query. I tried passing None as an alternative to the value if a key is missing but that did not work.
If the payload is missing ssn or state I would like to omit it from the query
I realized you can pass a dictionary in filter_by. I was able to filter using a Dict comprehension on the body before passing it to the query.
def search_database(body):
"""
Searches Database for user
"""
permitted = [
"username",
"ssn",
"state"
]
filtered = {key: body[key] for key in permitted if key in body}
query = db_session.query(DatabaseModel).filter_by(**filtered).all()
return query