I have this schema
from marshmallow import validate, ValidationError
from marshmallow_jsonapi import fields
from marshmallow_jsonapi.flask import Relationship, Schema
class UserSchema(Schema):
first_name = fields.Str(required=True])
last_name = fields.Str(required=True)
title = fields.Str(required=True)
class Meta:
type_ = 'users'
self_view = "blog_view.users_detail"
self_view_kwargs = {"user_id": "<id>", "_external": True}
self_view_many = "blog_view.users_list"
blog= Relationship(
many=False,
include_data=True,
type_="blogs",
include_resource_linkage=True,
schema="BlogSchema"
)
I want to load this data(coming from UI) for validation:
bulk_data = [
{ 'type': 'users',
'relationships': {'blog': {'data': {'type': 'blogs', 'id': blog_id}}},
{'first_name': 'Billy', 'last_name': 'Butcher', 'title': 'Supe Hunter'}
},
{ 'type': 'users',
'relationships': {'blog': {'data': {'type': 'blogs', 'id': blog_id}}},
{'first_name': 'Home', 'last_name': 'Lander', 'title': 'Leader'}
},
{ 'type': 'users',
'relationships': {'blog': {'data': {'type': 'blogs', 'id': blog_id}}},
{'first_name': 'Black', 'last_name': 'Noir', 'title': 'Super Ninja'}
}
]
For validation I did:
data = UserSchema(many=True).load(input_data)
I get an error saying,
AttributeError: 'list' object has no attribute 'get'
which is obvious because I'm passing a list. The validation works fine when I pass a single dictionary from the above list, but I want to pass the bulk data and do validation at once as shown in Marshmallow doc: https://marshmallow.readthedocs.io/en/stable/quickstart.html#validation
When
many=True
, load method expects a collection type so list, tuple, queryset etc.
Any suggestion on how to validate a list of data in Marshmallow? The marshmallow versions are:
marshmallow==2.18.0
marshmallow-jsonapi==0.23.1
Thanks!
Related
I have problem. I have a collection orders. I want to get all documents and all field except the field dataOriginSystem inside metaData.
Is there an option to exclude this value, or make it null?
SELECT * EXCEPT(ColumnNameX, [ColumnNameY, ...])
FROM TableA
This my collection orders:
[
{'_id': 'orders/213123',
'contactEditor': {'name': 'Max Power',
'phone': '1234567',
'email': 'max#power.com'},
'contactSoldToParty': {'name': 'Max Not',
'phone': '123456789',
'email': 'maxnot#power.com'},
'isCompleteDelivery': False,
'metaData': {'dataOriginSystem': 'Goods',
'dataOriginWasCreatedTime': '10:12:12',},
'orderDate': '2021-02-22',
'orderDateBuyer': '2021-02-22',
},
{'_id': 'orders/12323',
'contactEditor': {'name': 'Max Power2',
'phone': '1234567',
'email': 'max#power.com'},
'contactSoldToParty': {'name': 'Max Not',
'phone': '123456789',
'email': 'maxnot#power.com'},
'isCompleteDelivery': False,
'metaData': {'dataOriginSystem': 'Goods',
'dataOriginWasCreatedTime': '10:12:12',},
'orderDate': '2021-02-22',
'orderDateBuyer': '2021-02-22',
},
]
AQL
FOR doc IN orders RETURN doc
I want something like
FOR doc IN orders RETURN doc WITHOUT metaData_dataOriginSystem
FOR doc IN orders RETURN doc AND metaData_dataOriginSystem = NULL
So that the returned documents do not have the field metaData_dataOriginSystem or the field should be Null
What I want (the other fields should all exist!)
[
{'_id': 'orders/213123',
...
'isCompleteDelivery': False,
'metaData': {<Removed>,
'dataOriginWasCreatedTime': '10:12:12',},
...
},
OR
{'_id': 'orders/12323',
...
'isCompleteDelivery': False,
'metaData': {'dataOriginSystem': Null,
'dataOriginWasCreatedTime': '10:12:12',},
...
},
]
Use UNSET() or UNSET_RECURSIVE().
UNSET(document, attributeName1, attributeName2, ... attributeNameN) → doc
Remove the attributes attributeName1 to attributeNameN from document. All other attributes will be preserved.
FOR doc IN orders RETURN UNSET( doc, "orderData", "dataOriginSystem")
If you want to remove a nested value, you need to Merge
FOR doc IN orders RETURN MERGE(doc, {metaData: UNSET(doc.metaData, "dataOriginSystem")})
I'm trying to flattern this json into a pandas dataframe, but it's getting the better of me.
[{
'contact': {
'id': 101,
'email': 'email1#address.com',
},
'marketingPreference': [{
'marketingId': 1093,
'isOptedIn': True,
'dateModifed': '2022-05-10T14:29:24Z'
}]
},
{
'contact': {
'id': 102,
'email': 'email2#address.com',
},
'marketingPreference': [{
'marketingId': 1093,
'isOptedIn': True,
'dateModifed': '2022-05-10T14:29:24Z'
}]
}
]
I am looking for the columns to be: Id, Email, MarketingId, IsOptedIn, DateModifed.
Even though marketingPreference is an array, there is only ever one json object inside.
You can use pd.json_normalize
df = pd.json_normalize(data, record_path='marketingPreference', meta=[['contact', 'id'], ['contact', 'email']])
print(df)
marketingId isOptedIn dateModifed contact.id contact.email
0 1093 True 2022-05-10T14:29:24Z 101 email1#address.com
1 1093 True 2022-05-10T14:29:24Z 102 email2#address.com
I am new to python and trying to print value with assert function in python.
Below is example code:
HEADERS_VALID = {'Content-Type': 'application/json', 'fiware-Service': 'test', 'fiware-ServicePath': '/t1'}
HEADERS_INVALID = {'Content-Type': 'application/json', 'fiwareService': 'test', 'fiwareServicePath': '/t1'}
def test_for_valid_headers(notification):
notification['data'][0] = {
'id': 'Room0',
'type': 'Room',
'temperature': {'type': 'Number', 'value': '100', 'metadata': {'dateModified': {'type': 'DateTime','value': '1980-01-30T00:00:00.000+00:00'}}},
'pressure': {'type': 'Number', 'value': '10', 'metadata': {'dateModified': {'type': 'DateTime','value': '1980-01-30T00:00:00.000+00:00'}}},
}
res_post = requests.post(data=json.dumps(notification),
headers=HEADERS_VALID)
time.sleep(1)
assert res_post.status_code == 200
assert res_post.json() == 'Notification successfully processed'
get_url = "{}/entities/Room0".format(QL_URL)
res_get = requests.get(get_url, headers=HEADERS_VALID)
assert res_get.status_code == 200
exp_values = {
"attributes": [{'attrName': 'pressure', 'values': [10.0]}, {'attrName':
'temperature', 'values': [100.0]}],
"entityId": 'Room0',
"index": [
'1980-01-30T00:00:00.000+00:00'
]
}
assert res_get.json() == exp_values
Here in res_post.json() the actual response is "Notification successfully processed for : 'tenant' test, 'fiwareServicePath' /t1, 'entity_id' ['Room0']" which is not equal to 'Notification successfully processed' in assert function. So, want to add rest part in assert fucntion.
After this response i added ,
assert res_post.json() == "Notification successfully processed for: : 'tenant' %s, 'fiwareServicePath' %s" %(fiware_s, fiware_sp)
which is giving error as : NameError: name 'fiware' is not defined
I am not able to get what i am doing wrong in this statement.Any help on this will be great. Thanks in advance.
This does not seem to apply validation to the collection. No exceptions are thrown and documents can have attributes of the wrong type. Perhaps I am doing it in the wrong section of the codebase? Right now it is in __init__.py
__init__.py
db = database(client, settings.mongo_db_name)
from api.models import Company
validation_level = 'strict'
if 'companies' not in db.collection_names():
db.create_collection('companies', validator=Company.validator, validationLevel=validation_level)
else:
db.command({
'collMod': 'companies',
'validator': Company.validator,
'validationLevel': validation_level,
})
Company Model:
from api import db
class Company(Model):
collection = db.companies
validator = {
'$jsonSchema': {
'bsonType': 'object',
'required': ['name', 'description'],
'properties': {
'logo': {
'bsonType': 'string',
},
'name': {
'bsonType': 'string',
'description': 'name of company is required',
'minLength': 4,
},
'description': {
'bsonType': 'string',
'description': 'description of company is required',
'minLength': 4,
},
'website': {
'bsonType': 'string',
},
'request_delete': {
'bsonType': 'bool',
},
'deleted': {
'bsonType': 'bool',
},
}
}
}
As discussed here, I also tried this without success:
db.command(OrderedDict([
('collMod', 'companies'),
('validator', Company.validator),
('validationLevel', validation_level),
]))
If this were successful, would I see validation rules when running the following command?
pprint(db.command('collstats', 'companies'))
Update
I added OrderedDict to both command arguments and the validator. This works... when I run specific tests. It does not work with
python -m unittest discover
I'm using Python 3.6.8, PyMongo 3.8.0, and MongoDB 3.6.3
Usually, it's better to validate a document BEFORE inserting it to the database. In this case, you'll omit errors in schema
I have list of dict like this:
[{
'attr': 'bla',
'status': '1',
'id': 'id1'
}, {
'attr': 'bla',
'status': '1',
'id': 'id2'
}, {
'attr': 'bli',
'status': '0',
'id': 'id1'
}, {
'attr': 'bli',
'status': '1',
'id': 'id2'
}]
I wan't to get a simpler results dict like this:
result = {
'bla' : True,
'bli' : False
}
If the two id have a 1 for an attr, the value will be True. else, it will False.
I've tried with
for elem in dict:
for key, value in enumerate(elem):
# ???
But i don't see how to do. I've alos tried something like
if all( val == '1' for val in list ):
# ..
Here you go:
dicts = [{
'attr': 'bla',
'status': '1',
'id': 'id1'
}, {
'attr': 'bla',
'status': '1',
'id': 'id2'
}, {
'attr': 'bli',
'status': '0',
'id': 'id1'
}, {
'attr': 'bli',
'status': '1',
'id': 'id2'
}]
# First run is to create all nessecary items in the
# new Dictionary so i can use the and operator on them later.
newDict = {}
for dictio in dicts:
for key, value in dictio.items():
if key == 'attr':
newDict[value] = True
# The second run uses the and operator
for dictio in dicts:
for key, value in dictio.items():
if key == 'attr':
tmpAttr = value
if key == 'status':
newDict[tmpAttr] = newDict[tmpAttr] and (value == '1')
print(newDict)
Have a nice day!