Python Marshmallow AttributeError: 'list' object has no attribute 'get' - python-3.x

I have this schema
from marshmallow import validate, ValidationError
from marshmallow_jsonapi import fields
from marshmallow_jsonapi.flask import Relationship, Schema
class UserSchema(Schema):
first_name = fields.Str(required=True])
last_name = fields.Str(required=True)
title = fields.Str(required=True)
class Meta:
type_ = 'users'
self_view = "blog_view.users_detail"
self_view_kwargs = {"user_id": "<id>", "_external": True}
self_view_many = "blog_view.users_list"
blog= Relationship(
many=False,
include_data=True,
type_="blogs",
include_resource_linkage=True,
schema="BlogSchema"
)
I want to load this data(coming from UI) for validation:
bulk_data = [
{ 'type': 'users',
'relationships': {'blog': {'data': {'type': 'blogs', 'id': blog_id}}},
{'first_name': 'Billy', 'last_name': 'Butcher', 'title': 'Supe Hunter'}
},
{ 'type': 'users',
'relationships': {'blog': {'data': {'type': 'blogs', 'id': blog_id}}},
{'first_name': 'Home', 'last_name': 'Lander', 'title': 'Leader'}
},
{ 'type': 'users',
'relationships': {'blog': {'data': {'type': 'blogs', 'id': blog_id}}},
{'first_name': 'Black', 'last_name': 'Noir', 'title': 'Super Ninja'}
}
]
For validation I did:
data = UserSchema(many=True).load(input_data)
I get an error saying,
AttributeError: 'list' object has no attribute 'get'
which is obvious because I'm passing a list. The validation works fine when I pass a single dictionary from the above list, but I want to pass the bulk data and do validation at once as shown in Marshmallow doc: https://marshmallow.readthedocs.io/en/stable/quickstart.html#validation
When
many=True
, load method expects a collection type so list, tuple, queryset etc.
Any suggestion on how to validate a list of data in Marshmallow? The marshmallow versions are:
marshmallow==2.18.0
marshmallow-jsonapi==0.23.1
Thanks!

Related

Exclude a certian field or make it Null inside a AQL query

I have problem. I have a collection orders. I want to get all documents and all field except the field dataOriginSystem inside metaData.
Is there an option to exclude this value, or make it null?
SELECT * EXCEPT(ColumnNameX, [ColumnNameY, ...])
FROM TableA
This my collection orders:
[
{'_id': 'orders/213123',
'contactEditor': {'name': 'Max Power',
'phone': '1234567',
'email': 'max#power.com'},
'contactSoldToParty': {'name': 'Max Not',
'phone': '123456789',
'email': 'maxnot#power.com'},
'isCompleteDelivery': False,
'metaData': {'dataOriginSystem': 'Goods',
'dataOriginWasCreatedTime': '10:12:12',},
'orderDate': '2021-02-22',
'orderDateBuyer': '2021-02-22',
},
{'_id': 'orders/12323',
'contactEditor': {'name': 'Max Power2',
'phone': '1234567',
'email': 'max#power.com'},
'contactSoldToParty': {'name': 'Max Not',
'phone': '123456789',
'email': 'maxnot#power.com'},
'isCompleteDelivery': False,
'metaData': {'dataOriginSystem': 'Goods',
'dataOriginWasCreatedTime': '10:12:12',},
'orderDate': '2021-02-22',
'orderDateBuyer': '2021-02-22',
},
]
AQL
FOR doc IN orders RETURN doc
I want something like
FOR doc IN orders RETURN doc WITHOUT metaData_dataOriginSystem
FOR doc IN orders RETURN doc AND metaData_dataOriginSystem = NULL
So that the returned documents do not have the field metaData_dataOriginSystem or the field should be Null
What I want (the other fields should all exist!)
[
{'_id': 'orders/213123',
...
'isCompleteDelivery': False,
'metaData': {<Removed>,
'dataOriginWasCreatedTime': '10:12:12',},
...
},
OR
{'_id': 'orders/12323',
...
'isCompleteDelivery': False,
'metaData': {'dataOriginSystem': Null,
'dataOriginWasCreatedTime': '10:12:12',},
...
},
]
Use UNSET() or UNSET_RECURSIVE().
UNSET(document, attributeName1, attributeName2, ... attributeNameN) → doc
Remove the attributes attributeName1 to attributeNameN from document. All other attributes will be preserved.
FOR doc IN orders RETURN UNSET( doc, "orderData", "dataOriginSystem")
If you want to remove a nested value, you need to Merge
FOR doc IN orders RETURN MERGE(doc, {metaData: UNSET(doc.metaData, "dataOriginSystem")})

Flatterning JSON to Pandas Dataframe

I'm trying to flattern this json into a pandas dataframe, but it's getting the better of me.
[{
'contact': {
'id': 101,
'email': 'email1#address.com',
},
'marketingPreference': [{
'marketingId': 1093,
'isOptedIn': True,
'dateModifed': '2022-05-10T14:29:24Z'
}]
},
{
'contact': {
'id': 102,
'email': 'email2#address.com',
},
'marketingPreference': [{
'marketingId': 1093,
'isOptedIn': True,
'dateModifed': '2022-05-10T14:29:24Z'
}]
}
]
I am looking for the columns to be: Id, Email, MarketingId, IsOptedIn, DateModifed.
Even though marketingPreference is an array, there is only ever one json object inside.
You can use pd.json_normalize
df = pd.json_normalize(data, record_path='marketingPreference', meta=[['contact', 'id'], ['contact', 'email']])
print(df)
marketingId isOptedIn dateModifed contact.id contact.email
0 1093 True 2022-05-10T14:29:24Z 101 email1#address.com
1 1093 True 2022-05-10T14:29:24Z 102 email2#address.com

How to print value with assert in python

I am new to python and trying to print value with assert function in python.
Below is example code:
HEADERS_VALID = {'Content-Type': 'application/json', 'fiware-Service': 'test', 'fiware-ServicePath': '/t1'}
HEADERS_INVALID = {'Content-Type': 'application/json', 'fiwareService': 'test', 'fiwareServicePath': '/t1'}
def test_for_valid_headers(notification):
notification['data'][0] = {
'id': 'Room0',
'type': 'Room',
'temperature': {'type': 'Number', 'value': '100', 'metadata': {'dateModified': {'type': 'DateTime','value': '1980-01-30T00:00:00.000+00:00'}}},
'pressure': {'type': 'Number', 'value': '10', 'metadata': {'dateModified': {'type': 'DateTime','value': '1980-01-30T00:00:00.000+00:00'}}},
}
res_post = requests.post(data=json.dumps(notification),
headers=HEADERS_VALID)
time.sleep(1)
assert res_post.status_code == 200
assert res_post.json() == 'Notification successfully processed'
get_url = "{}/entities/Room0".format(QL_URL)
res_get = requests.get(get_url, headers=HEADERS_VALID)
assert res_get.status_code == 200
exp_values = {
"attributes": [{'attrName': 'pressure', 'values': [10.0]}, {'attrName':
'temperature', 'values': [100.0]}],
"entityId": 'Room0',
"index": [
'1980-01-30T00:00:00.000+00:00'
]
}
assert res_get.json() == exp_values
Here in res_post.json() the actual response is "Notification successfully processed for : 'tenant' test, 'fiwareServicePath' /t1, 'entity_id' ['Room0']" which is not equal to 'Notification successfully processed' in assert function. So, want to add rest part in assert fucntion.
After this response i added ,
assert res_post.json() == "Notification successfully processed for: : 'tenant' %s, 'fiwareServicePath' %s" %(fiware_s, fiware_sp)
which is giving error as : NameError: name 'fiware' is not defined
I am not able to get what i am doing wrong in this statement.Any help on this will be great. Thanks in advance.

Flask/Pymongo Where/How do I apply validation?

This does not seem to apply validation to the collection. No exceptions are thrown and documents can have attributes of the wrong type. Perhaps I am doing it in the wrong section of the codebase? Right now it is in __init__.py
__init__.py
db = database(client, settings.mongo_db_name)
from api.models import Company
validation_level = 'strict'
if 'companies' not in db.collection_names():
db.create_collection('companies', validator=Company.validator, validationLevel=validation_level)
else:
db.command({
'collMod': 'companies',
'validator': Company.validator,
'validationLevel': validation_level,
})
Company Model:
from api import db
class Company(Model):
collection = db.companies
validator = {
'$jsonSchema': {
'bsonType': 'object',
'required': ['name', 'description'],
'properties': {
'logo': {
'bsonType': 'string',
},
'name': {
'bsonType': 'string',
'description': 'name of company is required',
'minLength': 4,
},
'description': {
'bsonType': 'string',
'description': 'description of company is required',
'minLength': 4,
},
'website': {
'bsonType': 'string',
},
'request_delete': {
'bsonType': 'bool',
},
'deleted': {
'bsonType': 'bool',
},
}
}
}
As discussed here, I also tried this without success:
db.command(OrderedDict([
('collMod', 'companies'),
('validator', Company.validator),
('validationLevel', validation_level),
]))
If this were successful, would I see validation rules when running the following command?
pprint(db.command('collstats', 'companies'))
Update
I added OrderedDict to both command arguments and the validator. This works... when I run specific tests. It does not work with
python -m unittest discover
I'm using Python 3.6.8, PyMongo 3.8.0, and MongoDB 3.6.3
Usually, it's better to validate a document BEFORE inserting it to the database. In this case, you'll omit errors in schema

Transform a list of dict to an simpler dict

I have list of dict like this:
[{
'attr': 'bla',
'status': '1',
'id': 'id1'
}, {
'attr': 'bla',
'status': '1',
'id': 'id2'
}, {
'attr': 'bli',
'status': '0',
'id': 'id1'
}, {
'attr': 'bli',
'status': '1',
'id': 'id2'
}]
I wan't to get a simpler results dict like this:
result = {
'bla' : True,
'bli' : False
}
If the two id have a 1 for an attr, the value will be True. else, it will False.
I've tried with
for elem in dict:
for key, value in enumerate(elem):
# ???
But i don't see how to do. I've alos tried something like
if all( val == '1' for val in list ):
# ..
Here you go:
dicts = [{
'attr': 'bla',
'status': '1',
'id': 'id1'
}, {
'attr': 'bla',
'status': '1',
'id': 'id2'
}, {
'attr': 'bli',
'status': '0',
'id': 'id1'
}, {
'attr': 'bli',
'status': '1',
'id': 'id2'
}]
# First run is to create all nessecary items in the
# new Dictionary so i can use the and operator on them later.
newDict = {}
for dictio in dicts:
for key, value in dictio.items():
if key == 'attr':
newDict[value] = True
# The second run uses the and operator
for dictio in dicts:
for key, value in dictio.items():
if key == 'attr':
tmpAttr = value
if key == 'status':
newDict[tmpAttr] = newDict[tmpAttr] and (value == '1')
print(newDict)
Have a nice day!

Resources