I'm trying to compare 2 lists of dictionaries.
Please find an example below.
list1 = [
{'code': '1111', 'description': 'Test'},
{'code': '2222', 'description': 'Hello World'},
{'code': '3333', 'description': 'Stack'},
{'code': '4444', 'description': 'Gozilla'},
]
list2 = [
{'code': '3333', 'description': 'Stack'},
{'code': '4444', 'description': 'Megatron'},
{'code': '5555', 'description': 'Winnie the Pooh'}
]
I am trying to :
If ['code'] from list2 exist in list1, and if ['description'] is different, place it in a new list "updates".
If ['code'] from list2 does not exist in list1, place it in a new list "new".
At the end the 2 new lists from my example should look like that :
updates = [
{'code': '4444', 'description': 'Megatron'}
]
new = [
{'code': '5555', 'description': 'Winnie the Pooh'}
]
Any ideas how I could achieve that ?
You can convert list1 to dict to make comparing codes easier:
list1 = [
{'code': '1111', 'description': 'Test'},
{'code': '2222', 'description': 'Hello World'},
{'code': '3333', 'description': 'Stack'},
{'code': '4444', 'description': 'Gozilla'},
]
list2 = [
{'code': '3333', 'description': 'Stack'},
{'code': '4444', 'description': 'Megatron'},
{'code': '5555', 'description': 'Winnie the Pooh'}
]
updates = []
new = []
tmp = {d['code']: d['description'] for d in list1}
for d in list2:
if d['code'] in tmp and d['description'] != tmp[d['code']]:
updates.append(d)
elif not d['code'] in tmp:
new.append(d)
print(updates)
print(new)
Prints:
[{'code': '4444', 'description': 'Megatron'}]
[{'code': '5555', 'description': 'Winnie the Pooh'}]
#Otherwise,for example 4444 in between of list1 and list2
new_list = []
if list1[3].get('code')==list2[1].get('code'):
new_list.append(list2[1])
print(new_list)
Related
I have the following structure, which I convert from a .txt with pandas
[[000001, 'PEPE ', 'S', 'LAST_NAME ', 'CIP ', 'CELLPHONE'],
[0000002, 'LUIS ', 'S', 'ADRESS ', ' ', 'nan'],
[0000003, 'PEDRO ', 'S', 'STREET ', 'CITY', ' nan']]
My code
import pandas as pd
file = 'C:\\Users\\Admin\\Desktop\\PRUEBA.txt'
columns = ("service", "name", "Active", "reference1", "reference2", "reference3")
df = pd.read_csv(file, sep="|", names=columns, header=None)
cl = df.values.tolist()
print(cl)
but to be able to give it the treatment, which it requires, either by removing the empty strings and nan, how can I transform the service to int and create an object based on the service and the references in this way.
[
{ service: 1, name: 'PEPE', order: 0, ref: 'LAST_NAME' },
{ service: 1, name: 'PEPE', order: 1, ref: 'CIP' },
{ service: 1, name: 'PEPE', order: 2, ref: 'CELLPHONE' },
{ service: 2, name: 'LUIS', order: 0, ref: 'ADRESS' },
{ service: 3, name: 'PEDRO', order: 0, ref: 'STREET' },
{ service: 3, name: 'PEDRO', order: 1, ref: 'CITY' }
]
How can I achieve this, very grateful for your comments
Key: Use df.melt() to unpivot the table and subsequently perform df.to_dict(orient='records') to convert the dataframe to a record-oriented dict as mentioned by #QuangHoang. The rest are regular filtering and miscellaneous adjustments.
# data
ls = [['000001', 'PEPE ', 'S', 'LAST_NAME ', 'CIP ', 'CELLPHONE'],
['0000002', 'LUIS ', 'S', 'ADRESS ', ' ', 'nan'],
['0000003', 'PEDRO ', 'S', 'STREET ', 'CITY', ' nan']
]
df = pd.DataFrame(ls, columns=("service", "name", "Active", "reference1", "reference2", "reference3"))
# reformat and strip over each column
for col in df:
if col == "service":
df[col] = df[col].astype(int)
else:
df[col] = df[col].str.strip() # accessor
# unpivot and adjust
df2 = df.melt(id_vars=["service", "name"],
value_vars=["reference1", "reference2", "reference3"],
value_name="ref")\
.sort_values(by="service")\
.drop("variable", axis=1)\
.reset_index(drop=True)
# filter out empty or nan
df2 = df2[~df2["ref"].isin(["", "nan"])]
# generate order numbering by group
df2["order"] = df2.groupby("service").cumcount()
df2 = df2[["service", "name", "order", "ref"]] # reorder
# convert to a record-oriented dict
df2.to_dict(orient='records')
Out[99]:
[{'service': 1, 'name': 'PEPE', 'order': 0, 'ref': 'LAST_NAME'},
{'service': 1, 'name': 'PEPE', 'order': 1, 'ref': 'CIP'},
{'service': 1, 'name': 'PEPE', 'order': 2, 'ref': 'CELLPHONE'},
{'service': 2, 'name': 'LUIS', 'order': 0, 'ref': 'ADRESS'},
{'service': 3, 'name': 'PEDRO', 'order': 0, 'ref': 'STREET'},
{'service': 3, 'name': 'PEDRO', 'order': 1, 'ref': 'CITY'}]
I have this schema
from marshmallow import validate, ValidationError
from marshmallow_jsonapi import fields
from marshmallow_jsonapi.flask import Relationship, Schema
class UserSchema(Schema):
first_name = fields.Str(required=True])
last_name = fields.Str(required=True)
title = fields.Str(required=True)
class Meta:
type_ = 'users'
self_view = "blog_view.users_detail"
self_view_kwargs = {"user_id": "<id>", "_external": True}
self_view_many = "blog_view.users_list"
blog= Relationship(
many=False,
include_data=True,
type_="blogs",
include_resource_linkage=True,
schema="BlogSchema"
)
I want to load this data(coming from UI) for validation:
bulk_data = [
{ 'type': 'users',
'relationships': {'blog': {'data': {'type': 'blogs', 'id': blog_id}}},
{'first_name': 'Billy', 'last_name': 'Butcher', 'title': 'Supe Hunter'}
},
{ 'type': 'users',
'relationships': {'blog': {'data': {'type': 'blogs', 'id': blog_id}}},
{'first_name': 'Home', 'last_name': 'Lander', 'title': 'Leader'}
},
{ 'type': 'users',
'relationships': {'blog': {'data': {'type': 'blogs', 'id': blog_id}}},
{'first_name': 'Black', 'last_name': 'Noir', 'title': 'Super Ninja'}
}
]
For validation I did:
data = UserSchema(many=True).load(input_data)
I get an error saying,
AttributeError: 'list' object has no attribute 'get'
which is obvious because I'm passing a list. The validation works fine when I pass a single dictionary from the above list, but I want to pass the bulk data and do validation at once as shown in Marshmallow doc: https://marshmallow.readthedocs.io/en/stable/quickstart.html#validation
When
many=True
, load method expects a collection type so list, tuple, queryset etc.
Any suggestion on how to validate a list of data in Marshmallow? The marshmallow versions are:
marshmallow==2.18.0
marshmallow-jsonapi==0.23.1
Thanks!
I am new to python and trying to print value with assert function in python.
Below is example code:
HEADERS_VALID = {'Content-Type': 'application/json', 'fiware-Service': 'test', 'fiware-ServicePath': '/t1'}
HEADERS_INVALID = {'Content-Type': 'application/json', 'fiwareService': 'test', 'fiwareServicePath': '/t1'}
def test_for_valid_headers(notification):
notification['data'][0] = {
'id': 'Room0',
'type': 'Room',
'temperature': {'type': 'Number', 'value': '100', 'metadata': {'dateModified': {'type': 'DateTime','value': '1980-01-30T00:00:00.000+00:00'}}},
'pressure': {'type': 'Number', 'value': '10', 'metadata': {'dateModified': {'type': 'DateTime','value': '1980-01-30T00:00:00.000+00:00'}}},
}
res_post = requests.post(data=json.dumps(notification),
headers=HEADERS_VALID)
time.sleep(1)
assert res_post.status_code == 200
assert res_post.json() == 'Notification successfully processed'
get_url = "{}/entities/Room0".format(QL_URL)
res_get = requests.get(get_url, headers=HEADERS_VALID)
assert res_get.status_code == 200
exp_values = {
"attributes": [{'attrName': 'pressure', 'values': [10.0]}, {'attrName':
'temperature', 'values': [100.0]}],
"entityId": 'Room0',
"index": [
'1980-01-30T00:00:00.000+00:00'
]
}
assert res_get.json() == exp_values
Here in res_post.json() the actual response is "Notification successfully processed for : 'tenant' test, 'fiwareServicePath' /t1, 'entity_id' ['Room0']" which is not equal to 'Notification successfully processed' in assert function. So, want to add rest part in assert fucntion.
After this response i added ,
assert res_post.json() == "Notification successfully processed for: : 'tenant' %s, 'fiwareServicePath' %s" %(fiware_s, fiware_sp)
which is giving error as : NameError: name 'fiware' is not defined
I am not able to get what i am doing wrong in this statement.Any help on this will be great. Thanks in advance.
I have list of dict like this:
[{
'attr': 'bla',
'status': '1',
'id': 'id1'
}, {
'attr': 'bla',
'status': '1',
'id': 'id2'
}, {
'attr': 'bli',
'status': '0',
'id': 'id1'
}, {
'attr': 'bli',
'status': '1',
'id': 'id2'
}]
I wan't to get a simpler results dict like this:
result = {
'bla' : True,
'bli' : False
}
If the two id have a 1 for an attr, the value will be True. else, it will False.
I've tried with
for elem in dict:
for key, value in enumerate(elem):
# ???
But i don't see how to do. I've alos tried something like
if all( val == '1' for val in list ):
# ..
Here you go:
dicts = [{
'attr': 'bla',
'status': '1',
'id': 'id1'
}, {
'attr': 'bla',
'status': '1',
'id': 'id2'
}, {
'attr': 'bli',
'status': '0',
'id': 'id1'
}, {
'attr': 'bli',
'status': '1',
'id': 'id2'
}]
# First run is to create all nessecary items in the
# new Dictionary so i can use the and operator on them later.
newDict = {}
for dictio in dicts:
for key, value in dictio.items():
if key == 'attr':
newDict[value] = True
# The second run uses the and operator
for dictio in dicts:
for key, value in dictio.items():
if key == 'attr':
tmpAttr = value
if key == 'status':
newDict[tmpAttr] = newDict[tmpAttr] and (value == '1')
print(newDict)
Have a nice day!
I have a set of data:
{ name: 'item1', timeOfDay: '36000', dayOfWeek: '1', dayOfMonth: '15', room: '1'}
{ name: 'item2', timeOfDay: '3600', dayOfWeek: '2', dayOfMonth: '10', room: '2'}
{ name: 'item1', timeOfDay: '18000', dayOfWeek: '3', dayOfMonth: '20', room: '3'}
{ name: 'item3', timeOfDay: '72000', dayOfWeek: '4', dayOfMonth: '5', room: '4'}
Given a new item i'm looking for an algorithm to find the closest items order by distance
{ name: 'item2', timeOfDay: '36000', dayOfWeek: '5', dayOfMonth: '3', room: '2'}
First I looked at kMeans to organise items around centers but I have the feeling I need something sorting the item at the good location.
Multicriteria sort ? but that could learn which criteria is the stronger ?
I don't want to do Array.sort() because:
I need to add a new item without sorting all the array
I need to merge same (closest) values