group dictionaries and get count - python-3.x

I have a list of dictionaries like this:
list1 = [{'name': 'maik','is_payed': 1, 'brand': 'HP', 'count': 1, 'items': [{'device': 'mouse', 'count': 110}]},{'name': 'milanie','is_payed': 0, 'brand': 'dell', 'count':10, 'items': [{'device': 'bales', 'count': 200}]}]
list2 = [{'name': 'maik','is_payed': 0, 'brand': 'HP', 'count': 20, 'items': [{'device': 'mouse', 'count': 1}]},{'name': 'nikola','is_payed': 1, 'brand': 'toshiba', 'count':10, 'items': [{'device': 'hard', 'count': 20}]}]
my_list= list1 + list2
count = pd.DataFrame(my_list).groupby(['name', 'is_payed'])
final_list_ = []
for commande, group in count:
print(commande)
records = group.to_dict("records")
final_list_.append({"name": commande[0],
"payed": commande[1],
"occurrence": len(group),
"items": pd.DataFrame(records).groupby('device').agg(
occurrence=('device', 'count')).reset_index().to_dict('records')})
I don't know how can I get it like this:
the 'payed' field is like this payed/total_commands
for example lets take maik he has two commands one is payed and the other one is not, so the final result will be like this:
{'name': 'maik','payed': 1/2, 'brand': 'HP', 'count': 21, 'items': [{'device': 'mouse', 'count': 111}]}

Since you just want to group by "name" and are only interested in the "played" values, let's concentrate on that and ignore the other data.
So for our purposes, your starting data looks like:
my_list = [
{'name': 'maik', 'is_payed': 1},
{'name': 'milanie', 'is_payed': 0},
{'name': 'maik', 'is_payed': 0},
{'name': 'nikola', 'is_payed': 1}
]
Now let's take a first pass over this data and count up the number of times we see a name and the number of times that name corresponds to an "is_payed" flag
results = {}
for item in my_list:
key = item["name"]
results.setdefault(key, {"sum": 0, "count": 0})
results[key]["count"] += 1
results[key]["is_payed"] += item["is_payed"]
At this point we have a dictionary that will look like:
{
'maik': {'is_payed': 1, 'count': 2},
'milanie': {'is_payed': 0, 'count': 1},
'nikola': {'is_payed': 1, 'count': 1}
}
Now we will take a pass over this dictionary and create our true final result:
results = [
{"name": key, "payed": f"{value['is_payed']}/{value['count']}"}
for key, value in results.items()
]
Giving us:
[
{'name': 'maik', 'payed': '1/2'},
{'name': 'milanie', 'payed': '0/1'},
{'name': 'nikola', 'payed': '1/1'}
]

Related

Restructure TSV to list of list of dicts

A simplified look at my data right at parse:
[
{'id':'group1'},
{'id':'member1', 'parentId':'group1', 'size':51},
{'id':'member2', 'parentId':'group1', 'size':16},
{'id':'group2'},
{'id':'member1', 'parentId':'group2', 'size':21},
...
]
The desired output should be like this:
data =
[
[
{'id':'group1'},
{'id':'member1', 'parentId':'group1', 'size':51},
{'id':'member2', 'parentId':'group1', 'size':16}
],
[
{'id':'group2'},
{'id':'member1', 'parentId':'group2', 'size':21},
]
]
The issue is that it's very challenging to iterate through this kind of data structure because each list contains a different length of possible objects: some might have 10 some might have 3, making it unclear when to begin and end each list. And it's also not uniform. Note some have only 'id' entries and no 'parentId' or 'size' entries.
master_data = []
for i in range(len(tsv_data)):
temp = {}
for j in range(?????):
???
How can Python handle arranging vanilla .tsv data into a list of lists as seen above?
I thought one appropriate direction to take the code was to see if I could tally something simple, before tackling the whole data set. So I attempted to compute a count of all occurences of group1, based off this discussion:
group_counts = {}
for member in data:
group = member.get('group1')
try:
group_counts[group] += 1
except KeyError:
group_counts[group] = 1
However, this returned:
'list' object has no attribute 'get'
Which leads me to believe that counting text occurences may not be the solution afterall.
You could fetch all groups to create the new datastructure afterwards add all the items:
data = [
{
'id': 'group1'
}, {
'id': 'member1',
'parentId': 'group1',
'size': 51
}, {
'id': 'member2',
'parentId': 'group1',
'size': 16
}, {
'id': 'group2'
}, {
'id': 'member1',
'parentId': 'group2',
'size': 21
}, {
'id': 'member3',
'parentId': 'group1',
'size': 16
}
]
result = {} # Use a dict for easier grouping.
lastGrpId = 0
# extract all groups
for dct in data:
if 'group' in dct['id']:
result[dct['id']] = [dct]
# extract all items and add to groups
for dct in data:
if 'parentId' in dct:
result[dct['parentId']].append(dct)
nestedListResult = [v for k, v in result.items()]
Out:
[
[
{
'id': 'group1'
}, {
'id': 'member1',
'parentId': 'group1',
'size': 51
}, {
'id': 'member2',
'parentId': 'group1',
'size': 16
}, {
'id': 'member3',
'parentId': 'group1',
'size': 16
}
], [{
'id': 'group2'
}, {
'id': 'member1',
'parentId': 'group2',
'size': 21
}]
]

Python: Add a Key-Value pair when a condition is true

Starting point are the following lists of dicts:
product = [
{'_id': '5678', 'variantIds':[{'id':'1'},{'id':'2'},{'id':'3'}]},
{'_id': '1234', 'variantIds':[{'id':'1'},{'id':'2'},{'id':'3'}]}
]
inventoryItem = [
{'_id': 'a6fdcf69', 'productId': '1234', 'variants': [{'variantId': '1', 'quantity': 0}, {'variantId': '2', 'quantity': 100}]},
{'_id': 'a6fdcf70', 'productId': '5678', 'variants': [{'variantId': '1', 'quantity': 0}, {'variantId': '2', 'quantity': 199}, {'variantId': '3', 'quantity': 299}]},
{'_id': 'a6fdcf77', 'productId': '9999', 'variants': [{'variantId': '1', 'quantity': 1111}, {'variantId': '2', 'quantity': 2222}, {'variantId': '3', 'quantity': 3333}]}
]
what i want is to add the key-value pair 'quantity':'value of quantity' to the first list of products. specifically i want to add it the the sub-list of dicts 'variantIds'. And I only want to add it when product[_'id] == inventoryItem['productId'] AND product['variantIds']['id] == inventoryItem['variants']['variantId'], so that e get the following output:
product = [
{'_id': '5678', 'variantIds':[{'id':'1', 'stockQuantity':0},{'id':'2', 'stockQuantity':199},{'id':'3', 'stockQuantity':299}]},
{'_id': '1234', 'variantIds':[{'id':'1', 'stockQuantity':0},{'id':'2', 'stockQuantity':100},{'id':'3', 'stockQuantity':0}]}
]
i can loop and add everything as long as the order in one list is corresponding to the order in the other list. but if it's not the case, what could be, i struggle with addressing the right index of the second list. how do you do that?
i think here is my closest try. but i already struggle on line 2, because i do not know the corresponding index in the inventoryItem List:
for i in product:
if i['_id'] == inventoryItem['productId']:
for j in i['variantIds']:
if j['id'] == inventoryItem['variants']['variantId']:
j['stock'] = inventoryItem['variants']['quantity']
print (product)
product = [
{'_id': '5678', 'variantIds':[{'id':'1'},{'id':'2'},{'id':'3'}]},
{'_id': '1234', 'variantIds':[{'id':'1'},{'id':'2'},{'id':'3'}]}
]
inventoryItem = [
{'_id': 'a6fdcf69', 'productId': '1234', 'variants': [{'variantId': '1', 'quantity': 0}, {'variantId': '2', 'quantity': 100}]},
{'_id': 'a6fdcf70', 'productId': '5678', 'variants': [{'variantId': '1', 'quantity': 0}, {'variantId': '2', 'quantity': 199}, {'variantId': '3', 'quantity': 299}]},
{'_id': 'a6fdcf77', 'productId': '9999', 'variants': [{'variantId': '1', 'quantity': 1111}, {'variantId': '2', 'quantity': 2222}, {'variantId': '3', 'quantity': 3333}]}
]
for prod in product:
try:
loc = [i['productId'] for i in inventoryItem].index(prod['_id'])
variantList = inventoryItem[loc]['variants']
for item in prod['variantIds']:
try:
subloc = [i['variantId'] for i in variantList].index(item['id'])
quantity = variantList[subloc]['quantity']
except:
quantity = 0
item.update({'stockQuantity':quantity})
except Exception as e:
pass
print(e.args) # you can remove this line, if this disturbs your output
print(product)
Now this may not be the best way to do it, but its pretty straightforward.
This is based on what I understood about the structure of the data. Maybe there is more to the data, and based on that it can be optimized further. But it works on the example set you provided.
What this does is, it grabs items (dictionaries) from product list. Searches for similar id in inventoryItem. Grabs that index (of the dictionary) inside inventoryItem. And then just scans to proper places in that entry of inventoryItem and then updates the product based on that.
Note : I have assumed here that there is no specific order to anything.
If there is an explanation necessary to any part, let me know.
The thing is that you mixed lists and dictionnary so you need to access to the lists of variantIds. You can do it as follow I think :
for i in range(len(product)):
for j in range(len(product[0]['variantIds'])):
product[i]['variantIds'][j]['stockQuantity'] = 0
print(product)
Output:
[{'_id': '5678', 'variantIds': [{'id': '1', 'stockQuantity': 0}, {'id': '2', 'stockQuantity': 0}, ...

Sort json list from another list

I'm trying to sort a json list from another list.
example:
jsonList = [{'id': 'das', 'name': 'something'}, {'id': 'rtn', 'name': 'Something Else'}, {'id': 'ddsn', 'name': 'Something ElseElse'}]
orderList = ['rtn', 'ddsn', 'das']
goodList = someFunction(jsonList, orderList )
I need the output to be the json list sorted by the id:
goodList = [{'id': 'rtn', 'name': 'Something Else'}, {'id': 'ddsn', 'name': 'Something ElseElse'}, {'id': 'das', 'name': 'something'}]
goodList = sorted(jsonList, key=lambda x: orderList.index(x['id']))
or if you want just sort by id
sorted(jsonList, key=lambda x : x['id'])

nest dictionary list to another dictionary list based on key using comprehension list

I'm trying to use comprehension list to nest one dictionary list to another list of dictionary, I have two dictionary lists one is categories and another is oils. Add the result of list oils to each category if oils category_id is equal category id.
def nest(parent, child):
items = []
for element in child:
if element.get('category_id') == parent.get('id'):
items.append(element)
parent.update({'items': items})
return parent
def merge(parent, child):
results = []
for element in parent:
results.append(nest(element, child))
return results
categories = [
{'id': 1000, 'name': 'Single'},
{'id': 2000, 'name': 'Blend'}]
oils = [
{'id': 100, 'name': 'Orange', 'category_id': 1000},
{'id': 101, 'name': 'Lavender', 'category_id': 1000},
{'id': 102, 'name': 'Peppermint', 'category_id': 1000},
{'id': 104, 'name': 'Inspired', 'category_id': 2000},
{'id': 105, 'name': 'Focus', 'category_id': 2000},
{'id': 107, 'name': 'Tea Tree', 'category_id': 1000}]
results = merge(categories, oils)
print(results)
# output:
# [
# {'id': 1000, 'name': 'Single', 'items': [
# {'id': 100, 'name': 'Orange', 'category_id': 1000},
# {'id': 101, 'name': 'Lavender', 'category_id': 1000},
# {'id': 102, 'name': 'Peppermint', 'category_id': 1000},
# {'id': 107, 'name': 'Tea Tree', 'category_id': 1000}
# ]},
# {'id': 2000, 'name': 'Blend', 'items': [
# {'id': 104, 'name': 'Inspired', 'category_id': 2000},
# {'id': 105, 'name': 'Focus', 'category_id': 2000}
# ]}
# ]
I'm trying to convert the above to comprehension list without success
merged = [
element.update({'items': nest}) for nest in oils
for element in categories if element.get('id') == nest.get('category_id')
]
print(merged)
# output: [None, None, None, None, None, None]
merged = [dict(**c, items=[o for o in oils if o['category_id'] == c['id']]) for c in categories]
from pprint import pprint
pprint(merged)
Prints:
[{'id': 1000,
'items': [{'category_id': 1000, 'id': 100, 'name': 'Orange'},
{'category_id': 1000, 'id': 101, 'name': 'Lavender'},
{'category_id': 1000, 'id': 102, 'name': 'Peppermint'},
{'category_id': 1000, 'id': 107, 'name': 'Tea Tree'}],
'name': 'Single'},
{'id': 2000,
'items': [{'category_id': 2000, 'id': 104, 'name': 'Inspired'},
{'category_id': 2000, 'id': 105, 'name': 'Focus'}],
'name': 'Blend'}]
EDIT (to add dynamic variable):
variable = 'elements' # this is your dynamic variable
merged = [dict(**c, **{variable: [o for o in oils if o['category_id'] == c['id']]}) for c in categories]
I guess it would be nice as a simple function. Thanks.
def merge(parent, child, nested='nested', key='id', foreign='parent_id'):
return [
dict(**element, **{nested: [nest for nest in child if nest[foreign] == element[key]]})
for element in parent
]

python pandas Data frame column has string value looks like list of dictionaries?

I have a dataframe column that has string values in it (edited)
type of dataframe column is string BUT (edited)
its values look like list of dictionaries (edited)
how can i extract some key value from that string?
This STRING value looks like List of dictionaries.
How can I extract value of name key when key 'job' has a value as 'Director?
[{
'credit_id': '549e9edcc3a3682f2300824b',
'department': 'Camera',
'gender': 2,
'id': 473,
'job': 'Additional Photography',
'name': 'Brian Tufano',
'profile_path': None
}, {
'credit_id': '52fe4214c3a36847f8002595',
'department': 'Directing',
'gender': 2,
'id': 578,
'job': 'Director',
'name': 'Ridley Scott',
'profile_path': '/oTAL0z0vsjipCruxXUsDUIieuhk.jpg'
}, {
'credit_id': '52fe4214c3a36847f800259b',
'department': 'Production',
'gender': 2,
'id': 581,
'job': 'Producer',
'name': 'Michael Deeley',
'profile_path': None
}, {
'credit_id': '52fe4214c3a36847f800263f',
'department': 'Writing',
'gender': 2,
'id': 584,
'job': 'Novel',
'name': 'Philip K. Dick',
'profile_path': '/jDOKJN8SQ17QsJ7omv4yBNZi7XY.jpg'
}, {
'credit_id': '549e9f85c3a3685542004c7b',
'department': 'Crew',
'gender': 2,
'id': 584,
'job': 'Thanks',
'name': 'Philip K. Dick',
'profile_path': '/jDOKJN8SQ17QsJ7omv4yBNZi7XY.jpg'
}, {
'credit_id': '52fe4214c3a36847f800261b',
'department': 'Writing',
'gender': 2,
'id': 583,
'job': 'Screenplay',
'name': 'Hampton Fancher',
'profile_path': '/lrGecnLhzjzgwjKHvrmYtRAqOsP.jpg'
}
]
You can achieve this by converting your list into pandas dataframe:
pandas.DataFrame.
import pandas as pd
df =pd.DataFrame(yourList)
df['name'][df['job']=="Director"] # you will get a series of all the names matching your condition.
Or one more way is using .loc[]
df.loc[df["job"]=="Director",["name"]] # You will get a dataframe with name column having all the names matching the condition.
import pandas as pd
df =pd.DataFrame(LIST)
Name=df['name'][df['job']=="Director"]
print('Name =',Name)

Resources