Let's say I have the following structure:
Map<String,Map<String,Integer>> nestedMap = [
"x": [
"a": 2,
"b": 3,
"c": 4
],
"y": [
"a": 20,
"b": 30,
"c": 40
],
"z": [
"a": 200,
"b": 300,
"c": 400
]
]
I want to flatten this structure and get:
[
"x-a" : 2,
"x-b" : 3,
"x-c" : 4,
"y-a" : 20,
"y-b" : 30,
"y-c" : 40,
"z-a" : 200,
"z-b" : 300,
"z-c" : 400
]
How can I do this with collect/collectEnteries etc?
Alternatively, as you can see the values are quite similar. so the way I have constructed this nested map structure, was by running a for loop iterating through [x,y,z] and setting the entries of my map to be equal to a function that iterates through [a,b,c] and returns a map. Is there a way of skipping this nested structure and functionally append the elements in a big map?
You can create a recursive function that will loop over the map and flatten it.
Map<String,Map<String,Integer>> nestedMap = [
"x": ["a": 2, "b": 3, "z": 4],
"y": ["a": 20, "b": 30, "c": 40],
"z": ["a": 200, "b": 300, "c": 400]
]
String concatenate(String k, String v) {
"${k}-${v}"
}
def flattenMap(Map map) {
map.collectEntries { k, v ->
v instanceof Map ?
flattenMap(v).collectEntries { k1, v1 ->
key = concatenate(k,k1)
[ (key): v1 ]
} :
[ (k): v ]
}
}
print flattenMap(nestedMap)
Regarding the alternate question (and echoing a comment by #damahapatro), consider the following.
Given two lists:
def list1 = ['x', 'y', 'z']
def list2 = ['a', 'b', 'c']
and a function f that gives the appropriate values, then we can use inject (which will accumulate a map while iterating over the combos (e.g. [x,a]):
def m = [list1, list2].combinations().inject([:]) { map, combo ->
assert combo.size() == 2
def a = combo[0]
def b = combo[1]
map["${a}-${b}" as String] = f(a,b)
map
}
assert m == [
"x-a" : 2, "x-b" : 3, "x-c" : 4,
"y-a" : 20, "y-b" : 30, "y-c" : 40,
"z-a" : 200, "z-b" : 300, "z-c" : 400
]
Another way of building the map from scratch would be:
def result = ['a'..'c', 'x'..'z']
.combinations()
.collect { a, b -> b + '-' + a }
.indexed()
.collectEntries { idx, name ->
[name, ((idx % 3) + 2) * (int)Math.pow(10, idx.intdiv(3))]
}
I modified the define syntax of #Paul Walkington and tested it:
String concatenate(String k, String v) {
"${k}-${v}"
}
def flattenMap(Map map) {
map.collectEntries { k, v ->
v instanceof Map ?
flattenMap(v).collectEntries { k1, v1 ->
def key = concatenate(k,k1)
[ (key): v1 ]
} :
[ (k): v ]
}
}
You can do your task with a single istruction:
Map<String,Map<String,Integer>> nestedMap = [
"x": ["a": 2, "b": 3, "c": 4],
"y": ["a": 20, "b": 30, "c": 40],
"z": ["a": 200, "b": 300, "c": 400] ]
result = nestedMap.inject([:]){ a, k, v ->
a << v.collectEntries{ [k + '-' + it.key, it.value] }; a }
println result
(plus setting the source data, plus print the result).
Related
This question already has answers here:
Accessing nested keys in Python
(5 answers)
Closed 2 years ago.
Is there a way to return a list of all dictionary keys in the provided input (at all levels)?
The keys should be ordered by level, for example:
{
"A": 1,
"B": [{
"B1": 1,
"B2": 1
}, {
"B3": 1,
"B4": 1
}],
"C": {
"C1": 1,
"C2": 1
}
}
I have tried using dict.keys() and dict.items() but did not get the desired output.
The output should be like this ["A", "B", "C", "B1", "B2", "B3", "B4", "C1", "C2"].
Any help is appreciated.
I would do a recursive function:
d = {
"A": 1,
"B": [{
"B1": 1,
"B2": 1
}, {
"B3": 1,
"B4": 1
}],
"C": {
"C1": 1,
"C2": 1
}
}
def flatten(d, lst=None):
if not lst:
lst = []
if isinstance(d, list):
for item in d:
lst = flatten(item, lst)
elif isinstance(d, dict):
for k in d.keys():
lst.append(k)
for v in d.values():
lst = flatten(v, lst)
return lst
print(flatten(d)) # Output: ['A', 'B', 'C', 'B1', 'B2', 'B3', 'B4', 'C1', 'C2']
EDIT: Above code assumes you're using Python 3.7+ where dict are ordered by default. If you're running an older version, you can use collection.OrderedDict instead while initializing d to have the same behavior.
I think i found it
#In order to convert nested list to single list
import more_itertools
a={"A": 1,
"B": [{
"B1": 1,
"B2": 1
}, {
"B3": 1,
"B4": 1
}],
"C": {
"C1": 1,
"C2": 1
}
}
dic_key=[]
for i in a:
dic_key.append(i)
for i in a:
if type(a[i]) is list :
dic_key+=list(map(lambda x:list(x.keys()),a[i]))
elif type(a[i]) is dict:
dic_key+=list(map(lambda x: x,a[i]))
print(list(more_itertools.collapse(dic_key)))
I need help with a function to flatten a nested dictionary in the following format:
dict_test = {
"subdoc": {
"a": 1,
"b": 2,
"c": 3,
"array": [
{"name": "elmo"},
{"name": "oscar"}
]
}
}
I tried the function below, but is doesn´t flatten the nested keys inside the array element.
def flatten_dict(dd, separator='.', prefix=''):
return { prefix + separator + k if prefix else k : v
for kk, vv in dd.items()
for k, v in flatten_dict(vv, separator, kk).items()
} if isinstance(dd, dict) else { prefix : dd }
My current output is:
{'subdoc.a': 1,
'subdoc.b': 2,
'subdoc.c': 3,
'subdoc.array': [{'name': 'elmo'}, {'name': 'oscar'}]}
But actually I´m looking something like:
{
"subdoc.a": 1,
"subdoc.b": 2,
"subdoc.c": 3,
"subdoc.array.0.name": "elmo",
"subdoc.array.1.name": "oscar"
}
dict_test = {
"subdoc": {
"a": 1,
"b": 2,
"c": 3,
"array": [
{"name": "elmo"},
{"name": "oscar"}
]
}
}
def flatten(d):
agg = {}
def _flatten(d, prev_key=''):
if isinstance(d, list):
for i, item in enumerate(d):
new_k = '%s.%s' % (prev_key, i) if prev_key else i
_flatten(item, prev_key=new_k)
elif isinstance(d, dict):
for k, v in d.items():
new_k = '%s.%s' % (prev_key, k) if prev_key else k
_flatten(v, prev_key=new_k)
else:
agg[prev_key] = d
_flatten(d)
return agg
print(flatten(dict_test))
# gives {'subdoc.a': 1, 'subdoc.b': 2, 'subdoc.c': 3, 'subdoc.array.0.name': 'elmo', 'subdoc.array.1.name': 'oscar'}
Note: only tested for your case
i have a nested python dictionary where i am trying to remove all keys except those mentioned. I found a solution here where we can remove keys specified, what i am trying to do is reverse of it, i want to remove all keys except those mentioned. So i made a slight change in the function, but i am getting an empty dict.
from collections import MutableMapping
def remove_fields(d, list_of_keys_to_remove):
if not isinstance(d, (dict, list)):
return d
if isinstance(d, list):
return [v for v in (remove_fields(v, list_of_keys_to_remove) for v in d) if v]
return {k: v for k, v in ((k, remove_fields(v, list_of_keys_to_remove)) for k, v in d.items()) if
k in list_of_keys_to_remove}
Adding sample input
EDIT: let suppose i want to keep RENEWAL_TYPE
{
'G_1': [
{
'ACCOUNTING_RULE_ID': '1',
'PAYMENT_TERM_ID': '4',
'RENEWAL_TYPE': 'RENEW'
},
{
'ACCOUNTING_RULE_ID': '2',
'PAYMENT_TERM_ID': '4',
'RENEWAL_TYPE': 'RENEW'
},
{
'ACCOUNTING_RULE_ID': '3',
'PAYMENT_TERM_ID': '4',
'RENEWAL_TYPE': 'DO_NOT_RENEW'
},
{
'ACCOUNTING_RULE_ID': '4',
'PAYMENT_TERM_ID': '4',
'RENEWAL_TYPE': 'RENEW'
}
]
}
You're looking for the intersection between the set of keys in the dict and the set of values in the list. i.e. all keys that exist in both sets.
Try this:
from collections import MutableMapping
d = { 'A': 1, 'B': 2, 'C': 3, 'D': 4 }
list_of_keys_to_keep = ['B', 'C', 'F']
def remove_fields(d, list_of_keys_to_keep):
return {key: value for key, value in d.items() if key in list_of_keys_to_keep}
print(remove_fields(d, list_of_keys_to_keep)) # -> {'B': 2, 'C': 3}
Edit: Updated name of list_of_keys_to_remove to list_of_keys_to_keep since this seems to be what it actually represents.
Edit2: Updated after Askers update. The following works for the sample provided:
from collections import MutableMapping
d = {
'G_1': [
{
'ACCOUNTING_RULE_ID': '1',
'PAYMENT_TERM_ID': '4',
'RENEWAL_TYPE': 'RENEW'
},
{
'ACCOUNTING_RULE_ID': '2',
'PAYMENT_TERM_ID': '4',
'RENEWAL_TYPE': 'RENEW'
},
{
'ACCOUNTING_RULE_ID': '3',
'PAYMENT_TERM_ID': '4',
'RENEWAL_TYPE': 'DO_NOT_RENEW'
},
{
'ACCOUNTING_RULE_ID': '4',
'PAYMENT_TERM_ID': '4',
'RENEWAL_TYPE': 'RENEW'
}
]
}
list_of_keys_to_keep = ['RENEWAL_TYPE']
def filter_dict(di, keys):
if not isinstance(di, (dict, list)):
return di
if isinstance(di, list):
return [filter_dict(value, keys) for value in di]
return {key: filter_dict(value, keys) for key, value in di.items() if key in keys or isinstance(value, (dict, list))}
print(filter_dict(d, list_of_keys_to_keep))
Output:
{'G_1': [{'RENEWAL_TYPE': 'RENEW'}, {'RENEWAL_TYPE': 'RENEW'}, {'RENEWAL_TYPE': 'DO_NOT_RENEW'}, {'RENEWAL_TYPE': 'RENEW'}]}
try this one:
dict = {"a": 1, "b": 2, "c": 3, "d": 4, "e": 5}
keys_to_preserve = ["e", "b", "c"]
dict_final={k: v for k,v in dict.items() if k in keys_to_preserve}
def remove_fields(d, keys_to_preserve):
if isinstance(d, list):
return [remove_fields(v, keys_to_preserve) for v in d]
elif isinstance(d, dict):
return dict([(k, remove_fields(v, keys_to_preserve)) for k, v in d.items() if k in keys_to_preserve])
else:
return d
x = {"a": 1, "b": 2, "c": 3}
y = {"b": 4, "c": 5, "d": 6}
for key in x:
if key in y:
a = (x[key])
b = (y[key])
This returns a as 2 3 and b as 4 5. What i'm trying to do is multiply the matching key values together and then add those values together. I am not quite sure how to do this. If you guys could help me out that would be great. Thank you in advanced.
A simple way to do this would be to just keep a running total, e.g.:
total = 0
for key in x:
if key in y:
a = x[key]
b = y[key]
total += a*b
print(total) # 23
But python has powerful comprehensions/generators that can simplify this to:
>>> sum(x[key]*y[key] for key in x if key in y)
23
You can use sum with a generator:
x = {"a": 1, "b": 2, "c": 3}
y = {"b": 4, "c": 5, "d": 6}
sum(x[k] * y[k] for k in set(x) & set(y))
# 23
I have an array containing an unknown number of items that I would like to split into separate arrays so that each separate array contains no more than 4 items. What is the best way to do this in Groovy? Thanks!
We had this here: How to split a list into equal sized lists in Groovy?
I came up with this:
List.metaClass.partition = { size ->
def rslt = delegate.inject( [ [] ] ) { ret, elem ->
( ret.last() << elem ).size() >= size ? ret << [] : ret
}
!rslt.last() ? rslt[ 0..-2 ] : rslt
}
def list = [1, 2, 3, 4, 5, 6].partition( 4 )
Which should give you:
[ [ 1, 2, 3, 4 ], [ 5, 6 ] ]
Update!
With Groovy 1.8.6+ you can use list.collate( 4 ) to get the same result
Answer by tim_yates is cool, but it throws java.lang.ArrayIndexOutOfBoundsException on empty lists (for example: [].partition(4)). This can be fixed in this way:
List.metaClass.partition = {size ->
if (!delegate)
return []
def rslt = delegate.inject([[]]) {ret, elem ->
(ret.last() << elem).size() >= size ? (ret << []) : ret
}
!rslt.last() ? rslt[0..-2] : rslt
}
assert [].partition(4) == []
assert [1, 2, 3, 4, 5, 6].partition(4) == [[1, 2, 3, 4], [5, 6]]
Since Groovy 1.8.6, you can use collate:
def letters = 'a'..'g'
assert letters.collate(3) == [['a', 'b', 'c'], ['d', 'e', 'f'], ['g']]
def letters = 'a'..'g'
assert letters.collate(3) == [['a', 'b', 'c'], ['d', 'e', 'f'], ['g']]
Credit to Mrhaki's Groovy goodness series:
http://mrhaki.blogspot.com.au/2012/04/groovy-goodness-collate-list-into-sub.html