I have a list of identical dicts (same keys, same number of keys..) ..like this:
mydict_list = [
{'win32': [2, 45], 'https': [2, 13], 'nofollow': [3, 45], 'href': [4, 847]},
{'win32': [0, 5], 'https': [0, 13], 'nofollow': [1, 5], 'href': [2, 87]}
]
I want to create one dict that has the same keys and values added up nicely....like this:
{'win32': [2, 50], 'https': [2, 26], 'nofollow': [4, 50], 'href': [6, 934]}
Is there an elegant way to do it?
This is what I have working. It works and is readable also:
keys = list(mydict_list[0].keys())
final_dict = {item: [] for item in keys}
for k in list(final_dict.keys()):
v0 = v1 = 0
for d in mydict_list:
for kk, v in d.items():
if k == kk:
v0 += v[0]
v1 += v[1]
final_dict.update({k: [v0, v1]})
Short approach:
lst = [
{'win32': [2, 45], 'https': [2, 13], 'nofollow': [3, 45], 'href': [4, 847]},
{'win32': [0, 5], 'https': [0, 13], 'nofollow': [1, 5], 'href': [2, 87]}
]
res = lst[0] # start with the 1st dict
for d in lst[1:]:
for k, v in d.items():
res[k] = list(map(sum, zip(v, res[k])))
print(res)
The output:
{'win32': [2, 50], 'https': [2, 26], 'nofollow': [4, 50], 'href': [6, 934]}
Consider using a dictionary comprehension on this to make it more "pythonic" (though readability goes down significantly)
d1=[
{'win32': [2, 45], 'https': [2, 13], 'nofollow': [3, 45], 'href': [4, 847]},
{'win32': [0, 5], 'https': [0, 13], 'nofollow': [1, 5], 'href': [2, 87]}
]
Can do it in a a two-liner (and should be mostly generic for almost all cases). Assuming that all the keys are the same across each dictionary in the array.
d2={key:[d[key] for d in d1] for key in d1[0].keys()}
d3={key:[sum(d3) for d3 in list(map(list,zip(*d2[key])))] for key in d1[0].keys()}
print(d3)
{'win32': [2, 50], 'https': [2, 26], 'nofollow': [4, 50], 'href': [6, 934]}
The Explanation
Dictionary comprehensions allow you to collapse your array so that we can work with each key individually.
In the first step, we create d2 which is d1 rearranged by key.
Once you have that in place, d3 the list-map re-arranges the array so that you can sum each list element wise using the built-in sum as part of a list comprehension.
If data was stored as a numpy array, this process would probably go significantly faster (as is, it's a bit memory inefficient).
Related
I am on Python 3.9
What I have is two dictionaries.
dict1= {'ZS': [1, 2], 'ZP': [3], 'XS': [4, 5], 'XP': [6, 7]}
dict2= {'a': {}, 'b' : {}}
how do I take every two keys of dict1 and add them as dict-value in dict2.
Like below
required dict
result_dict= {'a': {'ZS': [1, 2], 'ZP': [3]}, 'b' : {'XS': [4, 5], 'XP': [6, 7]}}
The problem of your question is that you didn't explain clearly what is the criteria of the splitting.
Another problem is that when you iniciate a dictionary inside a dictionary, like you did in dict2. if you send 'a' or 'b' values to a variable for example c:
c = dict2['a']
It won't create a new { }, it will pass the reference of the '{ }' inside dict2['a'] or dict2['b'] and when you change c it will change both c and dict2['a'] value
That is quite hacky, got stuck while trying to make a solution hehehe.
If the criteria you are using is the first letter of the key, you can do:
dict1 = {'ZS': [1, 2], 'ZP': [3], 'XS': [4, 5], 'XP': [6, 7]}
dict2 = {'a': {}, 'b': {}}
result_dict = {}
for x in dict2.keys():
result_dict[x] = {}
for key in dict1.keys():
if key[0:1] == "Z":
result_dict['a'][key] = dict1[key]
elif key[0:1] == "X":
result_dict['b'][key] = dict1[key]
print(dict1)
print(dict2)
print(result_dict)
{'ZS': [1, 2], 'ZP': [3], 'XS': [4, 5], 'XP': [6, 7]}
{'a': {}, 'b': {}}
{'a': {'ZS': [1, 2], 'ZP': [3]}, 'b': {'XS': [4, 5], 'XP': [6, 7]}}
You could do something like:
keys1 = list(dict1)
keys2 = list(dict2)
result_dict = {
keys2[i]: {
keys1[2*i]:dict1[keys1[2*i]],
keys1[2*i+1]:dict1[keys1[2*i+1]]
}
for i in range(len(keys2))
}
But in my opinion it is a really bad habit to trust dictionary key's order (unless you use OrderedDict)
Related:
Pytorch, retrieving values from a tensor using several indices. Most computationally efficient solution
This is another question about retrieving values from a 3D tensor, using a list of indices.
In this case, I have a 3d tensor, for example
b = [[[4, 20], [1, -1]], [[1, 2], [8, -1]], [[92, 4], [23, -1]]]
tensor_b = torch.tensor(b)
tensor_b
tensor([[[ 4, 20],
[ 1, -1]],
[[ 1, 2],
[ 8, -1]],
[[92, 4],
[23, -1]]])
In this case, I have a list of 3D indices. So
indices = [
[[1, 0, 1], [2, 0, 1]],
[[1, 1, 1], [0, 0, 0]],
[[2, 1, 0], [0, 1, 0]]
]
Each triple is an index for tensor-b. The desired result is
[[2, 4], [-1, 4], [23, 1]]
Potential Approach
Like in the last question, the first solution that comes to mind is a nested for loop, but there is probably a more computationally efficient solution using pytorch function.
And like in the last question, perhaps reshape would be needed to get the desired shape for the last solution.
So a desired solution could be [2, 4, -1, 4, 23, 1], which can come from a flattened list of indices
[ [1, 0, 1], [2, 0, 1], [1, 1, 1], [0, 0, 0], [2, 1, 0], [0, 1, 0] ]
But I am not aware of any pytorch functions so far which allow for a list of 3D indices. I have been looking at gather and index_select.
You can use advanced indexing specifically integer array indexing
tensor_b = torch.tensor([[[4, 20], [1, -1]], [[1, 2], [8, -1]], [[92, 4], [23, -1]]])
indices = torch.tensor([
[[1, 0, 1], [2, 0, 1]],
[[1, 1, 1], [0, 0, 0]],
[[2, 1, 0], [0, 1, 0]]
])
result = tensor_b[indices[:, :, 0], indices[:, :, 1], indices[:, :, 2]]
results in
tensor([[ 2, 4],
[-1, 4],
[23, 1]])
Given a nested list say:
a = [[1, 5, 100],
[2],
[2, 100]]
The desired result to be obtained is as follows:
[[1, 2, 5], [1, 2, 100], [5, 2, 100], [100, 2, 5]]
Here is my code, but it does not give the output as desired. I am unable to progress further:
arr = [[i] for i in a[0]]
def poss(j, arr, tmp):
for i in range(len(tmp)):
arr[i] = tmp[i] + [j]
print(arr)
for i in a[1:]:
tmp = [k for k in arr] # deepcopy of arr
for j in i:
poss(j, arr, tmp)
Output for above code:
[[1, 2], [5, 2], [100, 2]]
[[1, 2, 5], [5, 2, 5], [100, 2, 5]]
[[1, 2, 100], [5, 2, 100], [100, 2, 100]]
I also feel this code is inefficient on large data, is that so? I'm looking for a better code to get the result.
This problem can be solved by using itertools module of python.
The itertools.combinations() function returns all the possible subsets of the given set without repetition of elements.
import math
import itertools
a = [[1, 5, 100],
[2],
[2, 100]]
dimx = max([len(el) for el in a])
uniqueEls={}
for el in a:
for subel in el:
uniqueEls[subel] = uniqueEls.get(subel,0)
desiredArr= [list(x) for x in list(itertools.combinations(uniqueEls.keys(), dimx))]
print(desiredArr)
[[1, 5, 100], [1, 5, 2], [1, 100, 2], [5, 100, 2]]
Can anyone help me. This is what i want to do.
x = [[1,2,3,4,5],[6,7,8,9,10]]
y= [0,1]
desired output = [
[[1,2,3,4,5],[0,1]],
[[6,7,8,9,10],[0,1]]
]
I try putting it in a for loop
>>> x = [[1,2,3,4,5],[6,7,8,9,10]]
>>> for value in x:
... a = []
... a += ([x,y])
... print(a)
...
[[[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]], [0, 1]]
[[[1, 2, 3, 4, 5], [6, 7, 8, 9, 10]], [0, 1]]
I also tried doing this
>>> for value in x:
... a = []
... a += ([x,y])
... print(a)
...
[[1, 2, 3, 4, 5], [0, 1]]
[[1, 2, 3, 4, 5], [0, 1]]
[[1, 2, 3, 4, 5], [0, 1]]
[[1, 2, 3, 4, 5], [0, 1]]
[[1, 2, 3, 4, 5], [0, 1]]
Thank you for helping. I need it for putting label on my data for neural networks.
You can use a list comprehension, and iterate over each sublist in x. Since you're inserting y into different sublists, you might want to insert a copy of the list, not the original.
[[i, y[:]] for i in x]
Or,
[[i, y.copy()] for i in x]
[[[1, 2, 3, 4, 5], [0, 1]], [[6, 7, 8, 9, 10], [0, 1]]]
The copy is done as a safety precaution. To understand why, consider an example,
z = [[i, y] for i in x] # inserting y (reference copy)
y[0] = 12345
print(z)
[[[1, 2, 3, 4, 5], [12345, 1]], [[6, 7, 8, 9, 10], [12345, 1]]] # oops
Modifying the original y or the y in any other sublist will reflect changes across all sublists. You can prevent that by inserting a copy instead, which is what I've done at the top.
Try this:
for i in range(len(x)):
z[i] = [x[i],y];
Suppose I have the following numpy.array:
In[]: x
Out[]:
array([[1, 2, 3, 4, 5],
[5, 2, 4, 1, 5],
[6, 7, 2, 5, 1]], dtype=int16)
In[]: y
Out[]:
array([[-3, -4],
[-4, -1]], dtype=int16)
I want to replace a sub array of x by y and tried the following:
In[]: x[[0,2]][:,[1,3]]= y
Ideally, I wanted this to happen:
In[]: x
Out[]:
array([[1, -3, 3, -4, 5],
[5, 2, 4, 1, 5],
[6, -4, 2, -1, 1]], dtype=int16)
The assignment line doesn't give me any error, but when I check the output of x
In[]: x
I find that x hasn't changed, i.e. the assignment didn't happen.
How can I make that assignment? Why did the assignment didn't happen?
The the "fancy indexing" x[[0,2]][:,[1,3]] returns a copy of the data. Indexing with slices returns a view. The assignment does happen, but to a copy (actually a copy of a copy of...) of x.
Here we see that the indexing returns a copy:
>>> x[[0,2]]
array([[1, 2, 3, 4, 5],
[6, 7, 2, 5, 1]], dtype=int16)
>>> x[[0,2]].base is x
False
>>> x[[0,2]][:, [1, 3]].base is x
False
>>>
Now you can use fancy indexing to set array values, but not when you nest the indexing.
You can use np.ix_ to generate the indices and perform the assignment:
>>> x[np.ix_([0, 2], [1, 3])]
array([[2, 4],
[7, 5]], dtype=int16)
>>> np.ix_([0, 2], [1, 3])
(array([[0],
[2]]), array([[1, 3]]))
>>> x[np.ix_([0, 2], [1, 3])] = y
>>> x
array([[ 1, -3, 3, -4, 5],
[ 5, 2, 4, 1, 5],
[ 6, -4, 2, -1, 1]], dtype=int16)
>>>
You can also make it work with broadcasted fancy indexing (if that's even the term) but it's not pretty
>>> x[[0, 2], np.array([1, 3])[..., None]] = y
>>> x
array([[ 1, -3, 3, -4, 5],
[ 5, 2, 4, 1, 5],
[ 6, -4, 2, -1, 1]], dtype=int16)
By the way, there is some interesting discussion at the moment on the NumPy Discussion mailing list on better support for "orthogonal" indexing so this may become easier in the future.