How to group stream of Map to Map<String, Map<String, String>>? - groovy

I have stream of Map [name:name1, type:type1, desc:desc1, ordinal:1]. How to convert/group (with Groovy) to Map>: Map(type1: Map (neme:name1, desc:desc1, ordinal:1)).
Stream of Map
[name:productName, type:IN, ordinal:1, description:desc]
[name:productName1, type:IN, ordinal:2, description:desc]
[name:productName2, type:OUT, ordinal:3, description:desc]
and I have get: Map:
IN: Map[
[name:productName, type:IN, ordinal:1, description:desc.],
[name:productName1, type:IN, ordinal:2, description:desc.]]
OUT: Map[
[name:productName2, type:OUT, ordinal:3, description:desc.]]

You can use Stream.collect() method with Collectors.groupingBy { it.type } to collect all elements as map of type key and value of list of elements. Consider the following example:
import java.util.stream.Collectors
import java.util.stream.Stream
def input = Stream.of(
[name: 'productName', type: 'IN', ordinal: 1, description: 'desc'],
[name: 'productName1', type: 'IN', ordinal: 2, description: 'desc'],
[name: 'productName2', type: 'OUT', ordinal: 3, description: 'desc'],
)
def result = input.collect(Collectors.groupingBy { it.type })
result.each { println it }
Output:
IN=[{name=productName, type=IN, ordinal=1, description=desc}, {name=productName1, type=IN, ordinal=2, description=desc}]
OUT=[{name=productName2, type=OUT, ordinal=3, description=desc}]
Alternatively, if your input is not a Stream but a List, you could use good old Groovy Collection.groupBy() that does the same effect:
def input2 = [[name: 'productName', type: 'IN', ordinal: 1, description: 'desc'],
[name: 'productName1', type: 'IN', ordinal: 2, description: 'desc'],
[name: 'productName2', type: 'OUT', ordinal: 3, description: 'desc']]
def result2 = input2.groupBy { it.type }
result2.each { println it }

Related

How to create yaml file according to this demo result?

I want to create a .yaml file by dict method in Python while reprint the following result:
MODEL:
MASK_ON: True
IMAGE_ONLY: True
META_ARCHITECTURE: "VLGeneralizedRCNN"
BACKBONE:
NAME: "build_vit_fpn_backbone"
VIT:
NAME: "layoutlmv3_base"
OUT_FEATURES: [ "layer3", "layer5", "layer7", "layer11" ]
DROP_PATH: 0.1
IMG_SIZE: [ 224,224 ]
POS_TYPE: "abs"
DATASETS:
TRAIN: ("publaynet_train",)
TEST: ("publaynet_val",)
so I tried to:
import yaml
def mk_yaml(file_path):
backbone = {
"NAME": "layoutlmv3_base"
}
vit = {
"NAME": "layoutlmv3_base",
"OUT_FEATURES": [ "layer3", "layer5", "layer7", "layer11" ],
"DROP_PATH": 0.1,
"IMG_SIZE": [ 224,224 ],
"POS_TYPE": "abs"
}
model = {
"MASK_ON": True,
"META_ARCHITECTURE": "VLGeneralizedRCNN",
"BACKBONE": backbone,
"VIT": vit}
datasets = {
"TRAIN": '("publaynet_train",)',
"TEST": '("publaynet_val",)'
}
desired_caps = {"MODEL":model, "DATASETS":datasets}
with open(file_path, 'w', encoding='utf-8') as f:
yaml.safe_dump(desired_caps, f)
However, my result is not on expected order even if I put "model" in front of "dataset" and why the key of "OUT_FEATURES" and "IMG_SIZE" is not on the list? I attched it below.
DATASETS:
TEST: ("publaynet_val",)
TRAIN: ("publaynet_train",)
MODEL:
BACKBONE:
NAME: layoutlmv3_base
MASK_ON: true
META_ARCHITECTURE: VLGeneralizedRCNN
VIT:
DROP_PATH: 0.1
IMG_SIZE:
- 224
- 224
NAME: layoutlmv3_base
OUT_FEATURES:
- layer3
- layer5
- layer7
- layer11
POS_TYPE: abs

pythonic way deal with DataError: Invalid input of type: 'dict'. Convert to a bytes, string, int or float first.?

redis veersion 3.4.1
must be use hash, can't use str or other data type
data:
{'_anno': {
'ctp': 'list',
'dt': [],
'ml': 0,
'na': 'apple',
'pos': -1,
'rel': '',
'st_var': '',
'tp': 'object'},
'_att': {'_cuser': 'apple card',
'_last_editor': 'apple card',
'_protext': 'authorize',
'_status': 'normal',
'_theme_id': 'apple card',
'_view': '12'},
}
my code
pool = redis.ConnectionPool(host=host, port=port)
conn = redis.StrictRedis(connection_pool=pool)
conn.hmset("aaaaaa",data)
raise error
DataError: Invalid input of type: 'dict'. Convert to a bytes, string,
int or float first.
now code
pool = redis.ConnectionPool(host=host, port=port)
conn = redis.StrictRedis(connection_pool=pool)
new_data={}
for key,value in data.items():
new_data[key]=json.dumps(value)
conn.hmset("aaaaaa",new_data)
Is there a more pythonic way?
The solution for you problem is to use hexdigest() or digest() to convert your dictionary, you can use that:
hashlib.sha256(mdp.encode()).hexdigest()

How do I know which topic this word comes in?

This code works fine but I want to know the topic name instead of Topic: 0 and Topic:1, How do i know which topic this word comes in?
for index, topic in lda_model.show_topics(formatted=False, num_words= 30):
print('Topic: {} \nWords: {}'.format(idx, [w[0] for w in topic]))
This is ouput
Topic: 0
Words: ['associate', 'incident', 'time', 'task', 'pain', 'amcare', 'work', 'ppe', 'train', 'proper', 'report', 'standard', 'pmv', 'level', 'perform', 'wear', 'date', 'factor', 'overtime', 'location', 'area', 'yes', 'new', 'treatment', 'start', 'stretch', 'assign', 'condition', 'participate', 'environmental']
Topic: 1
Words: ['work', 'associate', 'cage', 'aid', 'shift', 'leave', 'area', 'eye', 'incident', 'aider', 'hit', 'pit', 'manager', 'return', 'start', 'continue', 'pick', 'call', 'come', 'right', 'take', 'report', 'lead', 'break', 'paramedic', 'receive', 'get', 'inform', 'room', 'head']
I want "Topic Name" instead of Topic : 0
Topic: 0
Words: ['associate', 'incident', 'time', 'task', 'pain', 'amcare', 'work', 'ppe', 'train', 'proper', 'report', 'standard', 'pmv', 'level', 'perform', 'wear', 'date', 'factor', 'overtime', 'location', 'area', 'yes', 'new', 'treatment', 'start', 'stretch', 'assign', 'condition', 'participate', 'environmental']
Topic: 1
Words: ['work', 'associate', 'cage', 'aid', 'shift', 'leave', 'area', 'eye', 'incident', 'aider', 'hit', 'pit', 'manager', 'return', 'start', 'continue', 'pick', 'call', 'come', 'right', 'take', 'report', 'lead', 'break', 'paramedic', 'receive', 'get', 'inform', 'room', 'head']
This might work (Untested)
for index, topic in lda_model.show_topics(formatted=False, num_words= 30):
print('Topic: {} \nWords: {}'.format(lda_model.print_topic(index), [w[0] for w in topic]))
Try changing the Formatted parameter to True like this:
for index, topic in lda_model.show_topics(formatted=True, num_words= 30):
print('Topic: {} \nWords: {}'.format(topic[0], [w[0] for w in topic[1]]))
You can also check out the documentation for more information:
https://radimrehurek.com/gensim/models/ldamodel.html

Building Flasgger/Swagger API in python3 exception

I am building a Flasgger/Swagger API in python. When I input the values into the parameters and execute the code I am seeing the following error. I have also share exception images.
"""Example endpoint returning a list of colors by value
This is using docstrings for specifications.
---
tags:
- Iris Prediction API Input values
parameters:
- name: s_length
in: path
type: string
required: true
default:
- name: s_width
in: path
type: string
required: true
default:
- name: p_length
in: path
type: string
required: true
default:
- name: p_width
in: path
type: string
required: true
default:
definitions:
value:
type: object
properties:
value_name:
type: string
items:
$ref: '#/definitions/Color'
Color:
type: string
responses:
200:
value: prediction details
schema:
$ref: '#/definitions/value'
examples:
rgb: ['red', 'green', 'blue']
"""
ValueError: Input contains NaN, infinity or a value too large for dtype('float32')
Flasgger Localhost image
Exception Image
I was able to figure it out by changing these parameters.
in: query
type: integer

Getting tags from python 3 dictionary object. AWS Boto3 Python 3

I have a dictionary object that is being returned to me from AWS. I need to pull the tag "based_on_ami" out of this dictionary. I have tried converting to a list, but I am new to programming and have not been able to figure out how to access Tags since they are a few levels down in the dictionary.
What is the best way for me to pull that tag out of the dictionary and put it into a variable i can use?
{
'Images':[
{
'Architecture':'x86_64',
'CreationDate':'2017-11-27T14:41:30.000Z',
'ImageId':'ami-8e73e0f4',
'ImageLocation':'23452345234545/java8server_ubuntu16-2b71edd1-f95e-4ee5-8fd6-d8a46975fdb5',
'ImageType':'machine',
'Public':False,
'OwnerId':'23452345234545',
'State':'available',
'BlockDeviceMappings':[
{
'DeviceName':'/dev/sda1',
'Ebs':{
'Encrypted':False,
'DeleteOnTermination':True,
'SnapshotId':'snap-0c10e8f5ced5b5240',
'VolumeSize':8,
'VolumeType':'gp2'
}
},
{
'DeviceName':'/dev/sdb',
'VirtualName':'ephemeral0'
},
{
'DeviceName':'/dev/sdc',
'VirtualName':'ephemeral1'
}
],
'EnaSupport':True,
'Hypervisor':'xen',
'Name':'java8server_ubuntu16-2b71edd1-f95e-4ee5-8fd6-d8a46975fdb5',
'RootDeviceName':'/dev/sda1',
'RootDeviceType':'ebs',
'SriovNetSupport':'simple',
'Tags':[
{
'Key':'service',
'Value':'baseami'
},
{
'Key':'cloudservice',
'Value':'ami'
},
{
'Key':'Name',
'Value':'java8server_ubuntu16-2b71edd1-f95e-4ee5-8fd6-d8a46975fdb5'
},
{
'Key':'os',
'Value':'ubuntu 16.04 lts'
},
{
'Key':'based_on_ami',
'Value':'ami-aa2ea8d0'
}
],
'VirtualizationType':'hvm'
}
],
'ResponseMetadata':{
'RequestId':'2c376c75-c31f-4aba-a058-173f3b125a00',
'HTTPStatusCode':200,
'HTTPHeaders':{
'content-type':'text/xml;charset=UTF-8',
'transfer-encoding':'chunked',
'vary':'Accept-Encoding',
'date':'Fri, 01 Dec 2017 18:17:53 GMT',
'server':'AmazonEC2'
},
'RetryAttempts':0
}
}
The best way to approach this type of problem is to find the value you're looking for, and then work outwards until you find a solution. You need to look at what is at each of those levels.
So, what are you looking for? You're looking for the Value for based_on_ami's Key. So your final step is going to be:
if obj['Key'] == 'based_on_ami':
# do something with obj['Value'].
But how do you get there? Well, the object is inside of a list, so you'll need to iterate the list:
for tag in <some list>:
if tag['Key'] == 'based_on_ami':
# do something with tag['Value'].
What is that list? It's the list of tags:
for tag in image['Tags']:
if tag['Key'] == 'based_on_ami':
# do something with tag['Value'].
And where are those tags? In an image object that you find in a list:
for image in image_list:
for tag in image['Tags']:
if tag['Key'] == 'based_on_ami':
# do something with tag['Value'].
The image list is the value found at the Images key in your initial dict.
image_list = my_data['Images']
for image in image_list:
for tag in image['Tags']:
if tag['Key'] == 'based_on_ami':
# do something with tag['Value'].
And now you're collecting all of those values, so you'll need a list and you'll need to append to it:
result = []
image_list = my_data['Images']
for image in image_list:
for tag in image['Tags']:
if tag['Key'] == 'based_on_ami':
result.append(tag['Value'])
So, I took your example above, and added another based_on_ami node with the value quack:
{'ResponseMetadata': {'RequestId': '2c376c75-c31f-4aba-a058-173f3b125a00', 'RetryAttempts': 0, 'HTTPHeaders': {'vary': 'Accept-Encoding', 'transfer-encoding': 'chunked', 'server': 'AmazonEC2', 'content-type': 'text/xml;charset=UTF-8', 'date': 'Fri, 01 Dec 2017 18:17:53 GMT'}, 'HTTPStatusCode': 200}, 'Images': [{'Public': False, 'CreationDate': '2017-11-27T14:41:30.000Z', 'BlockDeviceMappings': [{'Ebs': {'SnapshotId': 'snap-0c10e8f5ced5b5240', 'VolumeSize': 8, 'Encrypted': False, 'VolumeType': 'gp2', 'DeleteOnTermination': True}, 'DeviceName': '/dev/sda1'}, {'VirtualName': 'ephemeral0', 'DeviceName': '/dev/sdb'}, {'VirtualName': 'ephemeral1', 'DeviceName': '/dev/sdc'}], 'OwnerId': '23452345234545', 'ImageLocation': '23452345234545/java8server_ubuntu16-2b71edd1-f95e-4ee5-8fd6-d8a46975fdb5', 'RootDeviceName': '/dev/sda1', 'ImageType': 'machine', 'Hypervisor': 'xen', 'RootDeviceType': 'ebs', 'State': 'available', 'Architecture': 'x86_64', 'Name': 'java8server_ubuntu16-2b71edd1-f95e-4ee5-8fd6-d8a46975fdb5', 'Tags': [{'Value': 'baseami', 'Key': 'service'}, {'Value': 'ami', 'Key': 'cloudservice'}, {'Value': 'java8server_ubuntu16-2b71edd1-f95e-4ee5-8fd6-d8a46975fdb5', 'Key': 'Name'}, {'Value': 'ubuntu 16.04 lts', 'Key': 'os'}, {'Value': 'ami-aa2ea8d0', 'Key': 'based_on_ami'}], 'EnaSupport': True, 'SriovNetSupport': 'simple', 'ImageId': 'ami-8e73e0f4'}, {'Public': False, 'CreationDate': '2017-11-27T14:41:30.000Z', 'BlockDeviceMappings': [{'Ebs': {'SnapshotId': 'snap-0c10e8f5ced5b5240', 'VolumeSize': 8, 'Encrypted': False, 'VolumeType': 'gp2', 'DeleteOnTermination': True}, 'DeviceName': '/dev/sda1'}, {'VirtualName': 'ephemeral0', 'DeviceName': '/dev/sdb'}, {'VirtualName': 'ephemeral1', 'DeviceName': '/dev/sdc'}], 'VirtualizationType': 'hvm', 'OwnerId': '23452345234545', 'ImageLocation': '23452345234545/java8server_ubuntu16-2b71edd1-f95e-4ee5-8fd6-d8a46975fdb5', 'RootDeviceName': '/dev/sda1', 'ImageType': 'machine', 'Hypervisor': 'xen', 'RootDeviceType': 'ebs', 'State': 'available', 'Architecture': 'x86_64', 'Name': 'java8server_ubuntu16-2b71edd1-f95e-4ee5-8fd6-d8a46975fdb5', 'Tags': [{'Value': 'baseami', 'Key': 'service'}, {'Value': 'ami', 'Key': 'cloudservice'}, {'Value': 'java8server_ubuntu16-2b71edd1-f95e-4ee5-8fd6-d8a46975fdb5', 'Key': 'Name'}, {'Value': 'ubuntu 16.04 lts', 'Key': 'os'}, {'Value': 'quack', 'Key': 'based_on_ami'}], 'EnaSupport': True, 'SriovNetSupport': 'simple', 'ImageId': 'ami-8e73e0f4'}]}
My result:
['ami-aa2ea8d0', 'quack']
info = {...}
tags = []
for image in info['Images']:
for tag in image['Tags']:
if tag['Key'] == 'based_on_ami':
tags.append(tag['Value'])
print(tags)

Resources