How to access key and value from this dictionary - python-3.x

Here is a snippet of function code in Python:
def restart_lab_server(aws_client):
response = aws_client.describe_instances()
###print(response)
for reservation in response["Reservations"]:
for instance in reservation["Instances"]:
# This sample print will output entire Dictionary object
# This will print will output the value of the Dictionary key 'InstanceId'
try:
print(instance["Tags"])
except:
print("This instance does not have any tags.")
print("Next Entry.")
return 0
The output that I am getting this the following:
[{'Key': 'Name', 'Value': 'EC2-Nessus-AmazonLinux2'}]
[{'Key': 'Name', 'Value': 'LCE Server'}]
[{'Key': 'Name', 'Value': 'Windows-2019-xxxxx'}]
[{'Key': 'Name', 'Value': 'SecurityCenter-Lab-5.19.1'}]
[{'Key': 'Name', 'Value': 'Nessus-Target-Lab'}]
[{'Key': 'Name', 'Value': 'Nessus-Agent-Lab'}]
[{'Key': 'Name', 'Value': 'NNM-Lab'}]
[{'Key': 'Name', 'Value': 'WindowsInstance'}]
This instance does not have any tags.
Next Entry.
[{'Key': 'CfStackId', 'Value': 'arn:aws:cloudformation:us-east-1:581987831513:stack/Nessus-Thomas-101-2/41d38410-5729-11ed-a52f-127af16651f7'}, {'Key': 'aws:cloudformation:stack-id', 'Value': 'arn:aws:cloudformation:us-east-1:581987831513:stack/Nessus-Thomas-101-2/41d38410-5729-11ed-a52f-127af16651f7'}, **{'Key': 'NessusScanner', 'Value': 'TRUE'},** {'Key': 'aws:cloudformation:logical-id', 'Value': 'NessusEC2Instance'}, {'Key': 'aws:cloudformation:stack-name', 'Value': 'Nessus-Thomas-101-2'}]
My main interest is to get "{'Key': 'NessusScanner', 'Value': 'TRUE'},"
I can't seem to extract this or not sure how to. I am not sure if I need to use a linescanner or something else.
I'm trying to restart all Nessus scanners in my and figured using key and value Nessus scanner True.
Please note, I'm using
describe_instances()
I have tried creating another for loop but keep getting an error message.

Found the answer or workaround:
def restart_lab_server(aws_client):
response = aws_client.describe_instances()
###print(response)
for reservation in response["Reservations"]:
for instance in reservation["Instances"]:
try:
x = instance["Tags"]
for y in x:
print(y["Value"])
except:
print("This instance does not have any tags. Next Entry.")
return 0

Related

How to Create an Apache Spark DataFrame from JSON or Dictonary or Key Value pairs format in Databricks

The Key:Value pairs from the following dictionary is as follows
result.items()
dict_items([('subjectArea', 'Work'), ('txn-timestamp', '2022-01-05 11:31:10'), ('foundation', {'schema': 'AZ_FH_ELLIPSE', 'table': 'AZ_FND_MSF620', 'keys': [{'key': 'DSTRCT_CODE', 'value': 'RTK1'}, {'key': 'WORKORDER', 'value': '11358186'}], 'dataMart': {'dependencies': [{'schema': 'AZ_DM_WORK', 'table': 'DIM_WORK_ORDER'}, {'schema': 'AZ_DM_WORK', 'table': 'FACT_WORK_ITEM'}]}})])
Can someone let me know if its possible to convert the above into a Spark DataFrame?
My apologies, I'mm not sure how to do a line break to make the code look neater
Here is the below pyspark code:
from pyspark.sql.types import *
items1 = [{'subjectArea': 'Work', 'txn-timestamp': '2022-01-05 11:31:10', 'foundation': {'schema': 'AZ_FH_ELLIPSE', 'table': 'AZ_FND_MSF620', 'keys': [{'key': 'DSTRCT_CODE', 'value': 'RTK1'}, {'key': 'WORKORDER', 'value': '11358186'}], 'dataMart': {'dependencies': [{'schema': 'AZ_DM_WORK', 'table': 'DIM_WORK_ORDER'}, {'schema': 'AZ_DM_WORK', 'table': 'FACT_WORK_ITEM'}]}}}]
schema1 = StructType([StructField('subjectArea',StringType()),StructField('txn-timestamp',StringType()),StructField('foundation',StructType([StructField('schema',StringType()),StructField('table',StringType()),StructField('keys',StructType([StructField('key',StringType()),StructField('value',StringType())])),StructField('dataMart',StructType([StructField('dependencies',StructType([StructField('schema',StringType()),StructField('table',StringType())]))]))]))])
ddf = spark.createDataFrame(items1, schema1)
ddf.printSchema()
ddf.show()
ddf.select(ddf['foundation'].datamart).show(truncate=False)

Unable to populate column in dataframe using value from dict which is present in dict inside list in one of the column

I have one column in data frame with values like this List with dictionary inside
[{'symbol': '$', 'value': 5.2, 'currency': 'USD', 'raw': '$5.20', 'name': '$5.20$5.20 ($1.30/Fl Oz)', 'asin': 'B07N31VZP8']
I want only 'value' from this dictionary.
for i in df["prices"]:
try:
#print(type(i[0]["value"]))
df["prices"] = df["prices"].apply(lambda i:i[0]["value"])
except Exception as e:
print("",e)
but I'm getting the following error even if I was able to get that value, I cant populate it in dataframe column
'float' object is not subscribable
How to overcome this issue ?
I think the problem is that you apply the transformation multiple times. The apply() already applies the transformation for every row in your dataframe. If you remove the loop it should work:
import pandas as pd
# Create your data
df = pd.DataFrame([
['test',[{'symbol': '$', 'value': 5.2, 'currency': 'USD', 'raw': '$5.20', 'name': '$5.20$5.20 ($1.30/Fl Oz)', 'asin': 'B07N31VZP8'}]]
, ['test',[{'symbol': '$', 'value': 6.8, 'currency': 'USD', 'raw': '$5.20', 'name': '$5.20$5.20 ($1.30/Fl Oz)', 'asin': 'B07N31VZP8'}]]
], columns=['test','prices'])
print(df)
df["prices"] = df["prices"].apply(lambda i:i[0]["value"])
print(df)
Input dataframe:
test prices
0 test [{'symbol': '$', 'value': 5.2, 'currency': 'US...
1 test [{'symbol': '$', 'value': 6.8, 'currency': 'US...
Output dataframe:
test prices
0 test 5.2
1 test 6.8

How to append the repeated values to same key in dictionary list

I have below dictionary list which contains repeated values. i need to append the repeated values to same key and remain should store as it in the dictionary list.
veh_entry=[{'name': 'scott', 'id': '17'},{'name': 'thomas', 'id': '18'}, {'name': 'tony', 'id': '17'}]
i tried with below approach, but not seems to be working as expected
add=[]
test={}
for item in veh_entry:
if item['id'] not in add:
test['name']=item['name']
add.append(item['id'])
else:
test['name']=(test['name']+ ','+item['name'])
#expected:
the expected dictionary must be as follows:
[{'name': 'scott, tony', 'id':'17'},{'name': 'thomas', 'id': '18'}]
So the basic logic was to compare the ids of different item in the list and if the id's match then join the name and remove the repeated item from the list
final_result = []
for i in range(len(veh_entry)-1):
for j in range(i+1,len(veh_entry)):
a = dict()
if veh_entry[i]['id'] == veh_entry[j]['id']:
a['name'] = veh_entry[i]['name'] +','+veh_entry[j]['name']
a['id'] = veh_entry[i]['id']
veh_entry.pop(j)
final_result.append(a)
else:
final_result.append(veh_entry[j])
print(final_result)
Output:- [{'name': 'thomas', 'id': '18'}, {'name': 'scott,tony', 'id': '17'}]
You can go with this:
def remove_dup(arr):
names=[] # using list here
ids=[] # instead of dict
for x in arr:
id,name=x['id'],x['name']
if id not in ids: # search for existing id
ids.append(id)
names.append(name)
else:
names[ids.index(id)]+=", "+name
return [{'name':name,'id':id} for name,id in zip(names,ids)]
veh_entry=[{'name': 'scott', 'id': '17'},{'name': 'thomas', 'id': '18'}, {'name': 'tony', 'id': '17'}]
print(remove_dup(veh_entry))

list of dict with same key name

I am trying to achieve the following output:
[{'Key': 'Language', 'Value': 'Python'}, {'Key': 'Version', 'Value': '3.7'}]
I have implemented a method to achieve the above output:
#cli.command('test', context_settings=dict(
ignore_unknown_options=True,
allow_extra_args=True
))
#click.pass_context
def test(ctx):
data = dict()
tags=dict()
tag_list = list()
for item in ctx.args:
data.update([item.split('=')])
for items in data.items():
tags['Key'] = items[0]
tags['Value'] = items[1]
tag_list.append(tags)
print(tag_list)
Method Call:
python test.py test Language=Python Version=3.7
But I am getting below output:
[{'Key': 'Version', 'Value': '3.7'}, {'Key': 'Version', 'Value': '3.7'}]
The is old dict is been replaced and then appended.
Can you please help me with this?
Thanks & Regards,
You are mutating the same dict in the last loop, so your list will have references to the same dict.
Move this:
tags=dict()
...inside the final for loop:
for items in data.items():
tags=dict()

how to convert dict value use lambda map in python

the format data like this, To explain, I have this dict:
{
'41': [
'1029136700',
'1028348931'
],
'42': ['12234454']
...
}
then i want to convert the format like this used lambda and map:
[
{
'key':'41','value':'1029136700'
},
{
'key':'41','value': '1028348931'
},
{
'key':'42', 'value': '12234454'
}
...
]
Can you give me a clue on how to achieve this in python?
Here is a clue on how to get what you want - you have to iterate over the initial dictionary:
for key, list_of_vals in initial_dictionary.items():
for val in list_of_vals:
# now you have pairs
# for example, key = '41', val = '1029136700'
# use them as you need
Just couple more steps and you get exactly what you want.
However, if you want to do this with map() and lambda it would not be that easy. You have to replace every for loop (there 2 of them) with map() containing lambda. map() returns generator so you have to iterate over it to get actual result, for example, list(map(...)). Here is complete code how you can get what you want:
result = list()
set(map(lambda item: result.extend(list(
map(lambda val: {'key': item[0], 'value': val},
item[1]))),
initial_dictionary.items()))
print(result)
Output:
[{'key': '41', 'value': '1029136700'}, {'key': '41', 'value': '1028348931'}, {'key': '42', 'value': '12234454'}]
This implementation is significantly more difficult to read and understand.

Resources