I'm trying to get values based on Dimensions and tags from costexplorer client using get_cost_and_usage, but I keep getting errors for bad formats.
Client: CostExplorer
Filters: Tags and Dimensions
My actual Filter block looks like:
And: [{
Dimensions: 'Key': 'USAGE_TYPE',
'Values': [
'EU-EBS:VolumeUsage.gp2',
'EU-EBS:VolumeUsage.gp3',
'EU-EBS:VolumeUsage.st1'
],
Tags: {
'Key': 'tag',
'Values': ['example']
}
}]
How do I add an A record for a lb where I'm able to add more then one ipaddress to the same record. I the console it is possible, but I'm not sure how to do it in Python. Below is my try which only add the last Value to the record.
#!/usr/bin/env python3
import boto3
#TODO #use env variables for names, Values and zonename
def lambda_handler(event, context):
client = boto3.client('route53')
response = client.change_resource_record_sets(
HostedZoneId='Z03115902SB93XHRQS9LT',
ChangeBatch={
'Changes': [
{
'Action': 'UPSERT',
'ResourceRecordSet': {
'Name': 'web-staging-lb.ggnp3ggdjcvwpfpqhsuwda.soemdomain.com',
'Type': 'A',
'TTL': 60,
'ResourceRecords': [
{
'Value': '10.201.11.246',
'Value': '10.201.10.12',
},
],
},
},
],
'Comment': 'Record to acces private ips of alb',
},
)
ResourceRecords is a list that can contain multiple elements.
'ResourceRecords': [
{
'Value': '10.201.10.12',
},
{
'Value': '10.201.11.246',
},
Managing an A record with multiple IPs in Route53 with python boto3
I wrote some code to get data from a web API. I was able to parse the JSON data from the API, but the result I gets looks quite complex. Here is one example:
>>> my_json
{'name': 'ns1:timeSeriesResponseType', 'declaredType': 'org.cuahsi.waterml.TimeSeriesResponseType', 'scope': 'javax.xml.bind.JAXBElement$GlobalScope', 'value': {'queryInfo': {'creationTime': 1349724919000, 'queryURL': 'http://waterservices.usgs.gov/nwis/iv/', 'criteria': {'locationParam': '[ALL:103232434]', 'variableParam': '[00060, 00065]'}, 'note': [{'value': '[ALL:103232434]', 'title': 'filter:sites'}, {'value': '[mode=LATEST, modifiedSince=null]', 'title': 'filter:timeRange'}, {'value': 'sdas01', 'title': 'server'}]}}, 'nil': False, 'globalScope': True, 'typeSubstituted': False}
Looking through this data, I can see the specific data I want: the 1349724919000 value that is labelled as 'creationTime'.
How can I write code that directly gets this value?
I don't need any searching logic to find this value. I can see what I need when I look at the response; I just need to know how to translate that into specific code to extract the specific value, in a hard-coded way. I read some tutorials, so I understand that I need to use [] to access elements of the nested lists and dictionaries; but I can't figure out exactly how it works for a complex case.
More generally, how can I figure out what the "path" is to the data, and write the code for it?
For reference, let's see what the original JSON would look like, with pretty formatting:
>>> print(json.dumps(my_json, indent=4))
{
"name": "ns1:timeSeriesResponseType",
"declaredType": "org.cuahsi.waterml.TimeSeriesResponseType",
"scope": "javax.xml.bind.JAXBElement$GlobalScope",
"value": {
"queryInfo": {
"creationTime": 1349724919000,
"queryURL": "http://waterservices.usgs.gov/nwis/iv/",
"criteria": {
"locationParam": "[ALL:103232434]",
"variableParam": "[00060, 00065]"
},
"note": [
{
"value": "[ALL:103232434]",
"title": "filter:sites"
},
{
"value": "[mode=LATEST, modifiedSince=null]",
"title": "filter:timeRange"
},
{
"value": "sdas01",
"title": "server"
}
]
}
},
"nil": false,
"globalScope": true,
"typeSubstituted": false
}
That lets us see the structure of the data more clearly.
In the specific case, first we want to look at the corresponding value under the 'value' key in our parsed data. That is another dict; we can access the value of its 'queryInfo' key in the same way, and similarly the 'creationTime' from there.
To get the desired value, we simply put those accesses one after another:
my_json['value']['queryInfo']['creationTime'] # 1349724919000
I just need to know how to translate that into specific code to extract the specific value, in a hard-coded way.
If you access the API again, the new data might not match the code's expectation. You may find it useful to add some error handling. For example, use .get() to access dictionaries in the data, rather than indexing:
name = my_json.get('name') # will return None if 'name' doesn't exist
Another way is to test for a key explicitly:
if 'name' in resp_dict:
name = resp_dict['name']
else:
pass
However, these approaches may fail if further accesses are required. A placeholder result of None isn't a dictionary or a list, so attempts to access it that way will fail again (with TypeError). Since "Simple is better than complex" and "it's easier to ask for forgiveness than permission", the straightforward solution is to use exception handling:
try:
creation_time = my_json['value']['queryInfo']['creationTime']
except (TypeError, KeyError):
print("could not read the creation time!")
# or substitute a placeholder, or raise a new exception, etc.
Here is an example of loading a single value from simple JSON data, and converting back and forth to JSON:
import json
# load the data into an element
data={"test1": "1", "test2": "2", "test3": "3"}
# dumps the json object into an element
json_str = json.dumps(data)
# load the json to a string
resp = json.loads(json_str)
# print the resp
print(resp)
# extract an element in the response
print(resp['test1'])
Try this.
Here, I fetch only statecode from the COVID API (a JSON array).
import requests
r = requests.get('https://api.covid19india.org/data.json')
x = r.json()['statewise']
for i in x:
print(i['statecode'])
Try this:
from functools import reduce
import re
def deep_get_imps(data, key: str):
split_keys = re.split("[\\[\\]]", key)
out_data = data
for split_key in split_keys:
if split_key == "":
return out_data
elif isinstance(out_data, dict):
out_data = out_data.get(split_key)
elif isinstance(out_data, list):
try:
sub = int(split_key)
except ValueError:
return None
else:
length = len(out_data)
out_data = out_data[sub] if -length <= sub < length else None
else:
return None
return out_data
def deep_get(dictionary, keys):
return reduce(deep_get_imps, keys.split("."), dictionary)
Then you can use it like below:
res = {
"status": 200,
"info": {
"name": "Test",
"date": "2021-06-12"
},
"result": [{
"name": "test1",
"value": 2.5
}, {
"name": "test2",
"value": 1.9
},{
"name": "test1",
"value": 3.1
}]
}
>>> deep_get(res, "info")
{'name': 'Test', 'date': '2021-06-12'}
>>> deep_get(res, "info.date")
'2021-06-12'
>>> deep_get(res, "result")
[{'name': 'test1', 'value': 2.5}, {'name': 'test2', 'value': 1.9}, {'name': 'test1', 'value': 3.1}]
>>> deep_get(res, "result[2]")
{'name': 'test1', 'value': 3.1}
>>> deep_get(res, "result[-1]")
{'name': 'test1', 'value': 3.1}
>>> deep_get(res, "result[2].name")
'test1'
I have a requirement to get the name of the autoscaling group based on its tags.
I have tried following code:
kwargsAsgTags = {
'Filters': [
{
'Name': 'key',
'Values': ['ApplicationName']
},
{
'Name': 'value',
'Values': ['my-app-name']
}
]
}
by using above filter I can get the autoscaling group name but since I have same 'ApplicationName' tag used in multiple environments like dev/qa/uat, the output prints all autoscaling groups belong to all environments. How do I filter the EnvironmentName as well?
For that I've tried following but this time it prints all auto-scaling groups belonging to 'dev' environment as well.
kwargsAsgTags = {
'Filters': [
{
'Name': 'key',
'Values': ['ApplicationName', 'EnvName']
},
{
'Name': 'value',
'Values': ['my-app-name', 'dev']
}
]
}
I would like to tag the host that I am spining up using boto3 python api
response = client.allocate_hosts(
AutoPlacement='on'|'off',
AvailabilityZone='string',
ClientToken='string',
InstanceType='string',
Quantity=123,
TagSpecifications=[
{
'ResourceType': 'dedicated-host',
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
},
])
Here is what I am doing
Availability Zone,Instance Type , Quantity are parameterized and I use dictionary to input data
count = 10
input_dict = {}
input_dict['AvailabilityZone'] = 'us-east-1a'
input_dict['InstanceType'] = 'c5.large'
input_dict['Quantity'] = int(count)
instance = client.allocate_hosts(**input_dict,)
print(str(instance))
This code works for me but i need to tag the resource too
TagSpecifications=[
{
'ResourceType': 'customer-gateway'|'dedicated-host'|'dhcp-options'|'elastic-ip'|'fleet'|'fpga-image'|'image'|'instance'|'internet-gateway'|'launch-template'|'natgateway'|'network-acl'|'network-interface'|'reserved-instances'|'route-table'|'security-group'|'snapshot'|'spot-instances-request'|'subnet'|'transit-gateway'|'transit-gateway-attachment'|'transit-gateway-route-table'|'volume'|'vpc'|'vpc-peering-connection'|'vpn-connection'|'vpn-gateway',
'Tags': [
{
'Key': 'string',
'Value': 'string'
},
]
},
]
how can I input that into the dictionary .. It seems like tag specification has dictionary inside dict .. I am making syntax errors. I tried the below code without success.
input_dict['TagSpecifications'] = [{'ResourceType':'dedicated-host','Tags':[{'key':'Name','Value':'demo'},]},]
The easiest way is to simply pass values directly:
response = client.allocate_hosts(
AvailabilityZone='us-east-1a',
InstanceType='c5.large',
Quantity=10,
TagSpecifications=[
{
'ResourceType': 'dedicated-host',
'Tags': [
{
'Key': 'Name',
'Value': 'Demo'
}
]
}
])