as per documentation you can pass the multiple security groups using security_groups as a list inside aws_cdk.aws_lambda.Function then it should simply worl so, I have initialized
sg_names = ['sg-a', 'sg-b', 'sg-c', 'sg-d']
now when I try to call, lambda. Function like below:
Function(self, 'lambda_a',
code=_lambda.Code.from_asset('lambdas'),
handler='lambda_a.handler',
function_name='lambda_a',
runtime=_lambda.Runtime.PYTHON_3_8,
role=lambda_role,
vpc=vpc,
security_groups=sg_names,
timeout=cdk.Duration.seconds(30),
)
Now it throws me error:
Expected object reference, got "sg-a"
it seems I should get the objects instead of the values.
So, maybe i think I need to call SecurityGroup.from_security_group_id, but I don't see how to add multiple groups to this call.
Is there any suggestion how to add multiple security groups to lambda?
You can use below code, I am a beginner in python, please adjust if there is some mistake :)
def security_group(id):
return SecurityGroup.from_security_group_id(self, id, id)
sg_names = ['sg-a', 'sg-b', 'sg-c', 'sg-d']
security_groups = map(security_group, sg_names)
Related
I was Googling around to understand how boto3 paginator works, and found a solution that potentially doesn't require writing any logic with NextToken and While loops.
Still, I'm not quite sure what I'm getting when I'm using this:
client = boto3.client('ec2', region_name='eu-west-1')
results = (
client.get_paginator('describe_instances')
.paginate()
.build_full_result()
)
print(results)
I got a huge JSON output and I'm not sure whether I got what I wanted, which is basically the output of all of my EC2 instances.
I'm also not sure how to loop over it, I keep getting TypeError: string indices must be integers which didn't happen before when I used something like:
for instance in response_iterator:
instance = instance['Reservations'][0]
instance_id = instance['Instances'][0]['InstanceId']
print(instance_id)
I would love to understand how to use the build_full_result() method.
I saw a post that says that it's not documented yet, pretty recent to now (as of writing this post).
Interesting find.. this isn't mentioned anywhere in the latest version of boto3 documentation, however it does appear to properly return all available results.
Below is an example from Lambda that shows how to perform a simple loop through the response.. you can update the last two lines to handle the response syntax from EC2 describe instances.
import boto3
client = boto3.client('lambda')
results = (
client.get_paginator('list_functions')
.paginate()
.build_full_result()
)
for result in results['Functions']:
print(result['FunctionName'])
I am trying to list of all Parameters along with all their tags, I am trying to do so without listing the value of the parameters.
My initial approach was to do a describe_parameters and then loop through the Key Names and then perform list_tags, while doing so I found out that the ARNs are needed to perform list_tags which are not returned in the describe parameters.
Is there a way to get the parameters along with their tags without actually getting the parameters?
You can do this with the resource groups tagging api IF THEY ARE ALREADY TAGGED. Here's a basic example below without pagination.
import boto3
profile = "your_profile_name"
region = "us-east-1"
session = boto3.session.Session(profile_name=profile, region_name=region)
client = session.client('resourcegroupstaggingapi')
response = client.get_resources(
ResourceTypeFilters=[
'ssm',
],
)
print(response)
If you're wanting to discover untagged parameters, this won't work. Better would be to setup config rules to highlight these issues without you having to manage searching for them.
I saw a few similar posts, but unfortunately none helped me.
I have an s3 bucket (on scaleway), and I'm trying to simply list all objects contained in that bucket, using boto3 s3 client as follow:
s3 = boto3.client('s3',
region_name=AWS_S3_REGION_NAME,
endpoint_url=AWS_S3_ENDPOINT_URL,
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY
)
all_objects = s3.list_objects_v2(Bucket=AWS_STORAGE_BUCKET_NAME)
This simple piece of code responds with an error:
botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the ListObjects operation: The specified key does not exist.
First, the error seems inapropriate to me since I'm not specifying any key to search. I also tried to pass a Prefix argument to this method to narrow down the search to a specific subdirectory, same error.
Second, I tried to achieve the same thing using boto3 Resource rather than Client, as follow:
session = boto3.Session(
region_name=AWS_S3_REGION_NAME,
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY
)
resource = session.resource(
's3',
endpoint_url=AWS_S3_ENDPOINT_URL,
)
for bucket in resource.buckets.all():
print(bucket.name)
That code produces absolutely nothing. One weird thing that strikes me is that I don't pass the bucket_name anywhere here, which seems to be normal according to aws documentation
There's no chance that I misconfigured the client, since I'm able to use the put_object method perfectly with that same client. One strange though: when I want to put a file, I pass the whole path to put_object as Key (as I found it to be the way to go), but the object is inserted with the bucket name prepend to it. So let's say I call put_object(Key='/path/to/myfile.ext'), the object will end up to be /bucket-name/path/to/myfile.ext.
Is this strange behavior the key to my problem ? How can I investigate what's happening, or is there another way I could try to list bucket files ?
Thank you
EDIT: So, after logging the request that boto3 client is sending, I noticed that the bucket name is append to the url, so instead of requesting https://<bucket_name>.s3.<region>.<provider>/, it requests https://<bucket_name>.s3.<region>.<provider>/<bucket-name>/, which is leading to the NoSuchKey error.
I took a look into the botocore library, and I found this:
url = _urljoin(endpoint_url, r['url_path'], host_prefix)
in botocore.awsrequest line 252, where r['url_path'] contains /skichic-bucket?list-type=2. So from here, I should be able to easily patch the library core to make it work for me.
Plus, the Prefix argument is not working, whatever I pass into it I always receive the whole bucket content, but I guess I can easily patch this too.
Now it's not satisfying, since there's no issue related to this on github, I can't believe that the library contains such a bug that I'm the first one to encounter.
Does anyone can explain this whole mess ? >.<
For those who are facing the same issue, try changing your endpoint_url parameter in your boto3 client or resource instantiation from https://<bucket_name>.s3.<region>.<provider> to https://s3.<region>.<provider> ; i.e for Scaleway : https://s3.<region>.scw.cloud.
You can then set the Bucket parameter to select the bucket you want.
list_objects_v2(Bucket=<bucket_name>)
you can try this. you'll have to use your resource instead of my s3sr.
s3sr = resource('s3')
bucket = 'your-bucket'
prefix = 'your-prefix/' # if no prefix, pass ''
def get_keys_from_prefix(bucket, prefix):
'''gets list of keys for given bucket and prefix'''
keys_list = []
paginator = s3sr.meta.client.get_paginator('list_objects_v2')
# use Delimiter to limit search to that level of hierarchy
for page in paginator.paginate(Bucket=bucket, Prefix=prefix, Delimiter='/'):
keys = [content['Key'] for content in page.get('Contents')]
print('keys in page: ', len(keys))
keys_list.extend(keys)
return keys_list
keys_list = get_keys_from_prefix(bucket, prefix)
After looking more closely into things, I've found out that (a lot) of botocore services endpoints patterns starts with the bucket name. For example, here's the definition of the list_objects_v2 service:
"ListObjectsV2":{
"name":"ListObjectsV2",
"http":{
"method":"GET",
"requestUri":"/{Bucket}?list-type=2"
},
My guess is that in the standard implementation of AWS S3, there's a genericendpoint_url (which explains #jordanm comment) and the targeted bucket is reached through the endpoint.
Now, in the case of Scaleway, there's an endpoint_url for each bucket, with the bucket name contained in that url (e.g https://<bucket_name>.s3.<region>.<provider>), and any endpoint should directly starts with a bucket Key.
I made a fork of botocore where I rewrote every endpoint to remove the bucket name, if that can help someone in the future.
Thank's again to all contributors !
According to this answer one can retrieve immediate "subdirectories" by querying by prefix and then obtaining CommonPrefix of the result of Client.list_objects() method.
Unfortunately, Client is a part of so-called "low level" API.
I am using different API:
session = Session(aws_access_key_id=access_key,
aws_secret_access_key=secret_key)
s3 = session.resource('s3')
my_bucket = s3.Bucket(bucket_name)
result = my_bucket.objects.filter(Prefix=prefix)
and this method does not return dictionary.
Is it possible to obtain common prefixes with higher level API in boto3?
As noted in this answer, it seems that the Resource doesn't handle Delimiter well. It is often annoying, when your entire stack relies on Resource, to be told that, ah, you should have instantiated a Client instead...
Fortunately, a Resource object, such as your Bucket above, contains a client as well.
So, instead of the last line in your code sample, do:
paginator = my_bucket.meta.client.get_paginator('list_objects')
for resp in paginator.paginate(Bucket=my_bucket.name, Prefix=prefix, Delimiter='/', ...):
for x in resp.get('CommonPrefixes', []):
print(x['Prefix'])
You can access client from session.
session.client('s3').list_objects(Bucket=bucket_name, Prefix= prefix)
I am strugling to get resource ID based on context with babelTranslation?
Here is my example
[[!getContext]]
[[BabelTranslation? &contextKey=`[[!getContext]]` &resourceId=`4`]]
Snippet getContext
<?php
$contextKey = $modx->context->key;
return $contextKey
With this approach I alsways get nothing back, i dont know why, i hope it will return ID od translated resource from context
;
I don't think you need the [[!getContext]] in the &contextKey property there, the following snippet call should work better:
[[BabelTranslation? &contextKey=`other-context` &resourceId=`4`]]
The reason for that is the &contextKey property is used to determine the context of the translated resource. In your case, that was trying to load a translation in the current context, for a resource that is also in the current context. As there is no translation there, it returns nothing.
By specifying a different context that actually has a translation instead of the original resource, it can properly look it up.