How to specify metadata_service_timeout during boto client creation? - python-3.x

I'm using the following way to create a botoclient
def create_boto_client(self, service_name: str):
return boto3.session.Session().client(
service_name=service_name
)
# or I can do: return boto3.client(service_name)
How I could set the "metadata_service_timeout" and "metadata_service_num_attempts" values?
I need to specify them, because otherwise boto can fail to fetch credentials in IAM role from Metadata server.
I read in doc that it's possible to specify this value like this:
# ~/.aws/config
[default]
metadata_service_timeout = 5.0
metadata_service_num_attempts = 10
but I don't have an ability to add this file to instance.

You can use env variables:
AWS_METADATA_SERVICE_TIMEOUT: The number of seconds before a connection to the instance metadata service should time out. When attempting to retrieve credentials on an EC2 instance that has been configured with an IAM role, a connection to the instance metadata service will time out after 1 second by default. If you know you are running on an EC2 instance with an IAM role configured, you can increase this value if needed.
AWS_METADATA_SERVICE_NUM_ATTEMPTS: When attempting to retrieve credentials on an EC2 instance that has been configured with an IAM role, boto3 will only make one attempt to retrieve credentials from the instance metadata service before giving up. If you know your code will be running on an EC2 instance, you can increase this value to make boto3 retry multiple times before giving up.
To set them from inside python:
os.environ["AWS_METADATA_SERVICE_TIMEOUT"] = "5.0"
os.environ["AWS_METADATA_SERVICE_NUM_ATTEMPTS"] = "10"

Related

Count the number of EC2 instances cross-account

I need to create a python lambda function which check a set of conditions.
One of the is to count the number of running ec2 instances with a specific name from another aws account.
I searched stackoverflow and found something like this, but this should only count the instances from the same account/region.
def ec2(event, context):
ec2_resource = boto3.resource('ec2')
instances = [instance.state['Name'] for instance in ec2_resource.instances.all()]
ec2_running_instances = instances.count('running')
print(ec2_running_instances)
You can't do this directly from your account. You must assume IAM role that is created in the second account, with permissions to describe the instances. Please check: Delegate access across AWS accounts using IAM roles .
Once the role exists, you have to use boto3's assume_role to assume the role, get temporary aws credentials, and then create new boto3 session with that credentials.

is there a way to get name of IAM role attached to an EC2 instance with boto3?

I am trying to list down all the EC2 instances with its IAM role attached using boto3 in python3. But I don't find any method to get the IAM role attached to existing EC2 instance. is there any method in boto3 to do that ?
When I describe an Instance, It has a key name IamInstanceProfile. That contains instance profile id and arn of the iam instance profile. I don't find name of IAM instance profile or any other info about IAM roles attached to it. I tried to use instance profile id to describe instance profile, But it seems to describe an instance profile, we need name of instance profile (not the id).
Can someone help on this ? I might be missing something.
Thanks
When we describe EC2 instance, We get IamInstanceProfile key which has Arn and id.
Arn has IamInstanceProfile name attached to it.
Arn': 'arn:aws:iam::1234567890:instance-profile/instanceprofileOrRolename'
This name can be used for further operation like get role description or listing policies attached to role.
Thanks
You can get the metadata of an EC2 instance by making an HTTP call to http://169.254.169.254/latest/meta-data/ from the instance.
In your case, you may want to navigate to http://169.254.169.254/latest/meta-data/iam/security-credentials to get the IAM role attached to the instance.
EC2 Instance Metadata
You can call IAM list_instance_profiles(), and then filter the results by the ARN or ID from EC2 describe_instances() result. This response will contain Roles for given instance profile.
Boto3 IAM documentation

User permissions to authorize AWS CLI via metadata inside EC2

I'm trying to get access from my EC2 instance to S3 bucket via CLI, but I cannot do this with my user.
The instance is build with ansible and got 2 users one is ec2-user and the other test-user. When connecting to the instance via SSH and user ec2-user I can easily download the files from s3 bucket, but when logging with the second user I got An error occurred (403) when calling the HeadObject operation: Forbidden.
What permissions should I gave the test-user to be able to get data from bucket?
Based on the comments.
The instance using instance role, thus both users have equal access to s3. However, the access was failing for test-user since the user was using different IAM credentials then those from instance role.
The credentials came from env variables or .aws/ which have higher priority then instance role, resulting in access deny.

How to setup awscli without setting up access key & secret access key?

I tried to find set aws-cli locally using IAM role & without using access key/secret access key. But unable to get information from meta url[http://169.256.169.256/latest/meta-data].
I am running Ec2 instance with Ubuntu Server 16.04 LTS (HVM), SSD Volume Type - ami-f3e5aa9c.I have tried to configure aws-cli on that instance.I am not sure what type of role/policy/user needed to get aws-cli configured in my Ec2 instance.
Please provide me step by step guide to achieve that.I just need direction.So useful link also appreciated.
To read Instance Metadata, you dont need to configure the AWS CLI. The problem in your case, is you are using a wrong URL to read the Instance Metadata. The correct URL to use is http://169.254.169.254/ . For example, if you want to read the AMI id of the Instance, you can use the follow command.
curl http://169.254.169.254/latest/meta-data/ami-id
However, if you would like to configure the AWS cli without using the Access/Secret Keys. Follow the below steps.
Create an IAM instance profile and Attach it to the EC2 instance
Open the IAM console at https://console.aws.amazon.com/iam/.
In the navigation pane, choose Roles, Create role.
On the Select role type page, choose EC2 and the EC2 use case. Choose Next: Permissions.
On the Attach permissions policy page, select an AWS managed policy that
grants your instances access to the resources that they need.
On the Review page, type a name for the role and choose Create role.
Install the AWS CLI(Ubuntu).
Install pip if it is not installed already.
`sudo apt-get install python-pip`
Install AWS CLI.
`pip install awscli --upgrade --user`
Configure the AWS CLI. Leave AWS Access Key ID and AWS Secret Access
Key as blank as we want to use a Role.
$ aws configure
AWS Access Key ID [None]:
AWS Secret Access Key [None]:
Default region name [None]: us-west-2
Default output format [None]: json
Modify the Region and Output Format values if required.
I hope this Helps you!
AWS Documentation on how to setup an IAM role for EC2
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

How to connect IAM login name with an EC2 instance?

I am using the boto3 Python3 module provided by AWS. I'm able to extract the security key name and the various tags associated with an EC2 instance. Unfortunately, none of that information tells me who created the instance.
Is there a way to use AWS IAM to see what active instances were created by a user?

Resources