I run this command and it works :
aws elb describe-load-balancers --query 'LoadBalancerDescriptions[?VPCId==`vpc-#########`]|[].LoadBalancerName' --region us-east-2
If I try and use an environemnt variable it does not work :
aws elb describe-load-balancers --query 'LoadBalancerDescriptions[?VPCId==`$VPC_ID`]|[].LoadBalancerName' --region us-east-2
I know that VPC_ID is valid - echo $VPC_ID returns the correct value
What am I not seeing?
Thanks!!!!!
I also tried this command with the same results :
This works fine :
aws elb describe-load-balancers --output text --query 'LoadBalancerDescriptions[?Instances[?InstanceId==`i-0############`]].[LoadBalancerName]' --region us-east-2
This returns nothing :
aws elb describe-load-balancers --output text --query 'LoadBalancerDescriptions[?Instances[?InstanceId=="$InstanceID"]].[LoadBalancerName]' --region us-east-2
I know that the environment variable $InstanceID is populated and correct - I perform an echo $InstanceID and get the correct ID output.
Got it!!
The environment variable need to be in brackets - { }
This works -
aws elb describe-load-balancers --output text --query "LoadBalancerDescriptions[?Instances[?InstanceId=='${InstanceID}']].LoadBalancerName" --region us-east-2
I am able to reproduce this using the following:
export MY_VPC_ID=vpc-1234
echo 'LoadBalancerDescriptions[?VPCId==`$MY_VPC_ID`]|[].LoadBalancerName'
OUTPUT:
LoadBalancerDescriptions[?VPCId==`$MY_VPC_ID`]|[].LoadBalancerName
I believe this has to do with how bash interprets quotes as shown in this other post
Evaluating variables in a string
Can you try using this?
echo "LoadBalancerDescriptions[?VPCId==\"$MY_VPC_ID\"]|[].LoadBalancerName"
OUTPUT:
LoadBalancerDescriptions[?VPCId=="vpc-1234"]|[].LoadBalancerName
Related
I try to list all my AutoScalingGroups with "Desiredcapacity" = 3.
I can run this but it gives me all ASG back and it's to many.
aws autoscaling describe-auto-scaling-groups --query "AutoScalingGroups[*].{NAME:AutoScalingGroupName,DesiredCapacity:DesiredCapacity} | sort_by([], &DesiredCapacity)" --profile MyProfile --output table
I tried :
aws autoscaling describe-auto-scaling-groups --query "AutoScalingGroups[?DesiredCapacity == '3'].{NAME:AutoScalingGroupName,DesiredCapacity:DesiredCapacity} | sort_by([], &DesiredCapacity)" --profile MyProfile --output table
I dont get any error msg but it does not return anything, as it should
I actually improved my search and find a solution.
aws autoscaling describe-auto-scaling-groups --query 'AutoScalingGroups[?DesiredCapacity>=`3`].[AutoScalingGroupName,DesiredCapacity,MinSize,MaxSize]' --output table --profile MyProfile
is exactly what i want.
Right now this following code segment will list the 'Name' tag of the resource of any instance that doesn't have a 'Grant' tag.
for region in `aws ec2 describe-regions --output text | cut -f3`
do
aws ec2 describe-instances \
--region $region \
--output text \
--query 'Reservations[].Instances[?!not_null(Tags[?Key == `Grant`].Value)] | [].Tags[?Key==`Name`].Value'
done
I've tried a few ways to get the Public IP address but I keep getting errors for bad syntax.
Is it possible to pull in the Public IP here?
Yes, It is possible to pull the PublicIp address along with the Tags value.
Replace query syntax as below,
--query 'Reservations[].Instances[?!not_null(Tags[?Key == `Grant`].Value)] | [].[PublicIpAddress, Tags[?Key==`Name`].Value]'
I have the command to list all the RDS endpoints I have running in my aws account but I want to find RDS endpoint for RDS running in the same VPC as the ec2 instance I want to use it from.
I have multiple VPC's up with multiple RDS's so when I issue the command it gives me all the running RDS's. How can i filter this to just show me the one in the same VPC?
I run the command -
aws rds --region us-east-2 describe-db-instances --query "DBInstances[*].Endpoint.Address"
And I get -
"acme-networkstack.vbjrxfom0phf.us-east-2.rds.amazonaws.com",
"acme-aws-beta-network.vbjrxfom0phf.us-east-2.rds.amazonaws.com",
"acme-demo.vbjrxfom0phf.us-east-2.rds.amazonaws.com",
"acme-dev.vbjrxfom0phf.us-east-2.rds.amazonaws.com"
I only want the one endpoint that is in the same VPC as the instance I am running the CLI command from.
Thanks!
Ernie
Here's a little script that should do the trick, just replace the ec2 describe-instanceswith your rds cli command:
#!/bin/bash
mac=`curl -s http://169.254.169.254/latest/meta-data/mac`
vpcID=`curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/$mac/vpc-id`
aws ec2 describe-instances --region eu-west-1 --filter "Name=vpc-id,Values=$vpcID"
You're first curling the instance meta-data to find it's VpcId, and then filtering the outputs of your cli command to limit to a certain vpc.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-output.html
describe-db-instances has a limited set of filters which doesn't include the VPC. The solution I suggest uses a combination of the meta-data information from the host and jq to select only the endpoints that match the VPC.
First, You can get the VPC ID as suggested by WarrenG.
#!/bin/bash
mac=`curl -s http://169.254.169.254/latest/meta-data/mac`
VPC_ID=`curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/$mac/vpc-id`
Then uses the AWS CLI in combination with jq to derive your desired output.
aws rds describe-db-instances | jq -r --arg VPC_ID "VPC_ID" '.DBInstances[] |select (.DBSubnetGroup.VpcId==$VPC_ID) | .Endpoint.Address'
I haven't run this from a script but it works from the command line. If it doesn't work in a script let me know.
References
https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-instances.html
Passing bash variable to jq select
I want to search for a file name abc.zip in s3 buckets and there are nearly 60 buckets and each buckets have 2 to 3 levels subdirectories or folders .I tried to perform search using AWS CLI commands and below are the commands which i tried but even though the file is existing in the bucket.The results are not being displayed for the file.
aws s3api list-objects --bucket bucketname --region ca-central-1 \
--recursive --query "Contents[?contains(Key, 'abc.zip')]"
aws s3 ls --summarize --human-readable --recursive bucketname \
--region ca-central-1 | egrep 'abc.zip'
For all the above commands execution i dont see the filename in command line and when i manually check the bucket the file exists.
Is there any way i can find the file.
Hmm.
I used your command from #1 without "--recursive" because this throws Unknown options: --recursive. The file I was searching for is on the second level of the bucket and it was found. --region is also not used.
My guess is you are using some old version of AWS client or pointing to an incorrect bucket.
My working command:
aws s3api list-objects --bucket XXXXX --query "Contents[?contains(Key, 'animate.css')]"
[
{
"LastModified": "2015-06-14T23:29:03.000Z",
"ETag": "\"e5612f9c5bc799b8b129e9200574dfd2\"",
"StorageClass": "STANDARD",
"Key": "css/animate.css",
"Owner": {
"DisplayName": "XXXX",
"ID": "XXXX"
},
"Size": 78032
}
]
If you decide to upgrade your CLI client: https://github.com/aws/aws-cli/tree/master
Current version is awscli-1.15.77 which you may check by aws --version.
I tried in the following way
aws s3 ls s3://Bucket1/folder1/2019/ --recursive |grep filename.csv
This outputs the actual path where the file exists
2019-04-05 01:18:35 111111 folder1/2019/03/20/filename.csv
Hope this helps!
I know this is ancient, but I found a way to do this without piping text to grep...
aws s3api list-objects-v2 --bucket myBucket --prefix 'myFolder' \
--query "Contents[*]|[?ends_with(Key,'jpg')].[Key]"
I think previous answers are correct but if you want make this, bucket agnostic, then you can use the below script all you have to do is change the variable value (search_value) on the first line to what you
are searching for and add your id and secret:
#!/usr/bin/sh
export AWS_ACCESS_KEY_ID=your_key; export AWS_SECRET_ACCESS_KEY=your_secret;
search_value="3ds"
my_array=( `aws s3api list-buckets --query "Buckets[].Name"|grep \" |sed 's/\"//g'|sed 's/\,//g'` )
my_array_length=${#my_array[#]}
for element in "${my_array[#]}"
do
echo "----- ${element}"
aws s3 ls s3://"${element}" --recursive |grep -i $search_value
done
Warning....it will search every single bucket in your account so be prepared for a long search....
It does pattern search so it will find any words that contains the value
Lastly this is case insensitive search ... (you can disable that by removing -i from grep line)
done
I am almost there, the "$i" is where I am having trouble. I have tried ${i}, "$i", $i. I am sure someone with more experience can help me here I have been working on this for 1 full day. Driving me nuts.
session_name="Some-sesh_name"
profile_name="ephemeral-${account_id}-${profile_path}-`date +%Y%m%d%H%M%S`"
roles=( "arn:aws:iam::11111111111111:role/role_name" "arn:aws:iam::222222222222:role/role_name" )
sts=( $(
aws sts assume-role \
--role-arn "$i" \
--role-session-name "$session_name" \
--query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' \
--output text
) )
for i in "${roles[#]}";
do $sts ; done
aws configure set aws_access_key_id ${sts[0]} --profile ${profile_name}
aws configure set aws_secret_access_key ${sts[1]} --profile ${profile_name}
aws configure set aws_session_token ${sts[2]} --profile ${profile_name}
That $i is expanded at the moment you define the sts array. After that, it doesn't exist.
To make that aws command reusable, use a function:
roles=(
"arn:aws:iam::11111111111111:role/role_name"
"arn:aws:iam::222222222222:role/role_name"
)
sts() {
aws sts assume-role \
--role-arn "$1" \
--role-session-name "$session_name" \
--query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' \
--output text
}
for role in "${roles[#]}"; do
sts "$role"
done
Note the use of $1 in the function, to retrieve the first argument. The global variable $session_name is still OK
I don't understand what you're thinking with the sts array. In the for loop you want to call it as a command, but the configure commands take elements of the array? After all the roles have been assumed? Are you wanting to use the returned data instead?
Do you want:
for role in "${roles[#]}"; do
data=( $(sts "$role") )
aws configure set aws_access_key_id "${data[0]}" --profile "$profile_name"
aws configure set aws_secret_access_key "${data[1]}" --profile "$profile_name"
aws configure set aws_session_token "${data[2]}" --profile "$profile_name"
done
?