Describe EBS details volume with AWSCLI - linux

I need to list all owner's aws ami and any details about it.
The following line returns the image_id, CreationDate, Name and organized by CreationDate:
aws ec2 describe-images --owners --query 'Images[*].[ImageId, CreationDate, Name]' | sort_by(#, &[1])' --output text
But, i need any information about BlockDevicesMapping like an Ebs volume [snapshot id, Volume size e etc...]
I did the following line aws ec2 describe-images --owners --query 'Images[*].[ImageId, CreationDate, Name, BlockDeviceMappings:[Ebs:{SnapshotID}]] --output text. But, the search on terminal stopped.
I tried a lot of ways.
Thank's for help-me.

The output from describe-images is as follows.
{
"Images": [
{
"VirtualizationType": "hvm",
"Description": "Provided by Red Hat, Inc.",
"PlatformDetails": "Red Hat Enterprise Linux",
"EnaSupport": true,
"Hypervisor": "xen",
"State": "available",
"SriovNetSupport": "simple",
"ImageId": "ami-1234567890EXAMPLE",
"UsageOperation": "RunInstances:0010",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"SnapshotId": "snap-111222333444aaabb",
"DeleteOnTermination": true,
"VolumeType": "gp2",
"VolumeSize": 10,
"Encrypted": false
}
}
],
"Architecture": "x86_64",
"ImageLocation": "123456789012/RHEL-8.0.0_HVM-20190618-x86_64-1-Hourly2-GP2",
"RootDeviceType": "ebs",
"OwnerId": "123456789012",
"RootDeviceName": "/dev/sda1",
"CreationDate": "2019-05-10T13:17:12.000Z",
"Public": true,
"ImageType": "machine",
"Name": "RHEL-8.0.0_HVM-20190618-x86_64-1-Hourly2-GP2"
}
]
}
As you see, BlockDeviceMappings is an array, so you should flatten it first before you attempt to access its objects.
I would suggest specifying --owners 12347989. If you attempt to get all amis of all owners, terminal will hang. I ran the below command with my accountId and got the desired output, but if I don't specify owner, my terminal just hangs.
Try this:
aws ec2 describe-images --owners amazon --query 'Images[*].[ImageId, CreationDate, Name, BlockDeviceMappings[0].DeviceName, BlockDeviceMapping[0].VolumeSize, BlockDeviceMappings[0].SnapshotId]' --output text

Related

How to edit docker volume device option?

How could I change the docker-volume-device option after the creation, With out removing the container
Previous command used for creation
docker volume create \
--driver local \
--opt type=none \
--opt device="/home/userA/Containers/vol1/home" \
--opt o=bind \
$DOCKER_VOLUME_NAME
Current Status
$ docker volume inspect vol1
[
{
"CreatedAt": "XXX",
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/vol1/_data",
"Name": "vol1-home",
"Options": {
"device": "/home/userA/Containers/vol1/home",
"o": "bind",
"type": "none"
},
"Scope": "local"
}
]
Is there any way we can safely alter the value of device "/home/userA/Containers/vol1/home" to "/Containers/vol1/home". I am not want to keep this in specific user.
I would like to change
" --opt device="/home/userA/Containers/vol1/home" "
I would like to see
$ docker volume inspect vol1
[
{
"CreatedAt": "XXX",
"Driver": "local",
"Labels": {},
"Mountpoint": "/var/lib/docker/volumes/vol1/_data",
"Name": "vol1-home",
"Options": {
"device": "/Containers/vol1/home",
"o": "bind",
"type": "none"
},
"Scope": "local"
}
]
Does any one have any suggestion for my problem?
Reason / Benefit of this:
Due to a lack of space in my disk, I moved all files from /var/lib/docker to some other disk and I mounted them as /var/lib/docker. I saved my OS space and separated docker from the os disk.
Now I am trying to separate the volume device also. so I can save more space, Unfortunately, I might lose my container data for doing so.

How to query nested value in AWS Cli?

"MetricAlarms": [
{
"AlarmName": "AWS_CPUUtilisation_Alarm",
"MetricName": "CPUUtilization",
"Namespace": "AWS/EC2",
"Statistic": "Average",
"Dimensions": [
{
"Name": "InstanceId",
"Value": "i-XXXXXXXXX"
}
],
"Period": 300,
"EvaluationPeriods": 1,
},
From the above data how can i get only instance id by using below command I am getting name and value:
aws cloudwatch describe-alarms --query "MetricAlarms[*].{Instanceid:Dimensions}" --output table
You're almost there:
aws cloudwatch describe-alarms --query "MetricAlarms[*].{Instanceid:Dimensions}[*].Instanceid[*].Value"
You might find it useful to play around with a tool like jmespath terminal or jmespath tutorial.

AWS Lambda function update command showing environment variables

I'm using gitlab pipelines to update configuration and code of my AWS Lambda functions. I noticed that when you perform AWS cli requests to update the Lambda like:
aws lambda update-function-configuration --region $your_region --function-name $your_lambda_arn --handler $your_lambda_handler --runtime $your_lambda_runtime
OR
aws lambda update-function-code --region $your_region --function-name $your_lambda_arn --s3-bucket $your_s3_bucket --s3-key $your_project_name.zip
Both requests responses in gitlab console reveals the environment variables (like db connection string) and I would like to keep it secret. Is there any way, any flag or whatsoever to keep these values hidden in response?
Regards
Could perhaps use jq to exclude the output? Otherwise use an SDK (e.g. boto3 or aws-sdk-go) and don't print the entire response.
$ cat ~/Desktop/Me.json
{
"FunctionName": "my-function",
"LastModified": "2019-09-26T20:28:40.438+0000",
"RevisionId": "e52502d4-9320-4688-9cd6-152a6ab7490d",
"MemorySize": 256,
"Version": "$LATEST",
"Role": "arn:aws:iam::123456789012:role/service-role/my-function-role-uy3l9qyq",
"Timeout": 3,
"Runtime": "nodejs10.x",
"TracingConfig": {
"Mode": "PassThrough"
},
"Environment": {
"foo": "bar"
},
"CodeSha256": "5tT2qgzYUHaqwR716pZ2dpkn/0J1FrzJmlKidWoaCgk=",
"Description": "",
"VpcConfig": {
"SubnetIds": [],
"VpcId": "",
"SecurityGroupIds": []
},
"CodeSize": 304,
"FunctionArn": "arn:aws:lambda:us-west-2:123456789012:function:my-function",
"Handler": "index.handler"
}
# no Environment key
$ cat ~/Desktop/Me.json | jq 'del(.Environment)'
{
"FunctionName": "my-function",
"LastModified": "2019-09-26T20:28:40.438+0000",
"RevisionId": "e52502d4-9320-4688-9cd6-152a6ab7490d",
"MemorySize": 256,
"Version": "$LATEST",
"Role": "arn:aws:iam::123456789012:role/service-role/my-function-role-uy3l9qyq",
"Timeout": 3,
"Runtime": "nodejs10.x",
"TracingConfig": {
"Mode": "PassThrough"
},
"CodeSha256": "5tT2qgzYUHaqwR716pZ2dpkn/0J1FrzJmlKidWoaCgk=",
"Description": "",
"VpcConfig": {
"SubnetIds": [],
"VpcId": "",
"SecurityGroupIds": []
},
"CodeSize": 304,
"FunctionArn": "arn:aws:lambda:us-west-2:123456789012:function:my-function",
"Handler": "index.handler"
}
I've found two additional solutions for that case:
Gitlab itself has mechanism of hiding secrets, to achieve that you can setup environment variable in gitlab cicd settings as protected, in that case console output should be hidden where needed
You can filter out the aws cli response
https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-filter.html
Regards

Where to find AWS Systems Manager (SSM) Document ARNs?

I am wanting to run an SSM Document on my EC2 instance, AWSFIS-Run-CPU-Stress.
It is being executed via AWS Fault Injection Simulator (FIS), which requires a documentARN to be specified, specifically in format:
arn:aws:ssm:us-east-1::document/AWSFIS-Run-CPU-Stress.
I have also tried arn:aws:ssm:us-west-2:aws-account-ID:document/AWSFIS-Run-CPU-Stress, replacing 'aws-account-ID' with my AWS account ID.
Here is the AWS FIS documentation that states that you must provide an SSM Document ARN: https://docs.aws.amazon.com/fis/latest/userguide/actions-ssm-agent.html#awsfis-run-cpu-stress
However, when I try to run my Fault Injection experiment with the above ARN (I've tried many different regions), the Document can not be found.
Furthermore, I have tried to use aws ssm describe-document on the Document, but ARN is not returned, so I have no idea what I need to use. It doesn't even look like SSM Documents are region specific.
Here is what is returned by aws ssm describe-document:
{
"Document": {
"Hash": "f2b00b4471e7236ddb11654c4e076473f5e493e916f09840abb229d5a07822b1",
"HashType": "Sha256",
"Name": "Test-AWSFIS-Run-CPU-Stress",
"Owner": "703381282345",
"CreatedDate": "2021-05-10T21:08:14.781000+01:00",
"Status": "Active",
"DocumentVersion": "1",
"Description": "Command Document Example JSON Template",
"Parameters": [
{
"Name": "Message",
"Type": "String",
"Description": "Example",
"DefaultValue": "Hello World"
}
],
"PlatformTypes": [
"Windows",
"Linux",
"MacOS"
],
"DocumentType": "Command",
"SchemaVersion": "2.2",
"LatestVersion": "1",
"DefaultVersion": "1",
"DocumentFormat": "JSON",
"Tags": []
}
}
Where can I find the Document ARN?
Apparently Systems Manager console does not show the document ARN. In the meantime you can construct the ARN following this format:
"arn:<partition>:ssm:<region>:<account_id>:document/<document_name>"
This is a valid example of a public FIS SSM "run command" document (available from any account in us-east-1 region), not sure why it failed in your case. No account id is expected because it's a public document:
"arn:aws:ssm:us-east-1::document/AWSFIS-Run-CPU-Stress"
And this is an example of a private document (i.e. you own it), note that this does have account id:
"arn:aws:ssm:us-east-1:012345678912:document/MyFISFaultDocument"
You can execute the specified document using the following aws ssm send-command --document-name "AWSFIS-Run-CPU-Stress" --document-version "4" --parameters '{"CPU":["0"],"InstallDependencies":["True"]}' --timeout-seconds 600 --max-concurrency "50" --max-errors "0" --region us-east-1
You can obtain the above command line by filling in the parameters from this link https://console.aws.amazon.com/systems-manager/run-command/send-command?region=us-east-1#

CloudFormation without snapshot

Cloudformation created a template for us which specifies both the AMI instance to start from, and also the snapshot ID of that AMI instance.
We create our base AMI instance with Packer, which reports the AMI instance it creates, but does not report the snapshot associated - we find that in the Amazon UI.
Can the Cloudformation template be modified so it does not specify the snapshot ID? Can you give an example of the stanza?
Sure you can! For example, something like this would work:
"Resources": {
"someEC2": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "...valid_ami_id...",
"InstanceType": "m3.medium",
"KeyName": "...",
"Monitoring": "false",
"NetworkInterfaces": [
{
...
}
],
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda",
"Ebs": {
"VolumeSize": 10
}
}
]
}
}
}

Resources