first of all thanks for taking the time in helping me out on this one.
I have a 12300 long list of snapshots, working on deleting certain snapshots, so im trying to list them all first thru the CLI.
I want to get the SnapshotID, the StartTime, and from the tags, the 'Name'
I tried quite a few querys, but all of them result in null on the name :/
THsi is my latest one:
aws ec2 describe-snapshots --query 'Snapshots[*].{ID:SnapshotId,Time:StartTime,Name:Tags[?Key=='Name'].Value[*]}'
Is this something one can do? or should i query all Key pairs, and then filter them out with --filters ?
Few issues to be considered:
Beware of the type of quote marks around the Key Names(backticks, not single quotes)
Forcing a single value out of the tag array.
You should specify the --owner-ids otherwise all accessible snapshots will be listed (including ones that don't belong to your account)
This command works:
aws ec2 describe-snapshots --query 'Snapshots[*].{ID:SnapshotId,Time:StartTime,Name:Tags[?Key==`Name`]|[0].Value}' --owner-ids *<YOUR-ACCOUNT-ID>*
Related
So I am trying to get the ID of all VMs across all subscriptions and regions, where a specific tag is null. For this I am using the following command
az vm list -d --query '[?!not_null(tags.run)]|[].id'
Please note: I want to get the ids only if the tag doesn't exist
Here notice I need to use single quotes to cover the query as I am using the '!' operator to inverse the not_null() function. If I were to use double quotes bash will throw an event not found error.
So now the problem arises when I also want to add a condition to check the current state of the VM and return id only if it is running and tag doesn't exist.
az vm list -d --query '[?!not_null(tags.run)] | [?powerState=="VM running"].id'
Here I have to wrap VM running in double quotes and this gives me an empty output as the string is not being matched because the query expects single quotes like so -
"[?powerState=='VM running'].id"
Could someone help me with a workaround for this?
Use raw string literals for VM running string. You just have to surround your string with a back tick and a double quote.
az vm list -d --query '[?!not_null(tags.run)]|[?powerState==`"VM running"`].id'
My team is working to integrate an infrastructure-as-code scanning solution into our build pipelines and we've discovered that the string "GCP" is being replaced with three asterisks when tasks are being executed in our build pipelines. This isn't unique to one task either whereas I created a bash script to execute and list our our repository and all directories that start with "GCP" are replaced by the three asterisks. The only variable set using the "GCP" value is the "system.teamProject" variable and we are not using any secret values that I know of and there are no variable groups used.
Any help would be greatly appreciated. Thanks!
Bash Asterisk Output "ls -a"
IaC Scanning Asterisk Task Failure
If you have set any secret variables in your pipeline, or have linked any variable groups that contain secret variables (include the secrets from the connected external and remote services services), generally the values of these secrets will be masked as asterisks.
When you try to print the values of the secrets to the output logs, the values will display as asterisks in the logs. If you try to output the values into a text file, the values will still display as asterisks in the file.
In addition, if a string that is not set as secret but its substrings are the values of some existing secrets in the pipeline, these substring parts may be masked as asterisks when trying to output this string.
If you do not set any secrets, for us to investigate this issue further, would you like to share us with the actual value that was masked as asterisks in the the logs? We well investigate and evaluate whether this string contains some special or sensitive characters that may be automatically identified as secrets by Azure DevOps.
I am trying to "explore" json results from an Azure CLI command using the --query switch (e.g. az functionapp list --query <something>), and to get started I'd like the length of the resulting array.
The Azure CLI help says nothing specific, and points to jmespath.org which does indeed show that a length function exists, however it seems to require an argument. I have no name for the argument, which is the root/outermost array returned by the list command.
It seems from jmespath.org that length(something) is what I want, but I don't know what to put in for the "something" part. What do I put here? Or am I going about this all wrong??
As we know az functionapp list returns a json where the root node is an array. In order to get the length of this array we can use the following syntax:
az functionapp list --query "[] | length(#)"
I need a list of RDS DBInstanceIdentifier that match the String "foobar" in their name. I found many solutions with exact match, but not substring matching. My approach looks as follows:
I get a list of all DBInstanceIdentifier using:
aws rds describe-db-instances --query "DBInstances[*].[DBInstanceIdentifier][]"
which looks like
[
"machine-001-alice-abcdefg",
"machine-002-bob-abcdefg",
"machine-003-foobar-abcdefg"
]
On the list I apply a filter like in the last example of the JMSES Tutorial
aws rds describe-db-instances --query "DBInstances[*].[DBInstanceIdentifier][]|[?contains(#,'dev') =='true']"
If I change the statement to != I get the full list, so it seems I have the filter statement wrong.
true needs to be backticked not quoted and then the backticks need to be escaped it seems - different shells may vary.
aws rds describe-db-instances --query "DBInstances[*].[DBInstanceIdentifier][]|[?contains(#,'dev')==\`true\`]"
aws rds describe-db-instances --query "DBInstances[*].[DBInstanceIdentifier][]|[?contains(#,'dev')!=\`true\`]"
You can also omit the comparison to true but I couldn't invert this successfully
aws --profile pollen-nonprod rds describe-db-instances --query "DBInstances[*].[DBInstanceIdentifier][]|[?contains(#,'dev')]"
(I'd normally do this sort of thing with jq but that's a different solution rather than necessarily a better one)
I am using:
aws s3api list-objects --endpoint-url https://my.end.point/ --bucket my.bucket.name --query 'Contents[].Key' --output text
to get the list of files in a bucket.
The aws s3api list-object documentation page says that this command returns only up to a 1000 objects, however I noticed that in my case it returns the names of all files in my bucket. For example when I run the following command:
aws s3api list-objects --endpoint-url https://my.end.point/ --bucket my.bucket.name --query 'Contents[].Key' --output text | tr "\t" "\n" | wc -l
I get 13512 displayed, meaning that more than 13 thousand file names were returned.
Am I missing smth?
I use the following aws cli version:
aws-cli/1.10.57 Python/2.7.3 Linux/3.2.0-4-amd64 botocore/1.4.47
Returns some or all (up to 1000) of the objects in a bucket. You can use the request parameters as selection criteria to return a subset of the objects in a bucket. [1]
I think that the part "(up to 1000)" in the documentation's description is highly misleading. It refers to the maximal page size per underlying HTTP request which is sent by the cli. The documentation of the --page-size option makes this clear:
The size of each page to get in the AWS service call. This does not affect the number of items returned in the command's output. Setting a smaller page size results in more calls to the AWS service, retrieving fewer items in each call. This can help prevent the AWS service calls from timing out.
It gets even clearer when reading the AWS documentation about pagination [2] which describes:
For commands that can return a large list of items, the AWS Command Line Interface (AWS CLI) adds three options that you can use to control the number of items included in the output when the AWS CLI calls a service's API to populate the list.
By default, the AWS CLI uses a page size of 1000 and retrieves all available items. For example, if you run aws s3api list-objects on an Amazon S3 bucket that contains 3,500 objects, the CLI makes four calls to Amazon S3, handling the service-specific pagination logic for you in the background and returning all 3,500 objects in the final output.
As Ankit already stated correctly, using the --max-items option is the correct solution to limit the result and stop the automatic pagination:
To include fewer items at a time in the AWS CLI output, use the --max-items option. The AWS CLI still handles pagination with the service as described above, but prints out only the number of items at a time that you specify. [2]
References
[1] https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html
[2] https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-pagination.html
Try using --max-items with the command.
The doc mentions it will return NextMarker when the no of items are more than max-items. You can pass it as starting-token in the next call to achieve pagination.