aws s3 ls Unknown options: --recursive - linux

I am trying to list the contents of an Amazon S3 bucket using the following command (documentation):
aws s3 ls s3://mybucket --recursive
However, I get the following error:
Unknown options: --recursive
The following is the version information for my Ubuntu Linux EC2 instance:
$aws s3 ls --version
aws-cli/1.2.9 Python/3.4.3 Linux/3.13.0-85-generic
How can I enable the --recursive option on my aws-cli?

'aws s3 ls --recursive' was added in version 1.2.11 - you are using version 1.2.9 - an outdated version. Please upgrade to the latest version.
pip install -U awscli

If you have installed aws-cli using command apt-get install awscli on ubuntu, it installs the older version of aws cli.
You can install the latest aws-cli using pip command, make sure pip is installed on your system. Install aws-cli using this command
pip install -U awscli
To install pip you can use follow commands
sudo apt-get install python-pip
sudo apt-get install python3-pip -> On python3

Related

Git Large File Storage / How to install git lfs on AWS EC2 Linux 2 / "No package git-lfs available."

How to install git-lfs on an Amazon EC2 (Amazon Linux 2) instance?
Based on https://github.com/git-lfs/git-lfs/blob/main/INSTALLING.md did I tried:
sudo yum install git -y;
cd /home/ec2-user;
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
-> which tells me afterwards:
The repository is setup! You can now install packages.
sudo yum install git-lfs
-> gives me:
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
No package git-lfs available.
Error: Nothing to do
Future commands (not possible yet duo git: 'lfs' is not a git command. See 'git --help'. after git lfs install) would than be based on https://git-lfs.github.com/
git lfs install
...
git clone https://yourrepo.git;
Amazon Linux 2 AMI (HVM) - Kernel 5.10, SSD Volume Type / Arm
With Amazon Linux 2, you can use the Extras Library to install application and software updates on your instances. These software updates are known as topics.
Below are the steps:
Connect to your EC2 Linux instance using SSH.
Use the which command to confirm that the amazon-linux-extras package is installed:
$ which amazon-linux-extras
/usr/bin/amazon-linux-extras
If the amazon-linux-extras package isn't installed, you can use yum to install it:
$ sudo yum install -y amazon-linux-extras
To list the available topics, you can use the following command:
$ amazon-linux-extras list
git-lfs usually comes as part of the epel-release, so you need to first to install epel release for Amazon Linux:
$ sudo amazon-linux-extras install epel -y
For the Amazon Linux AMI, access to the Extra Packages for Enterprise Linux (EPEL) repository is configured, but it is not enabled by default.
sudo yum-config-manager --enable epel
Now you can install git-lfs or whatever else you like from epel-release:
$ sudo yum install git-lfs
As described on packagecloud[1], you need to run:
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
to install the dependencies and the repos. Alternatively you can take the script from [1] directly. You can then simply install with yum:
sudo yum install git-lfs
[1] https://packagecloud.io/github/git-lfs/install#bash-rpm
You need to directly install the ARM64 git-lfs binaries:
wget https://github.com/git-lfs/git-lfs/releases/download/v3.1.2/git-lfs-linux-arm64-v3.1.2.tar.gz
sudo yum install git -y
tar -xvf git-lfs-linux-arm64-v3.1.2.tar.gz
sudo ./install.sh
git lfs install
Now, enjoy your GIT-LFS on ARM64.

How to properly install azure-cli in Ubuntu 20.04 using homebrew?

I am trying to install the azure-cli locally from a cloned repository. I successfully installed homebrew on my end. However, when I try to install the azure-cli, the error occurs below at the very end:
From the repository, the instructions work on other devices, both on windows and linux. I don't know why it won't work on my end:
Step 1: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Step 2: brew tap-new <name>/azure-cli
Step 3: brew extract azure-cli <name>/azure-cli --version 2.29.2
Step 4: brew install <name>/azure-cli/azure-cli#2.29.2 (This is where error occurs)
*Note: the section is the name of the user of my laptop.
Other part of the installation process I've successfully installed:
Decorator setup:
sudo apt install python3-pip
sudo /usr/bin/pip3 install --system decorator
Azure Functions Core Tools:
npm i -g azure-functions-core-tools#3 --unsafe-perm true
We have ran the below cmdlets in our local environment to install Home brew & azure cli on azure Linux virtual machine which is running with Ubuntu 20.04 image.
Here are the list of Linux cmdlets used:
sudo apt update
sudo apt-get install build-essential
sudo apt install git -y ##installing git
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" ##Run Homebrew installation Script
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" ##Add Homebrew to your path
To ensure everything is working correctly to use brew, we can run its command
brew doctor
It may give the warning to install GCC and to remove that simply install it using brew
brew install gcc
To install azure-cli with last version as per the documentation
brew install azure-cli
Here is the output screenshot for reference:
You can refer this blog to install home brew on Ubuntu20.04 linux machine.

Ansible install all required Azure modules

I'm trying a simple thing like creating a Resource group in Azure via an Ansible Playbook, but I need the Azure Python SDK for it. I can't install the SDK with pip install azure because Microsoft doesn't support it anymore, which is completely illogical because Ansible keeps showing ModuleNotFoundError errors every time I execute the playbook, and I have to install them all manually...
Any more efficient way to do this?
I believe you're looking to install Ansible Azure module. So please try the following,
If you already have python and pip installed, use the command pip install ansible[azure], to install Azure modules.
Otherwise Try the following, and see if that helps,
On Windows,
Install Python
Install PIP
Add python and pip installation path to Windows PATH variable
Install Ansible Azure Module: pip install ansible[azure]
On CentOS 7.4,
Install python and pip:
sudo yum check-update; sudo yum install -y gcc libffi-devel python-devel openssl-devel epel-release
sudo yum install -y python-pip python-wheel
Install Ansible Azure Module:
sudo pip install ansible[azure]
On Ubuntu 16.04 LTS,
Install python and pip:
sudo apt-get update && sudo apt-get install -y libssl-dev libffi-dev python-dev python-pip
Install Ansible Azure Module:
sudo pip install ansible[azure]

Install a specific nodejs version with apt-get

I am a newbie with Linux general, and here's what I am trying to achieve:
I am trying to install nodejs version on Debian Linux with the following command:
apt-get install nodejs=8.14.0
But I get this error in return:
E: Version '8.14.0' for 'nodejs' was not found
As far as I found, this is the correct way to specify a version. If I do this, then it works fine:
apt-get install nodejs
But I need this specific version, and not the latest one. I am doing this for a Docker image, so it has to be installed at runtime.
Make sure you have the following packages:-
sudo apt-get install \
apt-transport-https \
curl \
software-properties-common
Enable the NodeSource repository by using a command:-
sudo curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
After enabling the repository, install Node.js using a command:-
sudo apt-get install nodejs
If you're doing this for a Docker image, why not just use the Node Docker image with the version you need?
You can try installing your node using a package manager like nvm:
Installing Node.js to linux
Or download the binaries directly from here:
Node.js v8.14.0

How to upgrade AWS CLI to the latest version?

I recently noticed that I am running an old version of AWS CLI that is lacking some functionality I need:
$aws --version
aws-cli/1.2.9 Python/3.4.3 Linux/3.13.0-85-generic
How can I upgrade to the latest version of the AWS CLI (1.10.24)?
Edit:
Running the following command fails to update AWS CLI:
$ pip install --upgrade awscli
Requirement already up-to-date: awscli in /usr/local/lib/python2.7/dist-packages
Cleaning up...
Checking the version:
$ aws --version
aws-cli/1.2.9 Python/3.4.3 Linux/3.13.0-85-generic
From http://docs.aws.amazon.com/cli/latest/userguide/installing.html#install-with-pip
To upgrade an existing AWS CLI installation, use the --upgrade option:
pip install --upgrade awscli
On Linux and MacOS X, here are the three commands that correspond to each step:
$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
$ unzip awscli-bundle.zip
$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
This does not work:
pip install --upgrade awscli
This worked fine on Ubuntu 14.04( no need to reboot also .. You would have to first install pip3 ):
pip3 install --upgrade awscli
For Ubuntu 16.04 I used parts of the other answers and comments and just reloaded bash instead of rebooting.
I installed the aws-cli using apt so I removed that first:
sudo apt-get remove awscli
Then I could pip install (I chose to use sudo to install globally with pip2):
sudo pip install -U awscli
Since I was doing this on a server I didn't want to reboot it, but reloading bash did the trick:
source ~/.bashrc
At this point I could use the new version of aws cli
aws --version
Update: Upgrade instance using AWS CLI v1 to AWS CLI v2:
This question and answer was initially created when there was only an AWS CLI v1. There is now a AWS CLI v2. The installation instructions for the AWS CLI v2 can be found here.
The new AWS CLI v2 has different installation instructions based on whether your EC2 instance is using Linux x86 (64-bit) or Linux ARM architecture.
To upgrade to AWS CLI v2, on an EC2 instance using Linux ARM, I had to issue the following commands:
rm -rf /bin/aws
curl "https://awscli.amazonaws.com/awscli-exe-linux-aarch64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install -i /usr/local/aws -b /bin
Subsequently test your AWS CLI version by executing: aws --version
For the Linux x86 (64-bit) architecture I'm hoping the commands are the same except for replacing the curl command with the following: (as per the installation instructions)
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o
"awscliv2.zip"
The AMI I used was the most recent one currently available and it was still using the AWS CLI v1. In the future if AWS starts packaging AWS CLI v2 with their AMIs this answer might require an update.
Original answer: Upgrade instance using AWS CLI v1 to use the most recent version of AWS CLI v1:
If you are having trouble installing the AWS CLI using pip you can use the "Bundled Installer" as documented here.
The steps discussed there are as follows:
$ curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
$ unzip awscli-bundle.zip
$ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
Check your AWS CLI version subsequently as a sanity-check that everything executed correctly:
$ aws --version
If the AWS CLI didn't update to the latest version as expected maybe the AWS CLI binaries are located somewhere else as the previously-given commands assume.
Determine where AWS CLI is being executed from:
$ which aws
In my case, AWS CLI was being executed from /bin/aws, so I had to install the
"Bundled Installer" using that location as follows:
$ sudo ./awscli-bundle/install -i /user/local/aws -b /bin/aws
Try
sudo pip install --upgrade awscli, and open a new shell.
This worked well for me (no need to reboot).
Simple use
sudo pip install awscli --force-reinstall --upgrade
This will upgrade all the required modules.
On Mac you can use homebrew:
to install: brew install awscli
to upgrade: brew upgrade awscli
Make sure you don't have multiple installations: where aws
pip install awscli --upgrade --user
The --upgrade option tells pip to upgrade any requirements that are already installed. The --user option tells pip to install the program to a subdirectory of your user directory to avoid modifying libraries used by your operating system.
We can follow the below commands to install AWS CLI on UBUNTU:
sudo apt install curl
curl “https://s3.amazonaws.com/aws-cli/awscli-bundle.zip” -o
“awscli-bundle.zip”
unzip awscli-bundle.zip
sudo ./awscli-bundle/install -i /usr/local/ aws -b /usr/local/bin/aws
rm -rf awscli-bundle.zip awscli-bundle
To test: aws — version
For More Info :
https://gurudathbn.wordpress.com/2018/03/31/installing-aws-cli-on-ubuntu/
When using sudo pip install --upgrade awscli I got the following error:
ERROR: Could not install packages due to an EnvironmentError: [Errno 30] Read-only file system: '/lib'
By using sudo with -H option, I could fix the problem.
sudo -H pip install --upgrade awscli
Currently, using pip will get you the old version of awscli, 1.18.103.
The latest version of aws-cli, 2.0.33 is on the v2 branch. You can download the installer for Linux, Windows and macOS from here.
I was trying to install awscli on one of my ec2 instances where I tried both
sudo pip install --upgrade awscli,
sudo pip3 install --upgrade awscli
which didn't worked, as I was getting errors like
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-2nh71cs2/cryptography/
And rebooting servers were not an option.
Luckily, simple
sudo apt update
sudo apt install awscli worked.
I do it by removing & installing the awscli like described in this video
basically:
pip uninstall awscliv2
pip install awscliv2
pip install awscliv2==your-version
pip install awscliv
(you can keep v1 along with v2 if you want)
pip install --upgrade ...
works as well. sure.
I do not install it globally (like some ppl seems still do), btw. Because sometimes I need different v for different cases. so I keep it in separate python virtual environment.
For windows you can try this command
msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi
Install or update the AWS CLI on macOS
curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg ./AWSCLIV2.pkg -target /
Done!
You can verify the same with below cmd
aws --version
Try AWS Cloud Shell, Quick and easy
AWS CloudShell is a browser-based shell that makes it easy to securely manage, explore, and interact with your AWS resources. CloudShell is pre-authenticated with your console credentials.
Benefits
No extra credentials to manage
Always up to date
No cost
More Details here https://aws.amazon.com/cloudshell/
Customizable
To install globally, get on the sudo access
sudo su & then upgrade aws cli by
pip3 install --upgrade awscli

Resources