I'm trying to execute an ansible playbook that has a task utilizing the unarchive module. Because I'm doing this on OSX, I need it to use gnu-tar, instead of the bsd tar that typically comes with OSX, since BSD tar is not officially supported.
I've installed gnu-tar using brew, and placed this package's gnubin folder ahead of other paths in the $PATH variable. However, my playbook still looks for tar in the /usr/bin folder (the location of the original tar), instead of the newly installed gnu-tar.
My question is, how does the unarchive module look for the tar binary, and what would be the best way for me to override this behavior so that it utilizes gnu-tar, instead?
I've found the solution.
Before we proceed, this is my code for making sure that the brew-installed gnu-tar takes precedence over the bsd-tar that comes with OSX:
# gnu-tar
export PATH="/usr/local/opt/gnu-tar/libexec/gnubin:$PATH"
export MANPATH="/usr/local/opt/gnu-tar/libexec/gnuman:$MANPATH"
This code was originally located in my ~/.bash_profile file.
The way that ansible looks up the tar binary is as you'd expect: the $PATH variable, like any other process. However, running a play creates an interactive non-login shell. As explained here, the ~/.bashrc file gets loaded for interactive non-login shells, while ~/.bash_profile gets loaded for login shells.
Since my code was located in the ~/.bash_profile file, it never executes whenever I run a play. Again, this is because running a play creates an interactive non-login shell that does not load ~/.bash_profile. Hence, the code should actually be in ~/.bashrc.
True enough, the moment I moved my code to the ~/.bashrc was when my task started using the installed gnu-tar. I was able to confirm this by creating a task to register which tar into a variable and printing it in another debug task.
From the source code it looks like tar is looking for a gtar binary and if not a tar
# Prefer gtar (GNU tar) as it supports the compression options -zjJ
self.cmd_path = self.module.get_bin_path('gtar', None)
if not self.cmd_path:
# Fallback to tar
self.cmd_path = self.module.get_bin_path('tar')
Thus, aliasing your brew installed gnu tar as "gtar" or "tar" should fix the problem
Related
I am using the "Bash on Ubuntu on Windows" (Linux Subsystem) and want to add Terraform to my $PATH. Since Terraform can't be installed via apt-get, I did the following steps:
Navigated to this directory, where I wanted to install Terraform:
cd /usr/local
In the above path, I used wget to download Terraform:
wget
https://releases.hashicorp.com/terraform/0.9.8/terraform_0.9.8_linux_amd64.zip
Terraform successfully unzips! When I open the file in VIM it is all good:
unzip terraform_0.9.8_linux_amd64.zip
I then enter this command to check to see if the Terraform binary is accessible from the command line:
terraform -version
However the following message gets returned:
terraform: command not found
This tells me that the Terraform downloaded location needs to be added to my $PATH.
Already being logged in as the root user ("sudo su") I enter the following command to access ".profile":
vim ~/.profile
The following is already in this file, which I leave untouched:
# ~/.profile: executed by Bourne-compatible login shells.
if [ "$BASH" ]; then
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
fi
mesg n
Immediately below this text, I add the following, and successfully save the file using :wq!:
export PATH=/usr/local/bin:$PATH
export PATH=$PATH:/usr/local/terraform
6.
I then again enter the following command to check to see if terraform is detected
terraform -version
Still the same "terraform: command not found" message is returned. I even tried closing out and starting a new command line session and even restarting my computer. Still no change.
Anyone have any ideas on how to resolve this? Again, note that I am using "Bash on Ubuntu on Windows" (Linux Subsystem). Any input would be appreciated!
The direct answer to your problem is at the end.
But I think it will make more sense if you keep reading from here.
Before trying to add to PATH,
I recommend to test a program first.
In your case I would do like this:
wget https://releases.hashicorp.com/terraform/0.9.8/terraform_0.9.8_linux_amd64.zip
unzip terraform_0.9.8_linux_amd64.zip
./terraform
Notice the last line ./terraform.
The zip file contains a single file, terraform,
which now should be in the current directory,
so I can run it with ./terraform.
If it's executable.
If it's not executable then confirm it:
ls -l terraform
And make it executable if needed:
chmod +x terraform
Now let's add it to PATH.
But first,
let's decide where to put this executable.
/usr/local/bin seems a reasonable location.
So let's move the terraform executable into that directory.
Usually /usr/local/bin is already on PATH,
so you might not need to change anything.
Now you can try your check, and there's a good chance it already works:
terraform -version
If it doesn't, then /usr/local/bin is not on the PATH.
To add it, add this line in ~/.profile:
export PATH=$PATH:/usr/local/bin
Two things looked fundamentally wrong with your approach:
Adding /usr/local/terraform to PATH. This is fishy, because the entries on PATH must be directories, and in your post nothing indicates that you created a directory at /usr/local/terraform.
You cd into /usr/local, and then unzip the zip file of terraform. The linked zip contains a single file named terraform, so /usr/local/terraform in your example should be a file.
If it is a file, then you could make it executable as terraform by adding to add to PATH its base directory. But adding /usr/local to PATH would not be a good idea. It's conventional to put binaries into /usr/local/bin, not directly into /usr/local
You did not mention how you reloaded ~/.profile. After editing this file, the new commands you added do not get automatically executed in your current shell. They will get executed when you open a new shell. Or you could manually execute the added commands in the current shell.
Hit below command
export PATH=$PATH:/usr/local/bin
I've installed Anaconda and have added the environment variable export PATH="~/anaconda/bin:$PATH" to my .zshrc but still can't run any 'conda' commands in my terminal. I have followed these commands(source) to install anaconda.
# Go to home directory
cd ~
# You can change what anaconda version you want at
# https://repo.continuum.io/archive/
wget https://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh
zsh Anaconda3-4.2.0-Linux-x86_64.sh -b -p ~/anaconda
rm Anaconda3-4.2.0-Linux-x86_64.sh
echo 'export PATH="~/anaconda/bin:$PATH"' >> ~/.zshrc
# Refresh basically
source ~/.zshrc
Am I missing anything?
This kind of issue often arise due to incorrect paths to your Python distribution. Basically, your computer does not know where to look for the correct file or program and you need to tell it where to look using your system's $PATH variable. You will thus need to manually set the paths by editing your .profile or .rc file. The most commonly used shell is bash. Both Anaconda and Canopy assume you're running the Bash shell and will put the path to your install there.
The steps below worked for me on Mac, it should work on Linux as well:
1. Go to your home directory by typing cd ~ from the prompt, or by open that same directory in Finder.
2. Now get a list of all the files in this folder, including the hidden ones:
ls -la
or, directly in Finder (without the "+"):
CMD + SHIFT + .
Now, you probably have (a) both the .profile and .bash_profile, or (b) just one of them, or (c) none of them:
3.a If you are using Bash, you're looking for files called .profile and .bash_profile. Which you have will depend on your system configuration. If you have both, this is probably the root cause of your problem; .bash_profile overrides .profile if both are present. In this case, open .profile and copy everything you find inside across to .bash_profile.
3.b If you only have one of .bash_profile or .profile, open it in an editor and have a look. You're looking for any obvious references to Python or your distribution (Anaconda, Canopy) on any of the uncommented-out lines. You'll probably see several lines that look something like:
PATH="Users/<your user name>/Library/...:${PATH}"
export PATH
In the rare case that you don't see any reference to Anaconda at all, you will need to add the lines yourself. Add, for example, the following to the file:
export PATH=~/anaconda/bin:$PATH
3.c If it turns out that you don't have any of the files, you would have to create one by yourself. Do this by entering touch .bash_profile in the prompt, or, simply right click and create a new file directly in Finder. Add the following to the file (same as in (b)):
export PATH=~/anaconda/bin:$PATH
4. Save the file. Quit and restart any program that use Python (including the Terminal window), and you should be good to go.
More detailed info here (including for Windows):
https://github.com/landlab/landlab/wiki/Correcting-Install-Paths
I want to add a small script to the linux PATH so I don't have to actually run it where it's physically placed on disk.
The script is quite simple is about giving apt-get access through a proxy I made it like this:
#!/bin/bash
array=( $# )
len=${#array[#]}
_args=${array[#]:1:$len}
sudo http_proxy="http://user:password#server:port" apt-get $_args
Then I saved this as apt-proxy.sh, set it to +x (chmod) and everything is working fine when I am in the directory where this file is placed.
My question is : how to add this apt-proxy to PATH so I can actually call it as if it where the real apt-get ? [from anywhere]
Looking for command line only solutions, if you know how to do by GUI its nice, but not what I am looking for.
Try this:
Save the script as apt-proxy (without the .sh extension) in some directory, like ~/bin.
Add ~/bin to your PATH, typing export PATH=$PATH:~/bin
If you need it permanently, add that last line in your ~/.bashrc. If you're using zsh, then add it to ~/.zshrc instead.
Then you can just run apt-proxy with your arguments and it will run anywhere.
Note that if you export the PATH variable in a specific window it won't update in other bash instances.
You want to define that directory to the path variable, not the actual binary e.g.
PATH=$MYDIR:$PATH
where MYDIR is defined as the directory containing your binary e.g.
PATH=/Users/username/bin:$PATH
You should put this in your startup script e.g. .bashrc such that it runs each time a shell process is invoked.
Note that order is important, and the PATH is evaluated such that if a script matching your name is found in an earlier entry in the path variable, then that's the one you'll execute. So you could name your script as apt-get and put it earlier in the path. I wouldn't do that since it's confusing. You may want to investigate shell aliases instead.
I note also that you say it works fine from your current directory. If by that you mean you have the current directory in your path (.) then that's a potential security risk. Someone could put some trojan variant of a common utility (e.g. ls) in a directory, then get you to cd to that directory and run it inadvertently.
As a final step, after following the solution form proposed by #jlhonora (https://stackoverflow.com/a/20054809/6311511), change the permissions of the files in the folder "~/bin". You can use this:
chmod -R 755 ~/bin
make an alias to the executable into the ~/.bash_profile file and then use it from anywhere or you can source the directory containing the executables you need run from anywhere and that will do the trick for you.
adding to #jlhonora
your changes in ~./bashrc or ~./zshrc won't reflect until you do
source ~./zshrc or source ./bashrc , or restart your pc
I'm pretty new to programming for Linux environments, so I don't exactly know what to search for in order to answer this question for myself. I need to understand how applications set the shell to accept a certain command to start them. For example, you can start Firefox from the command line by executing the command: firefox.
I don't know where this is defined. Makefile? Configure script? In the source code itself?
Any resources / reading on Linux programming tidbits like these would be greatly appreciated!
Thank you.
Firefox is launched by the command "firefox" because there is an executable file in one of the folders in the $PATH environment variable called "firefox".
Are you talking about the PATH variable? It seems like you are.
In linux, you should be able to type: "echo $PATH" (without quotes) and get a ":"-separated list of locations where programs are located (like firefox).
If you need to add something to your path, you should be able to do:
export PATH=$PATH:/another/directory
In your shell (which is most likely bash)
You can also type:
which firefox
To display the location of the firefox executable.
Typically the shell is going to have an environment variable called $PATH set. This is just an ordered list of all the directories to look when somebody types in a command. As soon as it finds an executable file (by which I mean a file for which you have execute permissions, not a file ending in .exe) with the same name as whatever was typed, it will run that file. Common directories in $PATH might be /bin, /usr/local/bin, ~/bin, etc.
So, when you type 'firefox', the shell looks through all the directories in $PATH until it finds /usr/local/bin/firefox, which it then runs. To make your own programs run the same way, you'll either need to put them (or a symbolic link to them) in a directory that is likely to be in every user's path (/usr/local/bin/ is a good choice), or you'll need to get your users to add your program's directory to their $PATH.
For a more complete description, see the Wikipedia article about the $PATH variable.
As an alternative to the modification of $PATH mentioned earlier, you could also copy or link your executable in one of the directories already in your $PATH. more specifically, /usr/local/bin/ is available on most UNIX system for pretty much this purpose (installing software outside the default package management of the operating system).
It has to be in the path as everyone else mentioned, but you might also need to make it executable with something like this:
chmod +x /path/to/file
And if it's a script there's usually a shebang at the top that tells the os what to use to execute it:
#! /usr/bin/python
Often, large packages are installed in /opt with a wrapper script or link somewhere in the PATH. For example, on my system, Google Picasa is installed in /opt/google/picasa and there is a symlink at /usr/bin/picasa to /opt/google/picasa/3.0/picasa
Firefox is at /usr/bin/firefox on my system and that's a symlink to /usr/bin/firefox-3.0 which is itself a symlink to /usr/lib/firefox-3.0.11/firefox.sh - That shell file fumbles around until it finally runs /usr/lib/firefox-3.0.11/firefox (unless it finds a reason to do something else). That, finally, is a binary executable. So /usr/lib is where firefox is installed, for me.
You can use this command to find out where a program is:
type -a firefox
Then, you can find out what kind of file it is using this:
file /usr/bin/firefox
Also see the Filesystem Heirarchy Standard for more information about recommended locations for files and programs.
Some software (for ex. the NetBeans IDE) ship the Linux installers in .sh files. Curious about how exactly they 'package' a whole IDE into a 'shell script', I opened the file in an editor. I saw some plain text shell scripting code and then some random gibberish, which I reckon is 'binary' or non-plain text.
I am wondering how they mix plain shell scripts and then probably call the 'non-readable' stuff, which would be the binaries.
Any insight on this?
Basically, it's a shell script prepended to a compressed archive of some sort, such as a tar archive. You use the tail or sed command on yourself (the $0 variable in Bourne shell) to strip off the shell script at the front and pass the rest to your unarchiver.
For example, create the following script as self-extracting:
#!/bin/sh -e
sed -e '1,/^exit$/d' "$0" | tar xzf - && ./project/Setup
exit
The sed command above deletes all lines from the first line of the file to the first one that starts with "exit", and then passes the rest on through. If what starts immediately after the "exit" line is a tar file, the tar command will extract it. If that's successful, the ./project/Setup file (presumably extracted from the tarball) will be executed.
Then:
mkdir project
echo "#!/bin/sh" > project/Setup
echo "echo This is the setup script!" >> project/Setup
chmod +x project/Setup
tar czf - project >> self-extracting
Now, if you get rid of your old project directory, you can run self-extracting and it will extract that tar file and run the setup script.
You might want to check out makeself.sh
From the authors' notes.
makeself.sh is a small shell script that generates a self-extractable tar.gz archive from a directory. The resulting file appears as a shell script (many of those have a .run suffix), and can be launched as is. The archive will then uncompress itself to a temporary directory and an optional arbitrary command will be executed (for example an installation script).
Makeself archives also include checksums for integrity self-validation (CRC and/or MD5 checksums).
The makeself.sh script itself is used only to create the archives from a directory of files. The resultant archive is actually a compressed (using gzip, bzip2, or compress) TAR archive, with a small shell script stub at the beginning. This small stub performs all the steps of extracting the files, running the embedded command, and removing the temporary files when it's all over. All what the user has to do to install the software contained in such an archive is to "run" the archive [that is execute the script]
I am trying to keep the code of this script as portable as possible, i.e it's not relying on any bash-specific features and only calls commands that are installed on any functioning UNIX-compatible system. This script as well as the archives it generates should run on any Unix flavor, with any compatible Bourne shell, provided of course that the compression programs are available.
Finally, the makeself package itself comes as a self-extracting script called makeself.run.
Add a Binary Payload to your Shell Scripts
GNU sharutils:
http://www.gnu.org/software/sharutils/
is a toolset for creating shell archives, and provides some additional features that may be helpful (such as checksums to ensuring that the payload is not damaged in transit).
Protecting against malicious modifications is not really feasible when the final product has to be interpretable by the shell - anyone who understood the generation technique could modify the checksum as well.
There are also other/commercial software installer builder (like InstallAnywhere) they basically have their own version of shar/makeself.
Netbeans has their own installer engine, and part of it, which does the unpacking and launching is done in the NBI native launcher component: http://wiki.netbeans.org/NBINativeLaunchers
Creates a shell(script) archive for Linux/Unix/MacOS and native executable for Windows. You can use that tool for your own projects, also.