I have a utility function that I used a lot (assertReturnStatus()). I'd like to define it in a utility file (utility.sh) and reuse it in other bash scripts.
How can reuse the function from another bash script file? Thanks.
You need to "import" the first file in the second.
Be warned that this will litterally include the first, so any code in the first will be executed as if it were litterally in the place of the line.
The syntax is:
# if /path/to/file exists, then include it
[ -f /path/to/file ] && . /path/to/file
Note bash also support the keyword source (ie: source /path/to/file) but it is not POSIX compliant and might not work in other shell like ash, dash, posh.
If you don't want to do an explicit sourcing of your script file as bufh suggests: I put my often used functions in my .bashrc which gets always sourced thus having the functions always available.
As bufh pointed out in a comment always is not really always but limited to interactive shells. So, if you're planning to use the scripts from an interactive session, you could put it into the .bashrc, otherwise go for explicit sourcing.
Related
I am trying to run a Bash command from within my Perl program.
However Perl seems to be confusing my Bash $PWD environment variable as a Perl variable.
How can I make it just read it all as a string?
This is what I'm trying to run
my $path = /first/path;
`ln -s $path $PWD/second/path`
Those backticks runs the second line in Bash. Using System() produces the same problem.
Any ideas?
There are two queries here, on use of Bash variables and on running external commands.
There is the %ENV hash in Perl, with environment variables
perl -wE'say $ENV{PWD}'
However, you are often better off getting the equivalent within the script, as things may have a subtly different meaning for the script or change as the script runs.
More importantly, using shell commands exposes you to all kinds of potential problems with quoting, shell injection, and interpretation. For instance, the command you show is dangerous, as outlined in Charles Duffy comment. It is in principle better to use Perl's rich functionality. See for example
Executing system commands safely while coding in Perl
Using system commands in Perl instead of built in
libraries/functions [duplicate]
for a sober, and detailed, account of advantages.
In case you do need to run external commands, it is best to avoid the shell altogether, for example by using the multi-argument form of system. If you need the output of the command as well there are various modules in Perl that provide that. See links below.
If you also need to use the shell's capabilities, instead of quoting everything just right in order for the shell to receive what it needs better use a ready tool like String::ShellQuote.
Some examples:
How to use both pipes and prevent shell expansion in perl system function?
Perl is respecting '<' as a regular character rather an output redirection
How to pipe the content of a variable as STDIN in a qx{} statement in Perl?
Perl system command with multiple parameters output to file.
Note that qx operator (backticks) uses /bin/sh, which may or may not get relegated to Bash. So if you want Bash you need system('/bin/bash', '-c', $cmd), where $cmd need be built carefully to avoid problems. See the links with examples.
Here is a full example related to the objective behind the question.
Your program's working directory may be other than expected depending on how it's started. For one, it changes after chdir. I don't know your exact intent with PWD, but in Perl there are core Cwd::cwd and FindBin with $RealBin, for the current working directory and for the directory where the script resides (generally different things).
To create a symbolic link to $path, with the relative path following the current working directory
use warnings;
use strict;
use Cwd qw(cwd);
my $cwd = cwd;
my $path = '/first/path';
symlink($path, "$cwd/second/path") or die "Can't make a symlink: $!";
If the path is meant to be the script's location use $RealBin from FindBin instead of cwd.
Note that with symlink you cannot pass a directory instead of a link name. See this page.
I have made Bash scripts before and they all ran fine without #!/bin/bash at the beginning.
What's the point of putting it in? Would things be any different?
Also, how do you pronounce #? I know that ! is pronounced as "bang."
How is #! pronounced?
It's a convention so the *nix shell knows what kind of interpreter to run.
For example, older flavors of ATT defaulted to sh (the Bourne shell), while older versions of BSD defaulted to csh (the C shell).
Even today (where most systems run bash, the "Bourne Again Shell"), scripts can be in bash, python, perl, ruby, PHP, etc, etc. For example, you might see #!/bin/perl or #!/bin/perl5.
PS:
The exclamation mark (!) is affectionately called "bang". The shell comment symbol (#) is sometimes called "hash".
PPS:
Remember - under *nix, associating a suffix with a file type is merely a convention, not a "rule". An executable can be a binary program, any one of a million script types and other things as well. Hence the need for #!/bin/bash.
To be more precise the shebang #!, when it is the first two bytes of an executable (x mode) file, is interpreted by the execve(2) system call (which execute programs). But POSIX specification for execve don't mention the shebang.
It must be followed by a file path of an interpreter executable (which BTW could even be relative, but most often is absolute).
A nice trick (or perhaps not so nice one) to find an interpreter (e.g. python) in the user's $PATH is to use the env program (always at /usr/bin/env on all Linux) like e.g.
#!/usr/bin/env python
Any ELF executable can be an interpreter. You could even use #!/bin/cat or #!/bin/true if you wanted to! (but that would be often useless)
It's called a shebang. In unix-speak, # is called sharp (like in music) or hash (like hashtags on twitter), and ! is called bang. (You can actually reference your previous shell command with !!, called bang-bang). So when put together, you get haSH-BANG, or shebang.
The part after the #! tells Unix what program to use to run it. If it isn't specified, it will try with bash (or sh, or zsh, or whatever your $SHELL variable is) but if it's there it will use that program. Plus, # is a comment in most languages, so the line gets ignored in the subsequent execution.
Every distribution has a default shell. Bash is the default on the majority of the systems. If you happen to work on a system that has a different default shell, then the scripts might not work as intended if they are written specific for Bash.
Bash has evolved over the years taking code from ksh and sh.
Adding #!/bin/bash as the first line of your script, tells the OS to invoke the specified shell to execute the commands that follow in the script.
#! is often referred to as a "hash-bang", "she-bang" or "sha-bang".
The shebang is a directive to the loader to use the program which is specified after the #! as the interpreter for the file in question when you try to execute it. So, if you try to run a file called foo.sh which has #!/bin/bash at the top, the actual command that runs is /bin/bash foo.sh. This is a flexible way of using different interpreters for different programs. This is something implemented at the system level and the user level API is the shebang convention.
It's also worth knowing that the shebang is a magic number - a human readable one that identifies the file as a script for the given interpreter.
Your point about it "working" even without the shebang is only because the program in question is a shell script written for the same shell as the one you are using. For example, you could very well write a javascript file and then put a #! /usr/bin/js (or something similar) to have a javascript "Shell script".
The operating system takes default shell to run your shell script. so mentioning shell path at the beginning of script, you are asking the OS to use that particular shell. It is also useful for portability.
It is called a shebang. It consists of a number sign and an exclamation point character (#!), followed by the full path to the interpreter such as /bin/bash. All scripts under UNIX and Linux execute using the interpreter specified on a first line.
Bash standards for “Bourne-Again shell” is just one type of many available
shells in Linux.
A shell is a command line interpreter that accepts and runs commands.
Bash is often the default shell in most Linux distributions. This is why bash is
synonymous to shell.
The shell scripts often have almost the same syntaxes, but they also differ sometimes. For example, array index starts at 1 in Zsh instead of 0 in bash. A script
written for Zsh shell won’t work the same in bash if it has arrays.
To avoid unpleasant surprises, you should tell the interpreter that your shell script
is written for bash shell. How do you do that?
simply begin your bash script into #!/bin/bash
Also you will see some other parameters after #!/bin/bash,
for example
#!/bin/bash -v -x
read this to get more idea.
https://unix.stackexchange.com/questions/124272/what-do-the-arguments-v-and-x-mean-to-bash .
It can be useful to someone that uses a different system that does not have that library readily available. If that is not declared and you have some functions in your script that are not supported by that system, you should declare #/bin/bash. I've ran into this problem before at work and now I just include it as a practice.
I am trying to create a script that will run a program on each file in a list. I have been trying to do this using a .csh file (I have no clue if this is the best way), and I started with something as simple as hello world
echo "hello world"
The problem is that I cannot execute this script, or verify that it works correctly. (I was trying to do ./testscript.csh which is obviously wrong). I haven't been able to find anything that really explains how to run C Scripts, and I'm guessing there's a better way to do this too. What do I need to change to get this to work?
You need to mark it as executable; Unix doesn't execute things arbitrarily based on extension.
chmod +x testscript.csh
Also, I strongly recommend using sh or bash instead of csh, or you will soon learn about the idiosyncrasies of csh's looping and control flow constructs (some things only work inside them if done a particular way, in particular with the single-line versions things are very limited).
You can use ./testscript.csh. You will however need to make it executable first:
chmod u+x testscript.csh
Which means set testscript to have execute permissions for the user (who ever the file is owned by - which in this case should be yourself!)
Also to tell the OS that this is a csh script you will need put
#! /path/to/csh
on the first line (where /path/to/csh is the full path to csh on your system. You can find that out by issuing the command which csh).
That should give you the behvaiour you want.
EDIT As discussed in some of the comments, you may want to choose an alternative shell to C Shell (csh). It is not the friendliest one for scripting.
You have several options.
You can run the script from within your current shell. If you're running csh or tcsh, the syntax is source testscript.csh. If you're running sh, bash, ksh, etc., the syntax is . ./testscript.sh. Note that I've changed the file name suffix; source or . runs the commands in the named file in your current shell. If you have any shell-specific syntax, this won't work unless your interactive shell matches the one used by the script. If the script is very simple (just a sequence of simple commands), that might not matter.
You can make the script an executable program. (I'm going to repeat some of what others have already written.) Add a "shebang" as the first line. For a csh script, use #!/bin/csh -f. The -f avoids running commands in your own personal startup scripts (.cshrc et al), which saves time and makes it more likely that others will be able to use it. Or, for a sh script (recommended), used #!/bin/sh (no -f, it has a completely different meaning). In either case, run chmod +x the_script, then ./the_script.
There's a trick I often use when I want to perform some moderately complex action. Say I want to delete some, but not all, files in the current directory, but the criterion can't be expressed conveniently in a single command. I might run ls > tmp.sh, then edit tmp.h with my favorite editor (mine happens to be vim). Then I go through the list of files and delete all the ones that I want to leave alone. Once I've done that, I can replace each file name with a command to remove it; in vim, :%s/.*/rm -f &/. I add a #!/bin/sh at the top save it, chmod +x foo.sh, then ./foo.sh. (If some of the file names might have special characters, I can use :%s/.*/rm -f '&'/.)
I was wondering if there is a way to get Linux commands with a perl script. I am talking about commands such as cd ls ll clear cp
You can execute system commands in a variety of ways, some better than others.
Using system();, which prints the output of the command, but does not return the output to the Perl script.
Using backticks (``), which don't print anything, but return the output to the Perl script. An alternative to using actual backticks is to use the qx(); function, which is easier to read and accomplishes the same thing.
Using exec();, which does the same thing as system();, but does not return to the Perl script at all, unless the command doesn't exist or fails.
Using open();, which allows you to either pipe input from your script to the command, or read the output of the command into your script.
It's important to mention that the system commands that you listed, like cp and ls are much better done using built-in functions in Perl itself. Any system call is a slow process, so use native functions when the desired result is something simple, like copying a file.
Some examples:
# Prints the output. Don't do this.
system("ls");
# Saves the output to a variable. Don't do this.
$lsResults = `ls`;
# Something like this is more useful.
system("imgcvt", "-f", "sgi", "-t", "tiff", "Image.sgi", "NewImage.tiff");
This page explains in a bit more detail the different ways that you can make system calls.
You can, as voithos says, using either system() or backticks. However, take into account that this is not recommended, and that, for instance, cd won't work (won't actually change the directory). Note that those commands are executed in a new shell, and won't affect the running perl script.
I would not rely on those commands and try to implement your script in Perl (if you're decided to use Perl, anyway). In fact, Perl was designed at first to be a powerful substitute for sh and other UNIX shells for sysadmins.
you can surround the command in back ticks
`command`
The problem is perl is trying to execute the bash builtin (i.e. source, ...) as if they were real files, but perl can't find them as they don't exist. The answer is to tell perl what to execute explicitly. In the case of bash builtins like source, do the following and it works just fine.
my $XYZZY=`bash -c "source SOME-FILE; DO_SOMETHING_ELSE; ..."`;
of for the case of cd do something like the following.
my $LOCATION=`bash -c "cd /etc/init.d; pwd"`;
I have a script whose content simply exports a variable in linux.
export LD_LIBRARY_PATH=....
I want to run this script in my Perl script so whoever is running my Perl script will have their LD_LIBRARY_PATH set. Can i just do this in the beginning of my Perl script:
#!/usr/bin/perl -w
system(". /myfolder1/myfolder2/myScript.sh");
#!/bin/sh
. /myfolder1/myfolder2/myScript.sh
exec perl -wxS "$0" "$#"
#!/usr/bin/perl -w
# .. the rest of your script as normal
When you run this, it will first be executed by /bin/sh, which is capable of loading myScript.sh into the local environment. sh then execs Perl, which is told to continue from the following line.
This won't work. To change the environment inside your Perl script (and to change the environment that will be passed on to commands run from inside your Perl script), change the %ENV variable.
$ENV{"LD_LIBRARY_PATH"} = ... ;
This won't work. There is no way for a subshell to manipulate the environment of the parent process.
But you could make your script echo the string you want to set as LD_LIBRARY_PATH and then from within your Perl script you could do something like that:
$ENV{LD_LIBRARY_PATH} = `path/to/your/script.sh`;
Of course, a bit of error checking might also be a good idea.
No. Your environment changes made in a child cannot affect the parent. This means running a script will not affect perl. Also perl will not affect the shell from which it was called. You can edit the environment inside perl by changing the special variable %ENV. If there's some kind of unreproducible calculation done in that script, maybe the script should just echo the setting and perl can pick that up on STDOUT and use it.
I {changed directory, modified my environment} in a perl script. How come the change disappeared when I exited the script? How do I get my changes to be visible?
Unix In the strictest sense, it can't be done -- the script executes
as a different process from the shell
it was started from. Changes to a
process are not reflected in its
parent, only in its own children
created after the change.
I had a similar problem a few years ago and whipped up a little module, Env::Sourced, that should do the trick.
use Env::Sourced qw(/myfolder1/myfolder2/myScript.sh);
...
Another option (other than making the changes directly in Perl's %ENV) is to make the changes you want a Perl module, so that you can say:
use MyEnvironment;
and have it modify your environment in all your scripts. It would make it simple to make changes after the fact that will not require editing every script.
The module itself will be simple, something like this:
package MyEnvironment;
$ENV{LD_LIBRARY_PATH} .= ":/some/path/you/want/appended";
# Any other changes you want here.
1;
That won't work. An (unpleasant) alternative might be to replace /usr/bin/perl with a shell script that first executes your script and then executes the perl executable.
This can't be done in the way you're trying to do this.
It either needs a wrapper shell script that sets LD_LIBRARY_PATH and then calls your perl script, or any user executing the script needs to have LD_LIBRARY_PATH set correctly in the first place.
If doing the latter, then this can be managed globally by editing /etc/profile and /etc/cshrc (for ksh, sh, bash, csh and tcsh) shells. You can then test for the value of LD_LIBRARY_PATH in your script and if not set/set incorrectly then print a friendly message to the user. Alternatively individual users can set this in their local .profile/.cshrc files.
Note: you haven't given any information about the environment or useres that might run this, so there's also the possibility that users may set LD_LIBRARY_PATH to something they need. If you do check LD_LIBRARY_PATH for a "good" value in your script, then keep in mind that several paths may have been specified, so you will need to parse this environment variable properly.
If you can find the right place in your perl script, this works as in my example:
$ENV{"LD_LIBRARY_PATH"} = "/oracle/product/10g/lib";
And it didn't require me to call another script to set the env var.
The Env::Modify module addresses this issue, at least for POSIX-y platforms:
use Env::Modify 'source';
source("/myfolder1/myfolder2/myScript.sh");
... environment settings from myScript.sh are now available to Perl ...