I have been working on a node script that does the installation of external dependencies and was wondering if its possible to set the PATH variable using Node.
For example, If I want to install Go using script, I can execute command brew install Go and this works but when I do export GOPATH=$PATH, this is never set.
I also tried using process.env but this also does not work. Also my assumption with process is that it must run in a sandbox. So it will not update/ set anything outside.
So the question is, can I do this? If yes, how?
It is not possible in any programming language. At least on Unix (BSD, MacOS, Illumos etc.) and Linux, I'm not sure about Windows.
As a security feature, Unix processes are designed such that they cannot touch the environment of their parent's process. Only the parent process can influence the environment of their children.
When you are on a command line you are running a shell program (either Bash or Korn Shell or TCSH or Fish etc.). When you execute something on the command line such as a node.js script or Google Chrome or Minecraft these programs will be the children to your shell program. On Unix and Linux these child programs inherit their parent's environment.
That is why you can export environment (global) variables and your programs can access them. However, this relationship is one-way and it was designed that way on purpose.
You may notice that in shell scripts you can "save" commands to set environment variables in a file. For example in Bash you sometimes see people do this:
# variables.sh
export HELLO "World"
However, this will not work if you execute the script as a script:
$ bash variables.sh
$ echo $HELLO
<--------------------- nothing is displayed here
the variable is not exported
You sometimes see people do this instead:
$ . variables.sh
$ echo $HELLO
World
And you may wonder how that works. Well.. in some shell languages the dot command imports the script instead of executing the script as a separate process. Think of it like import or require() in javascript.
Of course, Bash does not understand javascript syntax. It also does not understand C++ or Java or Ruby or PHP. It only understands Bash syntax.
So the longer answer is it is not possible in any programming language except for the language of the shell you are using (eg. if you are using Fish you need to write a script in Fish syntax or if you are using Ksh you need to write a script in Korn syntax).
Related
I need some quick advice on perl script. I created a script that basically calls other perl scripts and many other shell scripts within those. The problem I'm facing, is simply trying to make this run on a universal level by setting one environment variable.
This is on linux RHEL/CentOS by the way...
so I add my variables to .bash_profile and it works without an issue if I MANUALLY source the file first, then run my perl script! This is, OK, but I would like the script to automate this part instead of needing the extra, manual sourcing step.
So my script looks like this... in short
#!/usr/bin/perl
use warnings;
use strict;
`/bin/bash ~/.bash_profile`;
blah blah blah more code etc;
When launching the main script (where this part of the code is) it works no problem. It's all the subsequent calls made to other scripts that are failing...as if it is not passing the variable on to the rest of the scripts.
Any ideas??
Thanks,
The easiest way would be to set the environment variables within perl with $ENV{"name"}=.... These variables then will be propagated automatically to any programs started from within the perl script, no matter if perl or shell scripts
alternatively you could open the .bash_profile, parse it within perl, extract the variables and then set them again within perl with $ENV. This is error prone because there might be several ways to declare the variables.
or you could spawn a shell, which reads the .bash_profile and calls env afterwards. Then you read and parse the output from this shell. This is similar to the previous proposal, but the output you have to parse is more clearly defined.
or you could use a shell script which sources .bash_profile and then spawns the perl script.
Environment variables are inherited by child processes from their parent, so you simply need to launch perl from a shell that sourced ~/.bash_profile.
bash login shells source that file automatically, so it's just a question of setting bash as your login shell. You could also use bash -l.
bash -lc 'exec script.pl args'
If you can't setup the correct environment before launching Perl, you can bootstrap a bash login shell.
if (#ARGV && $ARGV[0] eq 'setup') {
shift(#ARGV);
} else {
exec('bash', '-lc', 'exec "$#"', '-', $^X, $0, 'setup', #ARGV) or die $!;
}
In all three cases, the variable is accessed using
$ENV{VAR_NAME}
Backticks (and system, and open, etc.) spawns a separate process that inherits the environment from your Perl program, but can only affect its own environment, not propogate changes back to your Perl script.
Shell::GetEnv has a few tricks that will help you incorporate those changes to the child environment, but often times you will find it is easier to parse .bash_profile yourself.
The more recent Env::Modify module can handle this task, leveraging the tricks in Shell::GetEnv.
use Env::Modify 'source';
source("$ENV{HOME}/.bash_profile");
using $ENV we can get any value from .profile/.bash_profile
Example:
suppose any variable is store in your .profile/.bash_profile
export CONNECT="Connection"
Retrieve the same variable in your perl scripts from .profile/.bash_profile as:
my $DB_CONNECT = $ENV{CONNECT};
I infrequently have to write bash scripts for various unrelated purposes and while I usually have a good idea what commands I want in the script, I often have no idea what header to use or why I'm using one when I do find it. For example(s):
Standard shell script:
#!/bin/bash
Python:
#!/usr/bin/env python
Scripts seem to work fine without headers but if headers are the standard, there's a reason for them and they shouldn't be ignored. If it has an effect, then it's a valuable tool that could be used to accomplish more.
Minimally, I'd like to know what headers to use with MySQL scripts and what the headers do on Standard, Python, and MySQL scripts. Ideally, I'd like a generic list of headers or an understanding of how to create a header based on what program is being used.
How the Kernel Executes Things
Simplified (a bit), there are two ways the kernel in a POSIX system knows how to execute a program. One, if the program is in a binary format the kernel understands (such as ELF), the kernel can execute it "directly" (more detail out of scope). If the program is a text file starting with a shebang, such as
#!/usr/bin/somebinary -arg
or what-have-you, the kernel actually executes the command as if it had been directed to execute:
/usr/bin/somebinary -arg "$0"
where $0 here is the name of the script file you just tried to execute. (So you can immediately tell why so many scripting languages use # as a comment-starter – it means they don't have to treat the shebang as special.)
PATH and the env command
The kernel does not look at the PATH environment variable to determine which executable you're talking about, so if you are distributing a python script to systems that may have multiple versions of python installed, you can't guarantee that there will be a
#!/usr/bin/python
env, however, is POSIX, so you can count on it existing, and it will look up python in PATH. Thus,
#!/usr/bin/env python
will execute the script with the first python found in your PATH.
BASH, SH and Special Meanings for Invocation
Some programs have special semantics for how they're invoked. In particular, on many systems /bin/sh is a symlink to another shell, such as /bin/bash. While bash does not contain a perfectly POSIXLY_STRICT implementation of sh, when it is invoked as /bin/sh it is stricter than it would be if invoked as plain-old-bash.
MySQL and arg limitations
The shebang line can be length limited and technically, it can only support one argument, so mysql is a bit tricky – you can't expect to pass a username and database name to a mysql script.
#!/usr/bin/env mysql
use mydb;
select * from mytbl;
Will fail because the kernel will try mysql "$0". Even if you have your credentials in a .my.cnf file, mysql itself will try to treat "$0" as a database name. Likewise:
#!/usr/bin/mysql -e
use mydb;
select * from mytbl;
will fail because again, "$0" is not a table name (you hope).
There does not seem to be an appropriate syntax for directly executing a mysql script this way. Your best bet is to pipe the sql commands to mysql directly:
mysql < my_sql_commands
http://mywiki.wooledge.org/BashGuide/Practices#Choose_Your_Shell
When the first line of a script starts with #!, that's what's called a "shebang". When that script is run as an executable, the operating system uses that line to determine how to run the script -- that is to say, to find the program with which the script should be executed.
It's incorrect that "scripts work fine without headers" -- if you don't have a shebang line, you can't be invoked using the execve() call, which means that many (most?) programs won't be able to execute your script. Sometimes invocation from a shell will try to use that shell itself in the absence of a shebang, but you can't trust that to be the case.
(There's an exception to that -- if someone starts your script by running sh yourscript or bash yourscript, the shebang line isn't read at all, and the script they chose is used; however, running scripts this way is a bad practice, as the author typically knows better than the user what the correct interpreter is).
In short:
If you want to use modern features, and you want the user to be able to override the shell version in use by putting a different release of bash earlier in their path, use #!/usr/bin/env bash
If you want to use modern features and ensure that you always run with the system shell, use #!/bin/bash
If you're going to write your script to strictly conform with POSIX sh, use #!/bin/sh
There's not a limited list of shebang lines we can give you, since any native executable (non-script program) can be used as a script interpreter, and thus be placed in a shebang. If you created a file called myscript with #!/usr/bin/env yourprogram, gave it executable permissions, and ran ./myscript foo bar, this would result in /usr/bin/env yourprogram myscript foo bar being invoked; yourprogram would be run by /usr/bin/env (after a PATH lookup), and would be responsible for knowing what to do with myscript and its arguments.
For an extremely detailed history of shebang lines and how they work across systems both modern and ancient, see http://www.in-ulm.de/~mascheck/various/shebang/
This question already has answers here:
How to Share ENV Variables Among Perl Scripts
(2 answers)
Closed 5 years ago.
I am trying to set an environment variable, LD_LIBRARY_PATH, through a Perl script in the following way:
I have created .profile under /root
.profile has an export command say:
export LD_LIBRARY_PATH=/
My Perl script is test.pl and it has:
#!/usr/bin/perl
system(". /root/.profile");
When I execute ./test.pl, LD_LIBRARY_PATH doesn't change.
What am I doing wrong?
Your current script doesn't even change an environment variable in the Perl script itself. Rather, it invokes a shell as a subprocess; that shell process executes . /root/.profile, which updates $LD_LIBRARY_PATH only in that shell process.
You can change an environment variable in a Perl script (more precisely, in the process running the Perl script) by updating %ENV:
$ENV{LD_LIBRARY_PATH} = '/'; # or some more reasonable value
As perldoc -v %ENV says:
%ENV The hash %ENV contains your current environment. Setting a value in "ENV" changes the environment for any child processes you subsequently "fork()" off.
But that probably still won't do what you want; it won't (and can't) affect the environment of the process that invokes the Perl script (your interactive shell), only the Perl process itself and anything it invokes.
I'll assume you want to update $LD_LIBRARY_PATH in your current interactive shell process. To do that, you can have you Perl script print a shell command that will update $LD_LIBRARY_PATH. Then, rather than simply running your Perl script, you can execute it and then evaluate its output. For example:
$ cat env.pl
#!/usr/bin/perl
use strict;
use warnings;
print "export LD_LIBRARY_PATH=/\n";
$ ./env.pl # just prints the command without executing it
export LD_LIBRARY_PATH=/
$ eval $(./env.pl) # executes the command in the current shell
$ echo $LD_LIBRARY_PATH
/
$
This assumes that your current shell is bash or something similar.
Another option: After modifying %ENV, your Perl script can invoke another command, even a new interactive shell. The new process will inherit its environment from the Perl script. This can be a bit cumbersome, though; for example, if the new process is an interactive shell, it won't inherit unexported variables or history from the parent shell.
(One note, not directly related to your question: The fact that you're messing with /root/.profile implies that you're doing things as root (superuser). This can be dangerous. Use the root account (either by logging into it or via sudo only for things that actually need root privileges. For anything else, use a personal user account.
To change the environment in a Perl script, assign to the %ENV hash:
$ENV{'LD_LIBRARY_PATH'} = '/';
If you want to write a program that's used by a shell to change its environment, the way this is generally done is to have the script write shell commands to stdout. The shell then executes this with command substitution and uses eval to execute the resulting commands:
Perl script:
#!/usr/bin/perl
print 'LD_LIBRARY_PATH=\n';
Shell script:
eval "$(/path/to/perlscript)"
For examples of commands that work like this, see tset and ssh-agent.
system starts a new process, and changing the environment there won't affect the environment in the process of your script (usually—there are generally os-dependent means of changing other processes' environments).
The environment in a perl program is associated with %ENV, which is kind of like (it isn't actually) a tied hash to the environment: changing it will change the environment. Thus:
$ENV{LD_LIBRARY_PATH} = '/';
This can now be done with the Env::Modify module:
use Env::Modify 'source';
source("/root/.profile");
... env settings of .profile are now available to Perl ...
You can't do it.
This is from the Perl FAQ:
In the strictest sense, it can't be done--the script executes as a different process from the shell it was started from. Changes to a process are not reflected in its parent--only in any children created after the change. There is shell magic that may allow you to fake it by eval()ing the script's output in your shell; check out the comp.unix.questions FAQ for details.
I have made Bash scripts before and they all ran fine without #!/bin/bash at the beginning.
What's the point of putting it in? Would things be any different?
Also, how do you pronounce #? I know that ! is pronounced as "bang."
How is #! pronounced?
It's a convention so the *nix shell knows what kind of interpreter to run.
For example, older flavors of ATT defaulted to sh (the Bourne shell), while older versions of BSD defaulted to csh (the C shell).
Even today (where most systems run bash, the "Bourne Again Shell"), scripts can be in bash, python, perl, ruby, PHP, etc, etc. For example, you might see #!/bin/perl or #!/bin/perl5.
PS:
The exclamation mark (!) is affectionately called "bang". The shell comment symbol (#) is sometimes called "hash".
PPS:
Remember - under *nix, associating a suffix with a file type is merely a convention, not a "rule". An executable can be a binary program, any one of a million script types and other things as well. Hence the need for #!/bin/bash.
To be more precise the shebang #!, when it is the first two bytes of an executable (x mode) file, is interpreted by the execve(2) system call (which execute programs). But POSIX specification for execve don't mention the shebang.
It must be followed by a file path of an interpreter executable (which BTW could even be relative, but most often is absolute).
A nice trick (or perhaps not so nice one) to find an interpreter (e.g. python) in the user's $PATH is to use the env program (always at /usr/bin/env on all Linux) like e.g.
#!/usr/bin/env python
Any ELF executable can be an interpreter. You could even use #!/bin/cat or #!/bin/true if you wanted to! (but that would be often useless)
It's called a shebang. In unix-speak, # is called sharp (like in music) or hash (like hashtags on twitter), and ! is called bang. (You can actually reference your previous shell command with !!, called bang-bang). So when put together, you get haSH-BANG, or shebang.
The part after the #! tells Unix what program to use to run it. If it isn't specified, it will try with bash (or sh, or zsh, or whatever your $SHELL variable is) but if it's there it will use that program. Plus, # is a comment in most languages, so the line gets ignored in the subsequent execution.
Every distribution has a default shell. Bash is the default on the majority of the systems. If you happen to work on a system that has a different default shell, then the scripts might not work as intended if they are written specific for Bash.
Bash has evolved over the years taking code from ksh and sh.
Adding #!/bin/bash as the first line of your script, tells the OS to invoke the specified shell to execute the commands that follow in the script.
#! is often referred to as a "hash-bang", "she-bang" or "sha-bang".
The shebang is a directive to the loader to use the program which is specified after the #! as the interpreter for the file in question when you try to execute it. So, if you try to run a file called foo.sh which has #!/bin/bash at the top, the actual command that runs is /bin/bash foo.sh. This is a flexible way of using different interpreters for different programs. This is something implemented at the system level and the user level API is the shebang convention.
It's also worth knowing that the shebang is a magic number - a human readable one that identifies the file as a script for the given interpreter.
Your point about it "working" even without the shebang is only because the program in question is a shell script written for the same shell as the one you are using. For example, you could very well write a javascript file and then put a #! /usr/bin/js (or something similar) to have a javascript "Shell script".
The operating system takes default shell to run your shell script. so mentioning shell path at the beginning of script, you are asking the OS to use that particular shell. It is also useful for portability.
It is called a shebang. It consists of a number sign and an exclamation point character (#!), followed by the full path to the interpreter such as /bin/bash. All scripts under UNIX and Linux execute using the interpreter specified on a first line.
Bash standards for “Bourne-Again shell” is just one type of many available
shells in Linux.
A shell is a command line interpreter that accepts and runs commands.
Bash is often the default shell in most Linux distributions. This is why bash is
synonymous to shell.
The shell scripts often have almost the same syntaxes, but they also differ sometimes. For example, array index starts at 1 in Zsh instead of 0 in bash. A script
written for Zsh shell won’t work the same in bash if it has arrays.
To avoid unpleasant surprises, you should tell the interpreter that your shell script
is written for bash shell. How do you do that?
simply begin your bash script into #!/bin/bash
Also you will see some other parameters after #!/bin/bash,
for example
#!/bin/bash -v -x
read this to get more idea.
https://unix.stackexchange.com/questions/124272/what-do-the-arguments-v-and-x-mean-to-bash .
It can be useful to someone that uses a different system that does not have that library readily available. If that is not declared and you have some functions in your script that are not supported by that system, you should declare #/bin/bash. I've ran into this problem before at work and now I just include it as a practice.
I have a script whose content simply exports a variable in linux.
export LD_LIBRARY_PATH=....
I want to run this script in my Perl script so whoever is running my Perl script will have their LD_LIBRARY_PATH set. Can i just do this in the beginning of my Perl script:
#!/usr/bin/perl -w
system(". /myfolder1/myfolder2/myScript.sh");
#!/bin/sh
. /myfolder1/myfolder2/myScript.sh
exec perl -wxS "$0" "$#"
#!/usr/bin/perl -w
# .. the rest of your script as normal
When you run this, it will first be executed by /bin/sh, which is capable of loading myScript.sh into the local environment. sh then execs Perl, which is told to continue from the following line.
This won't work. To change the environment inside your Perl script (and to change the environment that will be passed on to commands run from inside your Perl script), change the %ENV variable.
$ENV{"LD_LIBRARY_PATH"} = ... ;
This won't work. There is no way for a subshell to manipulate the environment of the parent process.
But you could make your script echo the string you want to set as LD_LIBRARY_PATH and then from within your Perl script you could do something like that:
$ENV{LD_LIBRARY_PATH} = `path/to/your/script.sh`;
Of course, a bit of error checking might also be a good idea.
No. Your environment changes made in a child cannot affect the parent. This means running a script will not affect perl. Also perl will not affect the shell from which it was called. You can edit the environment inside perl by changing the special variable %ENV. If there's some kind of unreproducible calculation done in that script, maybe the script should just echo the setting and perl can pick that up on STDOUT and use it.
I {changed directory, modified my environment} in a perl script. How come the change disappeared when I exited the script? How do I get my changes to be visible?
Unix In the strictest sense, it can't be done -- the script executes
as a different process from the shell
it was started from. Changes to a
process are not reflected in its
parent, only in its own children
created after the change.
I had a similar problem a few years ago and whipped up a little module, Env::Sourced, that should do the trick.
use Env::Sourced qw(/myfolder1/myfolder2/myScript.sh);
...
Another option (other than making the changes directly in Perl's %ENV) is to make the changes you want a Perl module, so that you can say:
use MyEnvironment;
and have it modify your environment in all your scripts. It would make it simple to make changes after the fact that will not require editing every script.
The module itself will be simple, something like this:
package MyEnvironment;
$ENV{LD_LIBRARY_PATH} .= ":/some/path/you/want/appended";
# Any other changes you want here.
1;
That won't work. An (unpleasant) alternative might be to replace /usr/bin/perl with a shell script that first executes your script and then executes the perl executable.
This can't be done in the way you're trying to do this.
It either needs a wrapper shell script that sets LD_LIBRARY_PATH and then calls your perl script, or any user executing the script needs to have LD_LIBRARY_PATH set correctly in the first place.
If doing the latter, then this can be managed globally by editing /etc/profile and /etc/cshrc (for ksh, sh, bash, csh and tcsh) shells. You can then test for the value of LD_LIBRARY_PATH in your script and if not set/set incorrectly then print a friendly message to the user. Alternatively individual users can set this in their local .profile/.cshrc files.
Note: you haven't given any information about the environment or useres that might run this, so there's also the possibility that users may set LD_LIBRARY_PATH to something they need. If you do check LD_LIBRARY_PATH for a "good" value in your script, then keep in mind that several paths may have been specified, so you will need to parse this environment variable properly.
If you can find the right place in your perl script, this works as in my example:
$ENV{"LD_LIBRARY_PATH"} = "/oracle/product/10g/lib";
And it didn't require me to call another script to set the env var.
The Env::Modify module addresses this issue, at least for POSIX-y platforms:
use Env::Modify 'source';
source("/myfolder1/myfolder2/myScript.sh");
... environment settings from myScript.sh are now available to Perl ...