Set a temporary environment ($PATH) - linux

I may fall into a X-Y problem with this question and I encourage you guys to correct me if I am wrong.
I would like to configure a toolchain environment that can work on different platforms and compiler versions. I initially wrote a long Perl script that generates a configuration Makefile that contain only variables. I wanted to be simple so I did not write anything complex using automake or autoconf. Moreover I wanted the reconfiguration process to be very fast. In my case my own written ./configure does everything in less than a second. I am very happy with that.
However I feel I can use a better approach using environment variables. Instead of writing a Makefile with the specific variables I can set the current shell environment directly. For example:
export cc=gcc
Unfortunately, some variables are already declared in the $PATH. The solution is to add the new $PATH in the front of the other:
export PATH=/new/toolchain/path:$PATH
echo $PATH
/new/toolchain/path:/old/toolchain/path:/usr/bin:/bin...
I feel this is ugly I would like to remove the old path before adding the new one.
To conclude:
Is it better to use the environment instead of custom makefiles to set a build configuration?
How to properly adjust existing environment variables?

When I have several variables to set, I write a wrapper script which I then use as a prefix to the command that I want to modify. That lets me use the prefix either
applying to a single command, such as make, or
initializing a shell, so that subsequent commands use the altered settings.
I use wrappers for
setting compiler options (such as clang, to set the CC variable, making configure scripts "see" it as the chosen compiler),
setting locale variables, to test with POSIX C versus en_US versus en_US.UTF-8, etc.
testing with reduced environments, such as in cron.
Each of the wrappers does what is needed to identify the proper PATH, LD_LIBRARY_PATH, and similar variables.
For example, I wrote this ad hoc script about ten years ago to test with a local build of python:
#!/bin/bash
ver=2.4.2
export TOP=/usr/local/python-$ver
export PATH=$TOP/bin:$PATH
export LD_LIBRARY_PATH=`newpath -n LD_LIBRARY_PATH -bd $TOP/lib $TOP/lib/gcc/i686-pc-linux-gnu/$ver`
if test -d $TOP
then
exec $*
else
echo no $TOP
exit 1
fi
and used it as with-python-2.4.2 myscript.
Some wrappers simply call another script.
For example, I use this wrapper around the configure script to setup variables for cross-compiling:
#!/bin/sh
# $Id: cfg-mingw,v 1.7 2014/09/20 20:49:31 tom Exp $
# configure to cross-compile using mingw32
BUILD_CC=${CC:-gcc}
unset CC
unset CXX
TARGET=`choose-mingw32`
if test -n "$TARGET"
then
PREFIX=
test -d /usr/$TARGET && PREFIX="--prefix=/usr/$TARGET"
cfg-normal \
--with-build-cc=$BUILD_CC \
--host=$TARGET \
--target=$TARGET \
$PREFIX "$#"
else
echo "? cannot find MinGW compiler in path"
exit 1
fi
where choose-mingw32 and cfg-normal are scripts that (a) find the available target name for the cross-compiler and (b) provide additional options to the configure script.
Others may suggest shell aliases or functions. I do not use those for this purpose because my command-line shell is usually tcsh, while I run these commands from (a) other shell scripts, (b) directory editor, or (c) text-editor. Those use the POSIX shell (except of course, for scripts requiring specific features), making aliases or functions of little use.

You can create an individualized environment for a particular command invocation:
VAR1=val1 VAR2=val2 VAR3=val3 make
I find this cleaner than doing:
export VAR1=val1
export VAR2=val2
export VAR3=val3
make
unless you're in a wrapper script and maybe even then as with
VAR1=val1 VAR2=val2 VAR3=val3 make the VAR variables will be whatever they were before the make invocation (including but not limited to unexported and nonexistent).
Long lines is a non-issue, you can always split it across several lines:
VAR1=val1\
VAR2=val2\
VAR3=val3\
make
You can set up environment variables like this for any Unix command.
The shell will all set it up.
Some applications (such as make or rake) will modify their environment based on arguments that look like variable definitions (see prodev_paris's answer), but that depends on the application.

Is it better to use the environment instead of custom makefiles to set a build configuration?
The best practice for build systems is to not depend on any environment variables at all. So that nothing more is necessary to build your project than:
git clone ... my_project
make -C my_project
Having to set environment variables is error prone and may lead to inconsistent builds.
How to properly adjust existing environment variables?
You may not need to adjust those at all. By using complete paths to tools like compilers you disentangle your build system from the environment.

As we all know, it is preferrable to integrate standard tools for a task like building your products instead of creating your own approach. The effort usually pays off in the long term.
That being said, a simple approach would be to define different environment files (e.g. build-phone.env) setting working directory, PATH, CC etc. for your different products and source your environment files interactively on demand:
. /path/to/build-phone.env
[your build commands]
. /path/to/build-watch.env
[your build commands]

I think you may benefit from using direct variable definition when you call your makefile, like in the following:
make FOO=bar target
Where FOO is the variable you want to set with value bar.
Note that in this case it take precedence over environment definition! So you can easily override your PATH variable...
Please have a look at this detail topic for more info: https://stackoverflow.com/a/2826178/4716013

Related

Can fish, zsh and bash import the same configuration file?

I've recently been trying to switch from bash to zsh, or fish.
I have some alisa and PATH settings, but I don't want to manually copy them to zshrc or config.fish.
I tried writing them in a single file and using source ~/.myshrc to use them.
The alisa statement can be sourced normally. But when sourcing PATH in fish shell I got an error:
In fish, please use {$JAVA_HOME}. export
CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
I know that fish and bash syntax are incompatible.
So is there a general syntax that allows me to modify the PATH in myshrc and then all three shells can use it?
myshrc file like this:
# alias
alias apts="apt search"
alias sf="aptitude search -F '%c %p %d %D %I'"
alias apti="sudo aptitude install"
alias aptup="sudo aptitude update"
alias aptgr="sudo aptitude upgrade"
alias aptpu="sudo aptitude purge"
# transset xterm
transset -t -a 0.95 >> /dev/null 2>&1
# set npm path
export PATH=~/.npm-global/bin:$PATH
# set java path
export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
export JRE_HOME=${java_home}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
# set android path
export ANDROID_HOME="/home/moly/Launcher/AndroidSDK/"
export PATH="${PATH}:${ANDROID_HOME}tools/:${ANDROID_HOME}platform-tools/"
Short answer:
If you embrace the "fish way" of doing things, you can get your startup config down to just one line ...
Explanation:
First off, just to pass along some random knowledge (that I originally learned here in a related answer on Stack Overflow), code that is written in such a way that it runs the same way in two different languages is known as a polyglot. It's typically considered a "puzzle" of sorts, rather than a practical implementation. It's really not advised as a solution to your question.
Next up, my general recommendation is to embrace the fish way of doing things as much as possible. Sure, there are times you'll need to drop back to POSIX for compatibility reasons, but fish syntax is (IMHO) much cleaner.
I especially like that, under fish, my config files are almost empty. It's just not necessary under fish to have many of the items that you have in your bash startup.
Let's break down the four types of items you have in your startup config:
Aliases
$PATH changes
Other exported variables
Other Command(s)
Aliases
I recommend against putting aliases in your fish startup files. It's just not necessary. Instead, define the alias once at the commandline and use the -s (save) option. In your case:
alias -s apts="apt search"
alias -s sf="aptitude search -F '%c %p %d %D %I'"
alias -s apti="sudo aptitude install"
alias -s aptup="sudo aptitude update"
alias -s aptgr="sudo aptitude upgrade"
alias -s aptpu="sudo aptitude purge"
Your aliases will be "permanently" defined at that point in all fish instances, with the advantage that they are "lazy loaded" (only loaded into memory when you run them the first time) rather than being loaded at all times.
$PATH modifications
Likewise, fish provides a handy helper with which you can add to your path once and have it take effect in all instances (both currently running and future).
fish_add_path "~/.npm-global/bin" "/usr/lib/jvm/java-17-openjdk-amd64/bin" "~/Launcher/AndroidSDK/tools" "~/Launcher/AndroidSDK/platform-tools"
This prepends those paths to the built-in fish universal variable$fish_user_paths which is automatically prepended to the system path. Note that, in your bash rc, you have appended the Android SDK paths, but that's probably not necessary. Typically, you will want to prepend user paths ahead of the system path.
Other exported variables
This part is a little contentious, because fish universal variables can have unintended side-effects when exported to other processes.
On one hand, one of the fish maintainers (#faho) mentions in this answer that:
In general, what you want is to just put the set -gx into ~/.config/fish/config.fish. That's fish's configuration file.
Fish also has "universal" variables, which are stored persistently, but they interact awkwardly with exporting so I wouldn't recommend it.
I've had others (see comments on this answer) who have worked extensively with fish advise against it as well.
On the other hand, 196 upvotes on this answer seem to indicate that folks like universals for this purpose. That's not to say that the majority is correct -- I've seen some really bad answers with a lot of upvotes.
However, I personally like using them to simplify my config files.
If you so choose, you can:
set -Ux JAVA_HOME "/usr/lib/jvm/java-17-openjdk-amd64"
set -Ux JRE_HOME "$JAVA_HOME/jre"
set -Ux --path CLASSPATH ".:$JAVA_HOME/lib:$JRE_HOME/lib"
set -Ux ANDROID_HOME "/home/moly/Launcher/AndroidSDK/"
Again, as these are universal variables, they only need to be set once on the commandline. You can then remove those statements from your fish startup.
Understanding universal/global variable shadowing:
Primarily, if a global variable is set (for instance, by the parent bash process from which you launch fish), then it will override a universal variable of the same name.
For instance (pathologic example, but can easily occur in the real world, especially if you aren't aware of the potential):
# Start in Fish
-> set -Ux JAVA_HOME "/usr/lib/jvm/java-17-openjdk-amd64"
-> set --show JAVA_HOME
$JAVA_HOME: set in universal scope, exported, with 1 elements
$JAVA_HOME[1]: |/usr/lib/jvm/java-17-openjdk-amd64|
-> bash
-> export JAVA_HOME="~/.local/share/jvm/java-17-openjdk-amd64"
-> fish
-> set --show JAVA_HOME
$JAVA_HOME: set in global scope, exported, with 1 elements
$JAVA_HOME[1]: |~/.local/lib/jvm/java-17-openjdk-amd64|
$JAVA_HOME: set in universal scope, exported, with 1 elements
$JAVA_HOME[1]: |/usr/lib/jvm/java-17-openjdk-amd64|
In that particular fish session, the Universal variable will be shadowed by the global one that was exported in bash.
See this Github issue of many that demonstrate the real world potential for problems.
Again, with that in mind, I personally believe the benefits of exported universal variables outweigh the risks, but I wanted to be sure to present both viewpoints.
Other commands
If you've followed along so far (and haven't fallen asleep -- I known I over-explain ...), then you'll realize that there's (potentially) only the one command that must remain in your fish startup:
transset -t -a 0.95 >> /dev/null 2>&1
Even then, I have a suggestion. Fish automatically sources any .fish file in ~/.config/fish/conf.d. I like to keep my config "modularized", so that I can tell at-a-glance what is being loaded at startup.
I would just create:
~/.config/fish/conf.d/xterm.fish:
transset -t -a 0.95 >> /dev/null 2>&1
At that point, you are running like I do -- no fish.config at all! (Well, other than the fact that the newer fish versions irritatingly, automatically create a completely unnecessary empty one if it doesn't exist ...)
One hacky way to use your existing configuration while trying to transition to a new shell like fish is simply to end your ~/.bashrc with fish, like so:
# alias
alias apts="apt search"
alias sf="aptitude search -F '%c %p %d %D %I'"
alias apti="sudo aptitude install"
alias aptup="sudo aptitude update"
alias aptgr="sudo aptitude upgrade"
alias aptpu="sudo aptitude purge"
# transset xterm
transset -t -a 0.95 >> /dev/null 2>&1
# set npm path
export PATH=~/.npm-global/bin:$PATH
# set java path
export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
export JRE_HOME=${java_home}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
# set android path
export ANDROID_HOME="/home/moly/Launcher/AndroidSDK/"
export PATH="${PATH}:${ANDROID_HOME}tools/:${ANDROID_HOME}platform-tools/"
# start fish
exec fish
I don't think this is a very good solution, but this is the only one I know of to make fish inherit your bash environment without rewriting/translating your bashrc. While this is probably bad practice and could cause other sorts of issues, on the short term this could be what you need if you just want to try out fish in known territory with your aliases and PATH without spending time writing a configuration file perhaps for nothing.
I have no experience with zsh and don't know if it can equally inherit the bash environment like that, but wouldn't be surprised if it can.

Linux environment variables referencing other variables

I am using a boot script from a network vendor, I am using this on RedHat 7.2 The start script sets up the environment with several variables, however I don't think these variables are set-up correctly.
I have added the start-up script to /etc/environment and I can see that the variables are defined and available to all users.
This is an example of how the variables are defined in the script:
export V1=/opt/nameofsupplier/sdk/CentOS-RHEL-7-x86_64
export V2=${V1}/lib/cam
There are many more, if I try this from a terminal:
cd $V1
It works fine, however if I try:
cd $V2
I get:
base: cd $V1/lib/cam: No such file or directory
The path is valid, and if I do this in the shell:
export V2=${V1}/lib/cam
cd $V2
It works without any error, how do I fix the script?
You may be right in suspecting an ill-definition of these variables.
/etc/environment can only contain variable definitions - it is not executed like a normal script (see its documentation here, which says Variable expansion does not work in /etc/environment.), so no variable expansion of V1 in the definition of V2 takes place. Therefore V2 is not correctly defined.
Try to source /etc/environment lines in the system-wide /etc/profile (or its equivalent, depending on the shells of your users) or in specific users' ~/.profiles.As the last resort you can just plain copy the respective lines of /etc/environment to the above mentioned scripts (but this will make it harder to maintain).
You could also correct the definitions in /etc/environment not to rely on expansion, i.e. like this:
export V2=/opt/nameofsupplier/sdk/CentOS-RHEL-7-x86_64/lib/cam
(assuming there are not too much of them to be corrected). But this will also be hard to maintain.

Import environment settings into Makefile (ubuntu and osx)

In a shell script I can
. conf/environment
Can I do the same in Makefile?
Make has both include and -include (as well as sinclude that is kept for compatibility with other make tools) statements (later stands for “optional” inclusion). So you can do something like this:
PLATFORM := $(shell uname)
include conf/environment_$(PLATFORM).mk
Where every environment_*.mk defines the same variables but with different values depending on the platform they are targeting.
See §3.3 “Including Other Makefiles” of the GNU Make documentation for more details.
UPDATE:
If you are trying to actually import environment variables by running a shell script, there are two options. The first is to run your script before running make. Then you can access those variables inside Makefile. So you would do source conf/environment && make. Option number two is to modify your script and instead of doing export, do echo and then use Make's eval command to execute that output as commands.
Looks like there's no good solution, so here's the cleanest hack I can think of:
have three layers of build scripts
layer 1 is the Makefile
layer 2 consists of a bunch of shell scripts, one per make target
In each shell script in layer 2, do
. conf/environment
then run the actual build script in layer 3

Aliasing two different versions of the same program in linux?

I have an old version of a program sitting on my machine. This program recently had a version upgrade. The way I used to run my old program was by typing "runProgram". The path to the runscript of my program was specified in my PATH variable as
PATH = ....:/path/to/my/old/programs/bin
I want to run the new version of this same program alongside my old program and the way I was thinking of doing it was by modifying my PATH variable as follows:
PATH = ....:/path/to/my/old/programs/bin:/path/to/my/new/programs/bin
What I want to achieve is some way to alias these two paths so that when I type 'runVersion1', the previous version is executed and when I type 'runVersion2', the new version is executed?
Is there a way to achieve that?
Thanks
If the program itself runs other programs from the bin directory, then when you run a version 1 program, you want to ensure that the version 1 directory is on the PATH ahead of the version 2 directory, and vice versa when you run a version 2 program. That is something I deal with all the time, and I deal with it by ensuring that the PATH is set appropriately.
In my $HOME/bin, I would place two scripts:
RunVersion1
export PATH=/path/to/my/old/programs/bin:$PATH
# Set other environment variables as needed
exec runProgram "$#"
RunVersion2
export PATH=/path/to/my/new/programs/bin:$PATH
# Set other environment variables as needed
exec runProgram "$#"
This technique of placing shell scripts on my PATH ahead of other programs allows me to pick which programs I run.
Semi-Generic Version
Often, I'll use a single program to set the environment and then link it to the various program names that I want to handle. It then looks at $0 and runs that:
export PATH=/path/to/my/new/programs/bin:$PATH
# Set other environment variables as needed
exec $(basename $0 2) "$#"
If this script is linked to RunProgram2, the basename command lops off the 2 from the end of RunProgram2 and then executes RunProgram from the more recent directory.
I've used this general technique for accessing 32-bit and 64-bit versions of the software on a single machine, too. The programs I deal with tend to have more complex environments than just a setting of $PATH, so the scripts are bigger.
One of the main advantages of scripts in $HOME/bin over aliases and the like is that it doesn't much matter which shell I'm stuck with using; it works the same way. Plus I don't have so many places to look to find where the alias is defined (because it isn't defined).
I would put two alias definitions in your ~/.bashrc (depending what shell you are using).
alias runVersion1='/path/to/my/old/programs/bin/program'
alias runVersion2='/path/to/my/new/programs/bin/program'
After editing that file you need to relogin or simply execute
. ~/.bashrc
The way you suggest with $PATH won't do what you want. One way that might:
Given that usually, /usr/local/bin is in $PATH, and that that is the standard location for "local binaries", you do the following:
sudo ln -s /path/to/my/old/programs/bin/myprogram /usr/local/bin/runVersion1
sudo ln -s /path/to/my/new/programs/bin/myprogram /usr/local/bin/runVersion2
Alternatively, if you don't want it to be system-wide (i.e. instead, just for your user), you could:
ln -s /path/to/my/old/programs/bin/myprogram $HOME/bin/runVersion1
ln -s /path/to/my/new/programs/bin/myprogram $HOME/bin/runVersion2
(assuming $HOME/bin is in your $PATH)
Now this won't necessarily fix your problem - could use a little more information in the question, BUT it should help you get further with what you're trying to do.

Can a makefile update the calling environment?

Is it possible to update the environment from a makefile? I want to be able to create a target to set the client environment variables for them. Something like this:
AXIS2_HOME ?= /usr/local/axis2-1.4.1
JAVA_HOME ?= /usr/java/latest
CLASSPATH := foo foo
setenv:
export AXIS2_HOME
export JAVA_HOME
export CLASSPATH
So that the client can simply do:
make setenv all
java MainClass
and have it work without them needing to set the classpath for the java execution themselves.
Or am I looking to do this the wrong way and there is a better way?
No, you can't update the environment in the calling process this way. In general, a subprocess cannot modify the environment of the parent process. One notable exception is batch files on Windows, when run from a cmd shell. Based on the example you show, I guess you are not running on Windows though.
Usually, what you're trying to accomplish is done with a shell script that sets up the environment and then invokes your intended process. For example, you might write a go.sh script like this:
!#/bin/sh
AXIS2_HOME=/usr/local/axix2-1.4.1
JAVA_HOME=/usr/java/latest
CLASSPATH=foo foo
export AXIS2_HOME
export JAVA_HOME
export CLASSPATH
java MainClass
Make go.sh executable and now you can run your app as ./go.sh. You can make your script more elaborate too, if you like -- for example, you may want to make "MainClass" a parameter to the script rather than hard coding it.
From your question I am assuming you're using the bash shell.
You can place the variable definitions in a shell script, like so:
AXIS2_HOME=/usr/local/axis2-1.4.1
export AXIS2_HOME
#etc
And then source the script into the current environment, with
source <filename>
or just
. <filename>
That executes the script in the current shell (i.e. no child process), so any environment changes the script makes will persist.
The quick answer is yes, however in your code, you would need to define the variables in the setenv: directive. Doing it at the beginning of the Makefile makes it a local variable to the Makefile. I would use LOCAL_... at the top of the file then set it in the setenv: directive with VAR=LOCAL_VAR etc... Also remember that you will need to call the makefile with make setenv only. I would really look into doing this in a bash script as the variable needs to be created outside of the Makefile. Once the variable has been generated in the environment, you should be able to assign and export from the Makefile.

Resources