I am working on a bash prompt project which acts upon different states in the directory you are currently in.
What I need now is a way to load a mini configuration from the current directory, which is easily done with . .params.conf, except that this method is extremely unsecure, as anyone with write access to the directory you are in can create and execute command as you when you stumble upon a directory with a .params.conf file in it.
What is the best way of loading variables from a file like this?
The variables is going to be mostly in a true/false state, so I am not parsing them in any way so it can be executed.
Possible solutions:
A loop, loading each predefined variable is a possible, but I want to keep the code in a readable fashion.
Put the whole file in an bash array would be the best solution, but how can I populate a key/value dict in bash like this?
If bash can source a file only loading the variables..
I dont know how the format of the params.conf file will look like yet. But I think the easiest would be one line for each param, separated by space. Like one of this on each line: key value that can have space in it.
eval can be very insecure and can still execute malicious code. It's better to use declare:
while read varname value
do
declare "$varname=$value"
done < .params.conf
If your values do not contain quotes, it is rather easy:
while read varname value ; do
eval $varname="'$value'"
done < .params.conf
If there are quotes inside of the variables, you have to be more careful and add some escaping.
Related
I'm sure this is one of the dumbest problems asked on this site, but I am very new to linux, and a little out of my depths. I'm working off of this tutorial here and am stuck on the "add the path" and verify steps.
For this one the tutorial told me to use this:
export PATH=${PATH}:${DTITK_ROOT}/bin:${DTITK_ROOT}/utilities:${DTITK_ROOT}/scripts
I have already defined DTITK_ROOT, and have a few questions about the above instructions.
Should the ${} be left around the DTITK_ROOT?
My DTITK_ROOT is the full path (I think that's the right term) to the file I extracted the program to, should I change that?
What do I write for ${PATH} in that case? I understand that I'm supposed to replace it with something, but I don't know what. Everything I've tried doesn't pass the verify step.
I'm sorry if it seems like a dumb or really simple question, but I don't even know any keywords to google in order to find how to get the answer.
Yes. This is how you access the path stored in DTITK_ROOT. This is called parameter expansion. You can read more about it here.
No, don't change anything. Also, a more commonly used term is absolute path, in comparison to relative path. The absolute path is a path from the root directory, /. Relative path is a path from your current working directory. You can read more about paths in general and the difference between absolute and relative paths here.
You don't replace it with anything. Once again, parameter expansion comes into play and this will be replaced with what is already stored in your path variable. So really all this command is doing is taking your path variable, adding some more paths to it, and then storing it back into your path variable. If you didn't know, the path variable contains paths to all executable files that you would like to execute without typing the full path. Here is a good discussion on path variables, along with other environment variables.
1st command takes care of path
export DTITK_ROOT=mypathonSystem/dtitk
2nd command
export PATH=${PATH}:${DTITK_ROOT}/bin:${DTITK_ROOT}/utilities:${DTITK_ROOT}/scripts
I am not too sure but I think second command should run as is since you defined DDTITK_ROOT in first command
${PATH} is letting the system know where the resources can be found at
have you tried running first command, then running second command unmodified?
Should the ${} be left around the DTITK_ROOT?
Yes. In the case of the shell, it is not essential here because the / that follows the $DTITK_ROOT is enough to signal that we have reached the end of the variable name, but doing ${DTITK_ROOT} explicitly says that the variable name is DTITK_ROOT and not that plus whatever characters might be on the end of it. Other programs (such as make) which allow you to write shell commands to execute might not be so accommodating - make would think that $DTITK_ROOT would be the value of $D followed by the literal characters TITK_ROOT. So, it is a good practice to just get used to putting {} around shell variable names that are longer than a single character.
My DTITK_ROOT is the full path to the file I extracted the program to, should I change that?
If you mean the full path to the directory that you extracted the program to, then that is what you should use. I am assuming that you have something like "export DTITK_ROOT=/Users/huiz/unix/dtitk" (per the example).
On thing you can do is to verify that the value of DTITK_ROOT is available by executing a "echo ${DTITK_ROOT}" to verify that it has the proper value.
It goes like this:
exec("echo '" + param + "' > /etc/test")
And note that it's under root privilege and param has filtered characters like:
'
"
;
`
&
Your question will not get a yes/no answer without much more context. That's okay; instead let us look for weaknesses and highlight areas for concern.
Issue 1: it is almost always better to avoid passing untrusted/externally-supplied data into a shell command; the shell provides a dangerous level of power if someone manages to subvert your command with bad input. Open and write the file in your own code rather than using a shell to do it.
Issue 2: it's impossible to say if that exec() would be safe without knowing what shell the command line will be evaluated by.
Issue 3: if you must pass data via a shell (but see Issue 1), robust filtering is vital. But from your brief description of the filtering, it sounds like you are blacklisting known-bad characters - this is a poor approach, it is always better to whitelist known-good characters and keep that to a very limited set (such alphanumeric).
If you do not specify the PATH, then you cannot be sure what command is executed. You also did not say how you sanitize param. There are a few things to avoid:
Why do you use exec/echo at all? Any programming language can write to files directly.
the "single string" exec() method needs to deal with parsing/escaping. If you use a method which uses a string array, you dont need to be concerned with it.
In your case /etc is a safe dir, but if you use the same code on a less protected directory you might follow a symlink and destroy anything an attacker wants you to
You should be aware that the content of your newly created file contains blank delimited file names, but filesnames can contain blanks. So malicious attackers can produce new (non existing) names. (In fact all characters (including line breaks) besides \0 can be part of the filename).
You need to setup a clean (i.e. only add whitelisted entries) environment. LD_PRELOAD, PATH, IFS, CWD/PWD or SHELL, all can be tainted.
I want to create some bash scripts. They're actually going to be build scripts for Scala, so I'm going to identify them with my own .bld extension. They will be a sort of sub type of a shell script. Hence I want them to be easily recognised as a shell script. Should I call them
ProjectA.bld.sh //or
ProjectA.sh.bld
Edit: My natural inclination would be to go for the former but .tar.gz files seem to follow the latter naming convention.
A shell script doesn't mind what you call it.
It just needs to be..
executable (chmod +x)
in your path
contain a "shebang" as it's first line #!/bin/sh
The shebang determines which program is used to execute your script.
Call it ProjectA.bld.sh (or preferably buildProjectA.sh).
The .sh extension (although not necessary for the script to run) will allow you and everyone else to easily recognise it as a shell script.
While for the most part, naming conventions like this don't really matter at all to Unix/Linux, the usual convention is for the "extensions" to be in the order of the steps used to create the file. So, for example, a file named foo.tar.bz2.gpg.part01 would indicate a sequence of operations like the following:
Use tar to create foo.tar, which contains some other files
Use bzip2 to compress foo.tar into foo.tar.bz2
Use gnupg to encrypt foo.tar.bz2 into foo.tar.bz2.gpg
Use split or something similar to break the file into chunks for transmission/storage, resulting in one or more foo.tar.bz2.gpg.part* files.
The naming conventions are mostly just for human semantic meaning, though, and there's nothing stopping you from doing exactly the opposite, or even something completely random, except your own ability to remember exactly what you did...
Problem
I'm writing a set of scripts to help with automated batch job execution on a cluster.
The specific thing I have is a $OUTPUT_DIR, and an arbitrary $COMMAND.
I would like to execute the $COMMAND such that its output ends up in $OUTPUT_DIR.
For example, if COMMAND='cp ./foo ./bar; mv ./bar ./baz', I would like to run it such that the end result is equivalent to cp ./foo ./$OUTPUT_DIR/baz.
Ideally, the solution would look something like eval PWD="./$OUTPUT_DIR" $COMMAND, but that doesn't work.
Known solutions
[And their problems]
Editing $COMMAND: In most cases the command will be a script, or a compiled C or FORTRAN executable. Changing the internals of these isn't an option.
unionfs, aufs, etc.: While this is basically perfect, users running this won't have root, and causing thousands+ of arbitrary mounts seems like a questionable choice.
copying/ hard/soft links: This might be the solution I will have to use: some variety of actually duplicating the entire content of ./ into ./$OUTPUT_DIR
cd $OUTPUT_DIR; ../$COMMAND : Fails if $COMMAND ever reads files
pipes : only works if $COMMAND doesn't directly work with files; which it usually does
Is there another solution that I'm missing, or is this request actually impossible?
[EDIT:]Chosen Solution
I'm going to go with something where each object in the directory is symbolic-linked into the output directory, and the command is then run from there.
This has the downside of creating a lot of symbolic links, but it shouldn't be too bad.
You can't solve this without making some assumptions about the interface of $COMMAND. There is no single definition of what "output ends up in $OUTPUT_DIR" means. For one program this may be some files, but another program might just print something to stdout and yet another might try sending some data over the internet using some protocol or display something in a GUI and there isn't an obvious way of mapping all of these to "output goes to $OUTPUT_DIR".
So, you need to invent some assumptions and require any $COMMAND implementation to follow them. Then, it may get as simple as requesting that the command accept a parameter such as --target=<DIR>. If your command was some simple command, you would have to create a wrapper script around it to translate that parameter into what the app accepts. cp, mv and a few more utils already accept the parameter --target, so that may be a good starting point.
You cannot set the output directory, you can only set the working directory.
The problem is, once you set the working directory, other references are going to be invalid. For example in your code foo:
cp ./foo ./bar
If you have a specific command, there are workarounds (creating a script that alters arguments, prepending the directory to specific arguments), but in general this is not possible.
i want to make a script (to) that makes it easier for me to enter folders.
so eg. if i type "to apache" i want it to change the current directory to /etc/apache2.
however, when i use the "cd" command inside the script, it seems like it changes the path WITHIN the script, so the path in the shell has not changed.
how could i make this work?
Use an alias or function, or source the script instead of executing it.
BASH FAQ entry #60.
use a function
to_apache(){
cd /etc/apache
}
put in a file eg mylibrary.sh and whenever you want to use it, source the file. eg
#!/bin/bash
source /path/mylibrary.sh
to_apache
As Ignacio said, make it into a function (or perhaps an alias).
The way I tend to do it is have a shell script that creates the function - and the script and the function have the same name. Then once at some point in time, I will source the script ('. funcname') and thereafter I can simply use the function.
I tend to prefer functions to aliases; it is easier to manage arguments etc.
Also, for the specific case of changing directories, I use CDPATH. The trick with using CDPATH is to have the empty entry at the start:
export CDPATH=:/work4/jleffler:/u/jleffler:/work4/jleffler/src:\
/work4/jleffler/src/perl:/work4/jleffler/src/sqltools:/work4/jleffler/lib:\
/work4/jleffler/doc:/u/jleffler/mail:/work4/jleffler/work:/work4/jleffler/ids
On this machine, my main home directory is /work4/jleffler. I can get to most of the relevant sub-directories in one go with 'cd whatever'.
If you don't put the empty entry (or an explicit '.') first, then you can't 'cd' into a sub-directory of the current directory, which is disconcerting at least.
Ignacio Vazquez-Abrams gave a link to what probably answers the question, although I didn't really follow it. The short answer is to use either "source" or a single dot before the command, eg:
. to apache
But, I found there are down problems to this if you have a more complicated script. It seems that the original script filename variable ($0) is lost. I see "-bash" instead, so your script can't echo error text that that would include the full filename.
Also, you can't use the "exit" command, or your shell will exit (especially disconcerting from ssh).
Also, the "basename" function gives an error if you use that.
So, it seems to me that a function might be the only way to get around some of these problems, like if you are passing parameters.