Directory depth in recursive script - linux

hi i'd like to get some help with my linux bash homeworks.
i have to make a script that gets a directory and returns the depth of the deepest subdirectory (+1 for each directory).
I must do it recursively.
I must use 'list_dirs.sh' that takes the virable dir and echo its subdirs.
thats what i got so far:
dir=$1
sub=`source list_dirs.sh`
((depth++))
for i in $sub
do
if [ -n "$sub" ] ; then
./depthScript $dir/$i
fi
done
if ((depth > max)) ; then
max=$depth
echo $max
fi
after testing with a dir that supose to return 3 I got insted:
1
1
1
1
it seems like my depth counter forget previous values and I get output for
each directory.. need some help!

You can use bash functions to create recursive function calls.
Your function would ideally echo 0 in the base case where it is called on a directory with no subdirectories, and echo 1+$(getDepth $subdir) in the case where some subdirectory $subdir exists. See this question on recursive functions in bash for a framework.

When you run a script normally (i.e. it's in your PATH and you just enter its name, or you enter an explicit path to it like ./depthScript), it runs as a subprocess of the current shell. This is important because each process has its own variables. Variables also come in two kinds: shell variables (which are only available in that one process) and environment variables (the values of which get exported to subprocesses but not back up from them). And depending on where you want a variable's value to be available, there are three different ways to define them:
# By default, variables are shell variable that's only defined in this process:
shellvar=something
# `export` puts a variable into the environment, so it'll be be exported to subprocesses.
# You can export a variable either while setting it, or as a separate operation:
export envvar=something
export anotherenvvar
anotherenvvar=something
# You can also prefix a command with a variable assignment. This makes an
# environment variable in the command process's environment, but not the current
# shell process's environment:
prefixvar=something ./depthScript $dir/$i
Given the above assignments:
shellvar is defined in the current shell process, but not in any other process (including the subprocess created to run depthScript).
envvar and anotherenvvar will be inherited by the subprocess (and its subprocesses, and all subprocesses for later commands), but any changes made to it in those subprocesses have no effect at all in the current process.
prefixvar is available only in the subprocess created to run depthScript (and its subprocesses), but not in the current shell process or any other of its subprocesses.
Short summary: it's a mess because of the process structure, and as a result it's best to just avoid even trying to pass values around between scripts (or different invocations of the same script) in variables. Use environment variables for settings and such that you want to be generally available (but don't need to be changed much). Use shell variables for things local to a particular invocation of a script.
So, how should you pass the depth values around? Well, the standardish way is for each script (or command) to print its output to "standard output", and then whatever's using the script can capture its output to either a file (command >outfile) or a variable (var=$(command)). I'd recommend the latter in this case:
depth=$(./depthScript "$dir/$i")
if ((depth > max)) ; then
max=$depth
fi
Some other recommendations:
Think your control and data flow through. The current script loops through all subdirectories, then at the end runs a single check for the deepest subdir. But you need to check each subdirectory individually to see if it's deeper than the current max, and at the end report the deepest of them.
Double-quote your variable references (as I did with "$dir/$i" above). Unquoted variable references are subject to word splitting and wildcard expansion, which is the source of much grief. It looks like you'll need to leave $sub unquoted because you need it to be split into words, but this will make the script unable to cope with directory names with spaces. See BashFAQ #20: "How can I find and safely handle file names containing newlines, spaces or both?"
The if [ -n "$sub" ] ; then test is irrelevant. If $sub is empty, the loop will never run.
In a shell script, relative paths (like ./depthScript) are relative to whatever the working directory of the parent process, not to the location of the script. If someone runs your script from another directory, ./depthScript will not work. Use "$BASH_SOURCE" instead. See BashFAQ #28: "How do I determine the location of my script? I want to read some config files from the same place."
When trying to troubleshoot a script, it can help to put set -x before the troublesome section. This makes the shell print each command as it runs, so you can see what's going on.
Run your scripts through shellcheck.net -- it'll point out a lot of common mistakes.

Related

How can I get the name of the sourced script in tcsh?

I'm looking for a way to get the name of a script that's being sourced from another script that's being executed in tcsh.
If I needed to the the name of a script being executed (not sourced), it's $0. If I need to get the name of a script that's being sourced from the command line, I can get it from $_. But when an executed script sources a script, I get an empty value for $_ in the sourced script, so I can't get the script name or pathname from that.
I'm looking for a non-manual method for getting that information.
There isn't really anything for this; source is mostly just a way to read the file and run it in the current scope.
However, it does accept arguments; from the tcsh manpage:
source [-h] name [args ...]
The shell reads and executes commands from name. The commands
are not placed on the history list. If any args are given,
they are placed in argv. (+) source commands may be nested; if
they are nested too deeply the shell may run out of file
descriptors. An error in a source at any level terminates all
nested source commands. With -h, commands are placed on the
history list instead of being executed, much like `history -L'.
So for example source file.csh file.csh will have argv[1] set to file.csh.
Another option is to simple set a variable before the source command:
set src = "file.csh" # Will be available in file.csh
source file.csh
If you can't or don't want to modify the source call then you're out of luck as far as I know. (t)csh is an old crusty shell with many awkward things, large and small, and I would generally discourage using it for scripting unless you really don't have any option available.
$_ simply gets the last commandline from history; maybe, very maybe it's possible to come up with a super-hacky solution to (ab)use the history for this in some way, but it seems to me that just typing the filename twice is a lot easier.

What does the operator colon in the satement "export variable=lib:/dev/input/event0" mean in linux environment??

first time see such statement by exporting a variable. How to use it and what does it mean?
The : character itself doesn't mean anything on its own. An environment variable is just that - a variable, either unset or containing some value. The value is then used by another program, so what the : means depends on what program is using the variable.
Often it is used as a separator, as with the $PATH variable - you list various directories you want checked when you execute a command in a shell without specifying a full path (eg, /bin:/usr/bin:/usr/sbin - each directory is checked).
In the example you give, lib: looks like it might be a prefix of some sort. But in the end, it really depends on what will be using the variable.

bash: set array env variable and de-referencing it from any shell script fails

I set the a array as an environment variable in this manner
eg. script test.sh
in test.sh
#!/bin/bash
export STRING=( "str1" "str2" )
source test.sh
now in script test-1.sh
#!/bin/bash
echo ${STRING[0]}
the response is nothing, just a blank line,
whereas, if I try to set STRING="str1" in test.sh
and do echo $STRING in test-1.sh, this works.
tests are executed from root user only,
Now how to set array as env variable , so that I can call the elements of array as per requirement? Earlier, I have tried to even modify /etc/bashrc and that also didn't result in anything positive.
I need to set the array as env variable as there may be many scripts that i have to write which shall use these variable settings.
can anybody provide me suggestions to correct me where I am doing wrong?
Read the fine manual, "bugs" section.
Array variables may not (yet) be exported.
Though, I don't know that many consider this an actual bug. Other shells that support ksh-style arrays don't allow exporting them either.
You may pass around array definitions rather easily, through parameters or variables or the environment. It isn't usually very useful though.
function f {
unset -v "$2"
typeset "$2"
eval "${!1}"
typeset -p "$2"
}
typeset -a a=(a b c)
myArr=$(typeset -p a) f myArr a
The misunderstanding is in thinking that environment variables are only used by shells - they are not. No attributes, including readonly, integer, and arrays, can be exported into the environment block. Environment variables may be read by any language, C, C++, Perl, Java, Python, PHP, and so on. They also exist on Windows.
So, how could another language support Bash specific attributes? All environment variables are converted to strings, except in Bash where array values are not exported at all.
Korn shell will export just the first element. ksh93 also does some exec exploitation to preserve variable attributes exported to Korn shell children.
By the way, it is considered bad practice to use UPPERCASE for variable names, since they could collide with those used by the shell. Also, on Bash 3, the name STRING has issues when exported (fixed in Bash 4).
The environment variables passed from processes to their children are unstructured strings; arrays cannot be supported. You can demonstrate this in Bash:
export x=foo
printenv x
That outputs foo. If I now cause x to become an array
x=(foo bar)
printenv x
We see no output (x is not exported).
You're trying to put an array into an environment variable, and environment variables can only be strings. bash doesn't have a method for properly serializing/deserializing arrays; do so manually.
This says passing array as env variable is possible with bash script though. It works for defining and dereferencing the array in bash command, but it does not work when dereferencing from bash script (ubuntu 18.04).

exec() security

I am trying to add security of GET query to exec function.
If I remove escapeshellarg() function, it work fine. How to fix this issue?
ajax_command.php
<?php
$command = escapeshellarg($_GET['command']);
exec("/usr/bin/php-cli " . $command);
?>
Assume $_GET['command'] value is run.php -n 3
What security check I can also add?
You want escapeshellcmd (escape a whole command, or in your case, sequence of arguments) instead of escapeshellarg (escape just a single argument).
Notice that although you have taken special precautions, this code allows anyone to execute arbitrary commands on your server anyways, by specifying the whole php script in a -r option. Note that php.ini can not be used to restrict this, since the location of it can be overwritten with -c. In short (and with a very small error margin): This code creates a severe security vulnerability.
escapeshellarg returns a quoted value, so if it contains multiple arguments, it won't work, instead looking like a single stringesque argument. You should probably look at splitting the command up into several different parameters, then each can be escaped individually.
It will fail unless there's a file called run.php -n 3. You don't want to escape a single argument, you want to escape a filename and arguments.
This is not the proper way to do this. Have a single PHP script run all your commands for you, everything specified in command line arguments. Escape the arguments and worry about security inside that PHP file.
Or better yet, communicate through a pipe.

How do you pass on filenames to other programs correctly in bash scripts?

What idiom should one use in Bash scripts (no Perl, Python, and such please) to build up a command line for another program out of the script's arguments while handling filenames correctly?
By correctly, I mean handling filenames with spaces or odd characters without inadvertently causing the other program to handle them as separate arguments (or, in the case of < or > — which are, after all, valid if unfortunate filename characters if properly escaped — doing something even worse).
Here's a made-up example of what I mean, in a form that doesn't handle filenames correctly: Let's assume this script (foo) builds up a command line for a command (bar, assumed to be in the path) by taking all of foo's input arguments and moving anything that looks like a flag to the front, and then invoking bar:
#!/bin/bash
# This is clearly wrong
FILES=
FLAGS=
for ARG in "$#"; do
echo "foo: Handling $ARG"
if [ x${ARG:0:1} = "x-" ]; then
# Looks like a flag, add it to the flags string
FLAGS="$FLAGS $ARG"
else
# Looks like a file, add it to the files string
FILES="$FILES $ARG"
fi
done
# Call bar with the flags and files (we don't care that they'll
# have an extra space or two)
CMD="bar $FLAGS $FILES"
echo "Issuing: $CMD"
$CMD
(Note that this just an example; there are lots of other times one needs to do this and that to a bunch of args and then pass them onto other programs.)
In a naive scenario with simple filenames, that works great. But if we assume a directory containing the files
one
two
three and a half
four < five
then of course the command foo * fails miserably in its task:
foo: Handling four < five
foo: Handling one
foo: Handling three and a half
foo: Handling two
Issuing: bar four < five one three and a half two
If we actually allow foo to issue that command, well, the results won't be what we're expecting.
Previously I've tried to handle this through the simple expedient of ensuring that there are quotes around each filename, but I've (very) quickly learned that that is not the correct approach. :-)
So what is? Constraints:
I want to keep the idiom as simple as possible (not least so I can remember it).
I'm looking for a general-purpose idiom, hence my making up the bar program and the contrived example above instead of using a real scenario where people might easily (and reasonably) go down the route of trying to use features in the target program.
I want to stick to Bash script, I don't want to call out to Perl, Python, etc.
I'm fine with relying on (other) standard *nix utilities, like xargs, sed, or tr provided we don't get too obtuse (see #1 above). (Apologies to Perl, Python, etc. programmers who think #3 and #4 combine to draw an arbitrary distinction.)
If it matters, the target program might also be a Bash script, or might not. I wouldn't expect it to matter...
I don't just want to handle spaces, I want to handle weird characters correctly as well.
I'm not bothered if it doesn't handle filenames with embedded nul characters (literally character code 0). If someone's managed to create one in their filesystem, I'm not worried about handling it, they've tried really hard to mess things up.
Thanks in advance, folks.
Edit: Ignacio Vazquez-Abrams pointed me to Bash FAQ entry #50, which after some reading and experimentation seems to indicate that one way is to use Bash arrays:
#!/bin/bash
# This appears to work, using Bash arrays
# Start with blank arrays
FILES=()
FLAGS=()
for ARG in "$#"; do
echo "foo: Handling $ARG"
if [ x${ARG:0:1} = "x-" ]; then
# Looks like a flag, add it to the flags array
FLAGS+=("$ARG")
else
# Looks like a file, add it to the files array
FILES+=("$ARG")
fi
done
# Call bar with the flags and files
echo "Issuing (but properly delimited, not exactly as this appears): bar ${FLAGS[#]} ${FILES[#]}"
bar "${FLAGS[#]}" "${FILES[#]}"
Is that correct and reasonable? Or am I relying on something environmental above that will bite me later. It seems to work and it ticks all the other boxes for me (simple, easy to remember, etc.). It does appear to rely on a relatively recent Bash feature (FAQ entry #50 mentions v3.1, but I wasn't sure whether that was arrays in general of some of the syntax they were using with it), but I think it's likely I'll only be dealing with versions that have it.
(If the above is correct and you want to un-delete your answer, Ignacio, I'll accept it provided I haven't accepted any others yet, although I stand by my statement about link-only answers.)
Why do you want to "build up" a command? Add the files and flags to arrays using proper
quoting and issue the command directly using the quoted arrays as arguments.
Selected lines from your script (omitting unchanged ones):
if [[ ${ARG:0:1} == - ]]; then # using a Bash idiom
FLAGS+=("$ARG") # add an element to an array
FILES+=("$ARG")
echo "Issuing: bar \"${FLAGS[#]}\" \"${FILES[#]}\""
bar "${FLAGS[#]}" "${FILES[#]}"
For a quick demo of using arrays in this manner:
$ a=(aaa 'bbb ccc' ddd); for arg in "${a[#]}"; do echo "..${arg}.."; done
Output:
..aaa..
..bbb ccc..
..ddd..
Please see BashFAQ/050 regarding putting commands in variables. The reason that your script doesn't work is because there's no way to quote the arguments within a quoted string. If you were to put quotes there, they would be considered part of the string itself instead of as delimiters. With the arguments left unquoted, word splitting is done and arguments that include spaces are seen as more than one argument. Arguments with "<", ">" or "|" are not a problem in any case since redirection and piping is performed before variable expansion so they are seen as characters in a string.
By putting the arguments (filenames) in an array, spaces, newlines, etc., are preserved. By quoting the array variable when it's passed as an argument, they are preserved on the way to the consuming program.
Some additional notes:
Use lowercase (or mixed case) variable names to reduce the chance that they will collide with the shell's builtin variables.
If you use single square brackets for conditionals in any modern shell, the archaic "x" idiom is no longer necessary if you quote the variables (see my answer here). However, in Bash, use double brackets. They provide additional features (see my answer here).
Use getopts as Let_Me_Be suggested. Your script, though I know it's only an example, will not be able to handle switches that take arguments.
This for ARG in "$#" can be shortened to this for ARG (but I prefer the readability of the more explicit version).
See BashFAQ #50 (and also maybe #35 on option parsing). For the scenario you describe, where you're building a command dynamically, the best option is to use arrays rather than simple strings, as they won't lose track of where the word boundaries are. The general rules are: to create an array, instead of VAR="foo bar baz", use VAR=("foo" "bar" "baz"); to use the array, instead of $VAR, use "${VAR[#]}". Here's a working version of your example script using this method:
#!/bin/bash
# This is clearly wrong
FILES=()
FLAGS=()
for ARG in "$#"; do
echo "foo: Handling $ARG"
if [ x${ARG:0:1} = "x-" ]; then
# Looks like a flag, add it to the flags array
FLAGS=("${FLAGS[#]}" "$ARG") # FLAGS+=("$ARG") would also work in bash 3.1+, as Dennis pointed out
else
# Looks like a file, add it to the files string
FILES=("${FILES[#]}" "$ARG")
fi
done
# Call bar with the flags and files (we don't care that they'll
# have an extra space or two)
CMD=("bar" "${FLAGS[#]}" "${FILES[#]}")
echo "Issuing: ${CMD[*]}"
"${CMD[#]}"
Note that in the echo command I used "${VAR[*]}" instead of the [#] form because there's no need/point to preserving word breaks here. If you wanted to print/record the command in unambiguous form, this would be a lot messier.
Also, this gives you no way to build up redirections or other special shell options in the built command -- if you add >outfile to the FILES array, it'll be treated as just another command argument, not a shell redirection. If you need to programmatically build these, be prepared for headaches.
getopts should be able to handle spaces in arguments correctly ("file name.txt"). Weird characters should work as well, assuming they are correctly escaped (ls -b).

Resources