I need to execute "rm -rf" on remote machine running Ubuntu in order to clear specified folder.
If I use command like following everything goes fine.
rm -rf "home/blahblah/blah"/*
But if I run the same command in Linux PowerShell I would get ALL files removed.
Is there any way to specify path to be handled the same way in bash and PS? Thank you!
tl;dr
Unfortunately, as of PowerShell 7.0, you must do the following (if you want to use the external rm utility):
sh -c "rm -rf 'home/blahblah/blah'/*"
Note that I've switched to single-quoting ('...') around the path, so I could use it inside a double-quoted ("...") string. While the reverse ('... "..."/*') should work as-is, it currently requires additional escaping ('... \"...\"/*') - see this answer.
However, if the path preceding the /* doesn't actually need quoting - notably if it doesn't contain spaces - you can simply call:
rm -rf home/blahblah/blah/*
You're seeing a very unfortunate difference between PowerShell (as of 7.0) and POSIX-like shells such as Bash with respect to the handling of string arguments composed of both quoted and unquoted parts.
PowerShell parses "home/blahblah/blah"/* as two arguments:
home/blahblah/blah becomes the first argument, as-is in this case.
/* is interpreted as the second argument, which, due to being unquoted and due to an external utility being called, triggers PowerShell's emulation of the globbing (filename expansion) behavior of POSIX-like shells, which means that all files and directories in the root directory are being passed as individual arguments.
Therefore, the arguments that the rm utility actually receives are: -rf, home/blahblah/blah, /bin, /boot, /dev, ... - which is clearly not the intent.
This problematic behavior is discussed in this GitHub issue.
Passing the command to sh, the default shell on Unix-like platforms, instead, bypasses the problem.
Related
I found in a makefile the following commands:
$(var):
mkdir -p $(#D)
What is the meaning of this command?
$(VAR) expands to the value of the variable VAR. This is a Make variable (not a shell etc variable). For example, if earlier in your Makefile you define
VAR=ick/poo
then VAR expands to ick/poo, and #D in your recipe expands to the directory part, ick.
As you seem to be confused about the relationship between shell and make, I should perhaps point out that these are two different languages, though in a Makefile, you will encounter both; the recipes - the parts which are indented by a tab - will be passed to a shell for evaluation (though normally the shell will be /bin/sh, not Bash, unless you specifically override the Make variable SHELL to force it).
In the shell, by the way, the superficially similar construct $(cmd) performs a command substitution; that is, the command cmd will be evaluated and its output will be inserted as text. So for example,
echo Running in $(pwd)
will print
Running in /home/you
if executed in the directory /home/you (the command pwd prints out your current working directory). ... Though in a Makefile, the dollar sign will normally be evaluated and consumed by make itself; so to pass a literal dollar sign to the shell, you have to double it.
test:
echo Running in $$(pwd)
As already explained by #tripleee $(var) expands to the variable. Because it is here listed before a colon it means that it is a target in a Makefile.
For $(#D) see 10.5.3 Automatic Variables in the make manual:
The directory part of the file name of the target, with the trailing slash removed. If the value of ‘$#’ is dir/foo.o then ‘$(#D)’ is dir. This value is . if ‘$#’ does not contain a slash.
NOTE: This is NOT a shell script. This is a makefile. Please use "man make" for a description about what "make" does.
Some weeks ago, a senior team member removed an important oracle database file(.dbf) unexpectedly. Fortunately, We could restore the system by using back-up files which was saved some days ago.
After seeing that situation, I decided to implement a solution to make atleast a double confirmation when typing rm command on the prompt. (checks more than rm -i)
Even though we aliasing rm -i as default, super speedy keyboardists usually make mistakes like that member, including me.
At first, I replaced(by using alias) basic rm command to a specific bash script file which prints and confirms many times if the targets are related on the oracle database paths or files.
simply speaking, the script operates as filter before to operate rm. If it is not related with oracle, then rm will operate as normal.
While implementing, I thought most of features are well operated as I expected only user prompt environment except one concern.
If rm command are called within other scripts(provided oracle, other vendor modifying oracle path, installer, etc) or programs(by using system call).
How can i distinguish that situation?
If above provided scripts met modified rm, That execution doesn't go ahead anymore.
Do you have more sophisticated methods?
I believe most of reader can understand my lazy explanation.
If you couldn't get clear scenery from above, let me know. I will elaborate more.
We read at man bash:
Aliases are not expanded when the shell is not interactive, unless the
expand_aliases shell option is set using shopt.
Then if you use alias to make rm invoke your shell script, other scripts won't use it by default. If it's what you want, then you're already safe.
The problem is if you want your version of rm to be invoked by scripts and do something smart when it happens. Alias is not enough for the former; even putting your rm somewhere under $PATH is not enough for programs explicitly calling /bin/rm. And for programs that aren't shell scripts, unlink system call is much more likely to be used than something like system("rm ...").
I think that for the whole "safe rm" thing to be useful, it should avoid prompts even when invoked interactively. Every user will develop the habit of saying "yes" to it, and there is no known way around that. What might work is something that moves files to recycle bin instead of deletion, making damage easy to undo (as I seem to recall, there were ready to use solutions for this).
The answer is into the alias manpage:
Note aliases are not expanded by default in non-interactive
shell, and it can be enabled by setting the expand_aliases shell
option using shopt.
Check it by yourself with man alias ;)
Anyway, i would do it in the same way you've chosen
To distinguish the situation: You can create an env variable say, APPL, which will be set to say export APPL="DATABASE . In your customized rm script, perform the double checkings only if the APPL is DATABASE (which indicates a database related script), not otherwise which means the rm call is from other scripts.
If you're using bash, you can export your shell function, which will make it available in scripts, too.
#!/usr/bin/env bash
# Define a replacement for `rm` and export it.
rm() { echo "PSYCH."; }; export -f rm
Shell functions take precedence over builtins and external utilities, so by using just rm even scripts will invoke the function - unless they explicitly bypass the function by invoking /bin/rm ... or command rm ....
Place the above (with your actual implementation of rm()) either in each user's ~/.bashrc file, or in the system-wide bash profile - sadly, its location is not standardized (e.g.: Ubuntu: /etc/bash.bashrc; Fedora /etc/bashrc)
I read that the executables for the commands issued using exec() calls are supposed to be stored in directories that are part of the PATH variable.
Accordingly, I found the executables for ls, chmod, grep, cat in /bin.
However, I could not find the executable for cd.
Where is it located?
A process can only affect its own working directory. When an executable is executed by the shell it executes as a child process, so a cd executable (if one existed) would change that child process's working directory without affecting the parent process (the shell), hence the cd command must be implemented as a shell built-in that actually executes in the shell's own process.
cd is a shell built-in, unfortunately.
$ type cd
cd is a shell builtin
...from http://www.linuxquestions.org/questions/linux-newbie-8/whereis-cd-sudo-doesnt-find-cd-464767/
But you should be able to get it working with:
sh -c "cd /somedir; do something"
Not all utilities that you can execute at a shell prompt need actually exist as actual executables in the filesystem. They can also be so-called shell built-ins, which means – you guessed it – that they are built into the shell.
The Single Unix Specification does, in general, not specify whether a utility has to be provided as an executable or as a built-in, that is left as a private internal implementation detail to the OS vendor.
The only exceptions are the so-called special built-ins, which must be provided as built-ins, because they affect the behavior of the shell itself in a manner that regular executables (or even regular built-ins) can't (for example set, which sets variables that persist even after set exits). Those special built-ins are:
break
:
continue
.
eval
exec
exit
export
readonly
return
set
shift
times
trap
unset
Note that cd is not on that list, which means that cd is not a special built-in. In fact, according to the specification, it would be perfectly legal to implement cd as a regular executable. It's just not possible, for the reasons given by the other answers.
And if you scroll down to the non-normative section of the specification, i.e. to the part that is not officially part of the specification but only purely informational, you will find that fact explicitly mentioned:
Since cd affects the current shell execution environment, it is always provided as a shell regular built-in.
So, the specification doesn't require cd to be a built-in, but it's simply impossible to do otherwise.
Note that sometimes utilities are provided both as a built-in and as an executable. A good example is the time utility, which on a typical GNU system is provided both as an executable by the Coreutils package and as a shell regular built-in by Bash. This can lead to confusion, because when you do man time, you get the manpage of the time executable (the time builtin is documented in man builtins), but when you execute time you get the time built-in, which does not support the same features as the time executable whose manpage you just read. You have to explicitly run /usr/bin/time (or whatever path you installed Coreutils into) to get the executable.
According to this, cd is always a built-in command and never an executable:
Since cd affects the current shell execution environment, it is always provided as a shell regular built-in.
cd is part of the shell; an internal command. There is no binary for it.
The command cd is built-in in your command line shell. It could not affect the working directory of your shell otherwise.
I also searched the executable of "cd" and there is no such.
You can work with chdir (pathname) in C, it has the same effect.
I have just started using Linux and I am curious how shell built-in commands such as cd are defined.
Also, I'd appreciate if someone could explain how they are implemented and executed.
If you want to see how bash builtins are defined then you just need to look at Section 4 of The Bash Man Page.
If, however, you want to know how bash bultins are implemented, you'll need to look at the Bash source code because these commands are compiled into the bash executable.
One fast and easy way to see whether or not a command is a bash builtin is to use the help command. Example, help cd will show you how the bash builtin of 'cd' is defined. Similarly for help echo.
The actual set of built-ins varies from shell to shell. There are:
Special built-in utilities, which must be built-in, because they have some special properties
Regular built-in utilities, which are almost always built-in, because of the performance or other considerations
Any standard utility can be also built-in if a shell implementer wishes.
You can find out whether the utility is built in using the type command, which is supported by most shells (although its output is not standardized). An example from dash:
$ type ls
ls is /bin/ls
$ type cd
cd is a shell builtin
$ type exit
exit is a special shell builtin
Re cd utility, theoretically there's nothing preventing a shell implementer to implement it as external command. cd cannot change the shell's current directory directly, but, for instance, cd could communicate new directory to the shell process via a socket. But nobody does so because there's no point. Except very old shells (where there was not a notion of built-ins), where cd used some dirty system hack to do its job.
How is cd implemented inside the shell? The basic algorithm is described here. It can also do some work to support shell's extra features.
Manjari,
Check the source code of bash shell from ftp://ftp.gnu.org/gnu/bash/bash-2.05b.tar.gz
You will find that the definition of shell built-in commands in not in a separate binary executable but its within the shell binary itself (the name shell built-in clearly suggests this).
Every Unix shell has at least some builtin commands. These builtin commands are part of the shell, and are implemented as part of the shell's source code. The shell recognizes that the command that it was asked to execute was one of its builtins, and it performs that action on its own, without calling out to a separate executable. Different shells have different builtins, though there will be a whole lot of overlap in the basic set.
Sometimes, builtins are builtin for performance reasons. In this case, there's often also a version of that command in $PATH (possibly with a different feature set, different set of recognized command line arguments, etc), but the shell decided to implement the command as a builtin as well so that it could save the work of spawning off a short-lived process to do some work that it could do itself. That's the case for bash and printf, for example:
$ type printf
printf is a shell builtin
$ which printf
/usr/bin/printf
$ printf
printf: usage: printf [-v var] format [arguments]
$ /usr/bin/printf
/usr/bin/printf: missing operand
Try `/usr/bin/printf --help' for more information.
Note that in the above example, printf is both a shell builtin (implemented as part of bash itself), as well as an external command (located at /usr/bin/printf). Note that they behave differently as well - when called with no arguments, the builtin version and the command version print different error messages. Note also the -v var option (store the results of this printf into a shell variable named var) can only be done as part of the shell - subprocesses like /usr/bin/printf have no access to the variables of the shell that executed them.
And that brings us to the 2nd part of the story: some commands are builtin because they need to be. Some commands, like chmod, are thin wrappers around system calls. When you run /bin/chmod 777 foo, the shell forks, execs /bin/chmod (passing "777" and "foo") as arguments, and the new chmod process runs the C code chmod("foo", 777); and then returns control to the shell. This wouldn't work for the cd command, though. Even though cd looks like the same case as chmod, it has to behave differently: if the shell spawned another process to execute the chdir system call, it would change the directory only for that newly spawned process, not the shell. Then, when the process returned, the shell would be left sitting in the same directory as it had been in all along - therefore cd needs to be implemented as a shell builtin.
A Shell builtin -- http://linux.about.com/library/cmd/blcmdl1_builtin.htm
for eg. -
which cd
/usr/bin/which: no cd in (/usr/bin:/usr/local/bin......
Not a shell builtin but a binary.
which ls
/bin/ls
http://ss64.com/bash/ this will help you.
and here is shell scripting guide
http://www.freeos.com/guides/lsst/
I've run into a really silly problem with a Linux shell script. I want to delete all files with the extension ".bz2" in a directory. In the script I call
rm "$archivedir/*.bz2"
where $archivedir is a directory path. Should be pretty simple, shouldn't it? Somehow, it manages to fail with this error:
rm: cannot remove `/var/archives/monthly/April/*.bz2': No such file or directory
But there is a file in that directory called test.bz2 and if I change my script to
echo rm "$archivedir/*.bz2"
and copy/paste the output of that line into a terminal window the file is removed successfully. What am I doing wrong?
TL;DR
Quote only the variable, not the whole expected path with the wildcard
rm "$archivedir"/*.bz2
Explanation
In Unix, programs generally do not interpret wildcards themselves. The shell interprets unquoted wildcards, and replaces each wildcard argument with a list of matching file names.
if $archivedir might contain spaces, then rm $archivedir/*.bz2 might not do what you
You can disable this process by quoting the wildcard character, using double or single quotes, or a backslash before it. However, that's not what you want here - you do want the wildcard expanded to the list of files that it matches.
Be careful about writing rm $archivedir/*.bz2 (without quotes). The word splitting (i.e., breaking the command line up into arguments) happens after $archivedir is substituted. So if $archivedir contains spaces, then you'll get extra arguments that you weren't intending. Say archivedir is /var/archives/monthly/April to June. Then you'll get the equivalent of writing rm /var/archives/monthly/April to June/*.bz2, which tries to delete the files "/var/archives/monthly/April", "to", and all files matching "June/*.bz2", which isn't what you want.
The correct solution is to write:
rm "$archivedir"/*.bz2
Your original line
rm "$archivedir/*.bz2"
Can be re-written as
rm "$archivedir"/*.bz2
to achieve the same effect. The wildcard expansion is not taking place properly in your existing setup. By shifting the double-quote to the "front" of the file path (which is legitimate) you avoid this.
Just to expand on this a bit, bash has fairly complicated rules for dealing with metacharacters in quotes. In general
almost nothing is interpreted in single-quotes:
echo '$foo/*.c' => $foo/*.c
echo '\\*' => \\*
shell substitution is done inside double quotes, but file metacharacters aren't expanded:
FOO=hello; echo "$foo/*.c" => hello/*.c
everything inside backquotes is passed to the subshell which interprets them. A shell variable that is not exported doesn't get defined in the subshell. So, the first command echoes blank, but the second and third echo "bye":
BAR=bye echo `echo $BAR`
BAR=bye; echo `echo $BAR`
export BAR=bye; echo `echo $BAR`
(And getting this to print the way you want it in SO takes several tries is apparently impossible...)
The quotes are causing the string to be interpreted as a string literal, try removing them.
I've seen similar errors when calling a shell script like
./shell_script.sh
from another shell script. This can be fixed by invoking it as
sh shell_script.sh
Why not just rm -rf */*.bz2? Works for me on OSX.