Using ubuntu 10.10 I have the following that I run on the command-line:
result="$(ls -d !(*.*))"
chmod +x $result
This gets a list of files that have no extensions and makes them executable.
But when I move it to a script file (shell) it does not work. From what I have read around the forum this is something to do with command substitution being run in a different a subshell.
But I could not find a solution yet that works in my scrpt :(
So how do you get the result of a command and store it in a variable within a script?
(Since #user000001 does not seem to write their comment into an answer, I'll do the toiling of writing the answer. So credit should got to them, though.)
The feature you are using is the extglob (extended globbing) feature of the bash. This is per default enabled for interactive shells, and per default disabled for non-interactive shells (i. e. shell scripts). To enable it, use the command shopt -s extglob.
Note that this command only has effect for lines below it:
shopt -s extglob
ls -d !(*.*)
It does not effect parsing of the same line:
shopt -s extglob; ls -d !(*.*) # won't work!!
In general I want to warn about using such special features of the bash. It makes the code rather unportable. I'd propose to use POSIX features and tools instead which enable porting the code to another platform rather easily, and they also represent a certain subset of possibilities more developers understand without having to consult the documentation first.
What you want to achieve could also be done using find. This also has the advantage of being unproblematic in combination with strange file names (e. g. containing spaces, quotes, etc.):
find . -maxdepth 1 -type f -name '*.*' -o -exec chmod +x "{}" \;
Related
I am developing on mac and use the following command to confirm file names to a json array:
ls **/*.test.json | jq -R -s -c 'split("\n")[:-1]'
Which gives me the json array:
['folder1/a.test.json', 'folder2/b.test.json', 'c.test.json']
Which is exactly what I want. However, when executing on github action (with linux), the above command produces outcome:
['c.test.json]
And the files within folders are not included.
I confirmed that the folders where checkout successfully because echo $(ls folder1) gives a.test.json.
What is the best way to achieve what I want for the command?
** is not a standard sh feature; your Github action probably requires a POSIX shell script.
To traverse arbitrarily deep directory structure with a shell which doesn't support **, try find:
find . -name '*.test.json' -print |
jq -R -s -c 'split("\n")[:-1]'
If there is only a limited set of directory levels, maybe try
printf '%s\n' */*.test.json *.test.json | jq ...
(Also don't use ls in scripts and perhaps see also useless use of echo.)
Both of these have some gnarly corner cases if you have file names which contain newlines; find could probably be coerced to handle that case, too, but I'll not complicate this answer further; perhaps see https://mywiki.wooledge.org/BashFAQ/020 for a fuller discussion.
If your shell is Bash, ** is available, but typically not enabled out of the box; you enable it with shopt -s globstar ... but perhaps it's better to stick to proper sh in case Github changes the default shell for Actions.
I recently needed to use a Bash for loop to recurse through a few files in a directory:
for video in **/*.{mkv,mp4,webm}; do
echo "$video"
done
After way too much time spent debugging, I realised that the loop was run even when the pattern didn't match, resulting in:
file1.mkv
file2.mp4
**/*.webm # literal pattern printed when no .webm files can be found
Some detailed searching eventually revealed that this is known behaviour in Bash, for which enabling the shell's nullglob option with shopt -s nullglob is intended to be the solution.
Is there a reason that this counter-intuitive behaviour is the default and needs to be explicitly disabled with nullglob, instead of the other way around? Or to put the question another way, are there any disadvantages to always having nullglob enabled?
From man 7 glob:
Empty lists
The nice and simple rule given above: "expand a wildcard pattern
into the list of matching pathnames" was the original UNIX
definition. It allowed one to have patterns that expand into an
empty list, as in
xv -wait 0 *.gif *.jpg
where perhaps no *.gif files are present (and this is not an
error). However, POSIX requires that a wildcard pattern is left
unchanged when it is syntactically incorrect, or the list of
matching pathnames is empty. With bash one can force the
classical behavior using this command:
shopt -s nullglob
(Similar problems occur elsewhere. For example, where old
scripts have
rm `find . -name "*~"`
new scripts require
rm -f nosuchfile `find . -name "*~"`
to avoid error messages from rm called with an empty argument
list.)
In short, it is the behavior required to be POSIX compatible.
Granted though, you can now ask what the rationale for POSIX was to specify that behavior. See this unix.stackexchange.com question for some reasons/history.
ls /tmp
How can I run the same command but using reverse polish notation?
Is there a mode that would allow me to do this or something similar to that?
I could use xargs but that's a lot more typing:
echo /tmp | xargs ls
This would be ideal:
/tmp ls
or
/tmp | ls
Bash (I assume you are using it) is a shell for unixoid systems.
As far as I know, bash doesn't provide such a mode. You could use a different shell that provides this feature. Searching in the web, this was my first result: https://github.com/iconmaster5326/RPOS, but maybe it is far from stable ;)
Alternatively, you can make a command that reverses it's argument list and execute it.
The usage would be like this:
reversex /tmp ls
reversex A.txt B.txt cp
Here is an example of such a command:
#!/bin/bash
for i in "$#"
do
CMDLINE="$i $CMDLINE"
done
$CMDLINE
If you name it /usr/local/bin/reversex and make it executable, you should be able to use simple reverse commands with the prefix reversex. I can not give a warranty that it works. Note that the arguments are parsed twice and have to be escaped twice, too.
I need to find all the .psd files on my Linux system (dedicated web hosting). I tried something like this: ls -R *.psd, but that's not working. Suggestions?
You can use the following find command to do that:
find /path/to/search -iname '*.psd'
iname does a case insensitive search.
you also can
ls ./**/*.psd
but:
you must have bash version 4+
you must have shopt -s globstar #in your .bashrc or .profile, etc....
will search case sensitive (or you must set shopt -s nocaseglob too)
Is it possible to get, using Bash, a list of commands starting with a certain string?
I would like to get what is printed hitting <tab> twice after typing the start of the command and, for example, store it inside a variable.
You should be able to use the compgen command, like so:
compgen -A builtin [YOUR STRING HERE]
For example, "compgen -A builtin l" returns
let
local
logout
You can use other keywords in place of "builtin" to get other types of completion. Builtin gives you shell builtin commands. "File" gives you local filenames, etc.
Here's a list of actions (from the BASH man page for complete which uses compgen):
alias Alias names. May also be specified as -a.
arrayvar Array variable names.
binding Readline key binding names.
builtin Names of shell builtin commands. May also be specified as -b.
command Command names. May also be specified as -c.
directory Directory names. May also be specified as -d.
disabled Names of disabled shell builtins.
enabled Names of enabled shell builtins.
export Names of exported shell variables. May also be specified as -e.
file File names. May also be specified as -f.
function Names of shell functions.
group Group names. May also be specified as -g.
helptopic Help topics as accepted by the help builtin.
hostname Hostnames, as taken from the file specified by the HOSTFILE shell
variable.
job Job names, if job control is active. May also be specified as
-j.
keyword Shell reserved words. May also be specified as -k.
running Names of running jobs, if job control is active.
service Service names. May also be specified as -s.
setopt Valid arguments for the -o option to the set builtin.
shopt Shell option names as accepted by the shopt builtin.
signal Signal names.
stopped Names of stopped jobs, if job control is active.
user User names. May also be specified as -u.
variable Names of all shell variables. May also be specified as -v.
A fun way to do this is to hit M-* (Meta is usually left Alt).
As an example, type this:
$ lo
Then hit M-*:
$ loadkeys loadunimap local locale localedef locale-gen locate
lockfile-create lockfile-remove lockfile-touch logd logger login
logname logout logprof logrotate logsave look lorder losetup
You can read more about this in man 3 readline; it's a feature of the readline library.
If you want exactly how bash would complete
COMPLETIONS=$(compgen -c "$WORD")
compgen completes using the same rules bash uses when tabbing.
JacobM's answer is great. For doing it manually, i would use something like this:
echo $PATH | tr : '\n' |
while read p; do
for i in $p/mod*; do
[[ -x "$i" && -f "$i" ]] && echo $i
done
done
The test before the output makes sure only executable, regular files are shown. The above shows all commands starting with mod.
Interesting, I didn't know about compgen. Here a script I've used to do it, which doesn't check for non-executables:
#!/bin/bash
echo $PATH | tr ':' '\0' | xargs -0 ls | grep "$#" | sort
Save that script somewhere in your $PATH (I named it findcmd), chmod u+w it, and then use it just like grep, passing your favorite options and pattern:
findcmd ^foo # finds all commands beginning with foo
findcmd -i -E 'ba+r' # finds all commands matching the pattern 'ba+r', case insensitively
Just for fun, another manual variant:
find -L $(echo $PATH | tr ":" " ") -name 'pattern' -type f -perm -001 -print
where pattern specifies the file name pattern you want to use. This will miss commands that are not globally executable, but which you have permission for.
[tested on Mac OS X]
Use the -or and -and flags to build a more comprehensive version of this command:
find -L $(echo $PATH | tr ":" " ") -name 'pattern' -type f
\( \
-perm -001 -or \
\( -perm -100 -and -user $(whoami)\) \
\) -print
will pick up files you have permission for by virtue of owning them. I don't see a general way to get all those you can execute by virtue of group affiliation without a lot more coding.
Iterate over the $PATH variable and do ls beginningofword* for each directory in the path?
To get it exactly equivalent, you would need to filter out only executable files and sort by name (should be pretty easy with ls flags and the sort command).
What is listed when you hit are the binary files in your PATH that start with that string. So, if your PATH variable contains:
PATH=/usr/local/bin:/usr/bin:/bin:/usr/games:/usr/lib/java/bin:/usr/lib/java/jre/bin:/usr/lib/qt/bin:/usr/share/texmf/bin:.
Bash will look in each of those directories to show you the suggestions once you hit . Thus, to get the list of commands starting with "ls" into a variable you could do:
MYVAR=$(ls /usr/local/bin/ls* /usr/bin/ls* /bin/ls*)
Naturally you could add all the other directories I haven't.