I'm trying to build sophisticated command:
bin/console remote:exec [stage] [revision] -- arguments...
stage and revision arguments are optional (they are derived from current git revision, if omitted), while arguments is a variable length array of words that form command to be executed on remote host - and it gets tricky to find out if first argument is stage, revision, or part of arguments. Is it possible to get all Symfony command arguments that were specified after double dash, so i could segregate arguments?
Related
I'm executing git ls-remote ssh://git#git_repo:port * in two different computers under same network, one Linux another Windows, and on Windows I'm getting the list but on Linux not. No error at all just and empty list on Linux.
Both has the SSH key added to the remote repository and both are able to clone the repository.
Update 1:
Windows Git version: 2.19.2.windows.1
Linux Git version: 2.7.4
Update 2:
The repository is in Gerrit.
Update 3:
I'm facing this problem using the Jenkins plugin Extended Choice Parameter plugin. It has no change since 2016. Any workaround for this would be also an answer.
Any idea?
You probably should use:
git ls-remote ssh://git#git_repo:port
without any suffix, as it defaults to listing everything.
You can use:
git ls-remote ssh://git#git_repo:port '*'
(or the same with double quotes—one or both of these may work on Windows as well). In a Unix/Linux-style command shell, the shell will replace * with a list of all the files in the current directory before running the command, unless you protect the asterisk from the shell.
You can also use a single backlash:
git ls-remote ssh://git#git_repo:port \*
as there are a lot of ways to protect individual characters from shells. The rules get a little complicated, but in general, single quotes are the "most powerful" quotes, while double quotes quote glob characters1 but not other expansions.2 Backslashes quote the immediate next character if you're not already inside quotes (the behavior of backslash within double quotes varies in some shells).
1The glob characters are *, [, and ?. After [, characters inside the glob run to the closing ]. So echo foo[abc] looks for files named fooa, foob, and fooc. Note that . is generally not special: foo.* matches only files whose names start with foo., i.e., including the period: a file named foo does not start with foo., only with foo, and is not matched.
Globs are very different from regular expressions: in regular expressions, . matches any character (like ? does in glob) and asterisk means "repeat previous match zero or more times", so that glob * and regular-expression .* are similar. (In regular expression matches, we also need to consider whether the expression is anchored. Globs are always anchored so that the question does not arise.)
2Most expansions occur with dollar sign $, as in $var or ${var} or $(subcommand), but backquotes also invoke command substitution, as in echo `echo bar`.
I know how regex and wildcards work in general, but I don't really understand why you can use them as parameters.
ls /[!\(][!\(][!\(]/
command results in the following output
...
com.apple.launchd.AIPZ6SAfpO
com.apple.launchd.HarlOx3LWS
com.apple.launchd.VmTi5KDz1h
powerlog
/usr/:
X11 include libexec sbin standalone
bin lib local share
/var/:
agentx empty log netboot rwho
at folders ma networkd spool
audit install mail root tmp
backups jabberd msgs rpc vm
db lib mysql run yp
from my understanding this should match every three character folder name not containing slash /[!\(][!\(][!\(]/
But why can I use it as parameter?
You can't use regular expressions as parameters (or rather, the shell will not treat a string as a regular expression when placed in a parameter). The unquoted glob /[!\(][!\(][!\(]/ matches, in order:
A slash.
Three characters which are not starting brackets.
A slash.
In other words, three-letter root directories not containing ( anywhere.
The shell expands globs to zero (in case of Bash's nullglob, for example) or more arguments which may be passed to execve, as in this command:
$ strace -fe execve echo *
execve("/usr/bin/echo", ["echo", "directory1", "directory2"], 0x7ffcff705ce8 /* 44 vars */) = 0
Not, you don't know.... shell patterns are described in glob(3) while regular expressions (a more elaborate concept) are described in regex(3) Two different libraries used for similar purposes. sh(1) doesn't use regular expressions when substituting parameters at all. It only uses the glob(3) library.
Because that's how the shell works. Any arguments containing (unquoted) glob characters/expressions, are expanded to filenames. That's what happens in, say rm *.txt (since * is a glob character), and that's what happens in ls /[!\(][!\(][!\(]/ (since [abc] is a glob expression).
They're not regular expressions, though. See e.g. https://mywiki.wooledge.org/glob for the syntax.
Situation:
In my Node-RED flow there is a call to another program where I use the exec node.
The arguments are set-up in a function node and passed as the msg.payload to the exec node.
This works fine and returns the expected results as long as the command has no space in it.
Problem:
A customer cannot use the call to the command on his system, because the path tp the installation of the program contains a whitespace.
What I tried and didn't work:
Escaping the whitespace with ^ as read in another question: /opt/path^ toProgram^ directory/program
Escaping the whitespace with \\: /opt/path\\ toProgram\\ directory/program
Quoting the whole string with "": "/opt/path toProgram directory/program"
Quoting the whole string and escaping the whitespace: "/opt/path\\ toProgram\\ directory/program"
Quoting the whole string with '': '/opt/path toProgram directory/program'
Leaving the command line empty/""and combining any of the above points with its arguments to one string (in the function node that sets up the arguments for that exec node) and passing it on as the msg.payload -> input parameters in the exec node config panel. No Success.
What's not the problem:
The program itself works fine, on mine and on the customers system, it's only the path that is different
Other than that specific command string with whitespace, the configuration of the exec node and its msg.payload ( = input parameters or arguments) as well as the "use spawn() instead of exec()?" is fine and works
Request
Is there any other way to escape the whitespace that I'm not aware of so the path to the program can be found? From my understanding the whitespace is interpreted as a separator for the input arguments, which it should be, but not on the command string.
This should be a quick fix in my opinion, however nothing seems to work that usually works in node, js, php or bash.. thanks in advance for any hints and ideas!
Environment:
Node-RED version: v0.15.2 |
Node.js version: v5.12.0 |
Mac OS X 10.11.6 = Darwin 15.6.0 x64 LE
Screenshots
This is the part of the flow:
This config works:
This config does not work:
I've just tested this and option 3 (Quoting the path with "") works fine.
Putting "/home/foo/foo bar/date" in the command section of the exec node executed the binary correctly
After a good nights sleep the solution (at least what I thought at the time) was to adapt the exec node in node-red itself: The original code appended the additional arguments (the ones separated by whitespace) to the command, and then slice the whole string by whitespace into an array (node.cmd is the path to the command with or without whitespace):
before: var arg = node.cmd; arg += " "+msg.payload; arg = arg.match(/(?:[^\s"]+|"[^"]*")+/g);
and the (earlier, incomplete) solution was to prepend the command with the builtin array.unshift(<value>) function to the array after the slice operation, so that the command path with the whitespace is not affected by the array creation:
after: var arg = ""; ...add msg.payload and slice arg string into array.. arg.unshift(node.cmd);
The file can be found at: nodered node 75-exec.js on github, in the lines 46-51
Update to real solution:
After further investigation it was clear, that the Use spawn() instead of exec()? option was the problem: the command part with whitespace in double quotes and using exec() worked fine. And my approach couldn't be used when the command path contained an argument, e.g. in /home/foo bar/date -r 1000.
Following the issue at the official github repository of node-red the command part to the arguments array at the beginning after the slice but instead to have the command path with whitespace in quotes and later unwrapping them, copying the behaviour when exec() is used. He added this line in the original file after line 51: if (/^".*"$/.test(cmd)) { cmd = cmd.slice(1,-1); } and now both spawn() and exec() work the same - credits to him, unfortunately I don't know his username here.
I have a BASH script that has a long set of arguments and two ways of calling it:
my_script --option1 value --option2 value ... etc
or
my_script val1 val2 val3 ..... valn
This script in turn compiles and runs a large FORTRAN code suite that eventually produces a netcdf file as output. I already have all the metadata in the netcdf output global attributes, but it would be really nice to also include the full run command one used to create that experiment. Thus another user who receives the netcdf file could simply reenter the run command to rerun the experiment, without having to piece together all the options.
So that is a long way of saying, in my BASH script, how do I get the last command entered from the parent shell and put it in a variable? i.e. the script is asking "how was I called?"
I could try to piece it together from the option list, but the very long option list and two interface methods would make this long and arduous, and I am sure there is a simple way.
I found this helpful page:
BASH: echoing the last command run
but this only seems to work to get the last command executed within the script itself. The asker also refers to use of history, but the answers seem to imply that the history will only contain the command after the programme has completed.
Many thanks if any of you have any idea.
You can try the following:
myInvocation="$(printf %q "$BASH_SOURCE")$((($#)) && printf ' %q' "$#")"
$BASH_SOURCE refers to the running script (as invoked), and $# is the array of arguments; (($#)) && ensures that the following printf command is only executed if at least 1 argument was passed; printf %q is explained below.
While this won't always be a verbatim copy of your command line, it'll be equivalent - the string you get is reusable as a shell command.
chepner points out in a comment that this approach will only capture what the original arguments were ultimately expanded to:
For instance, if the original command was my_script $USER "$(date +%s)", $myInvocation will not reflect these arguments as-is, but will rather contain what the shell expanded them to; e.g., my_script jdoe 1460644812
chepner also points that out that getting the actual raw command line as received by the parent process will be (next to) impossible. Do tell me if you know of a way.
However, if you're prepared to ask users to do extra work when invoking your script or you can get them to invoke your script through an alias you define - which is obviously tricky - there is a solution; see bottom.
Note that use of printf %q is crucial to preserving the boundaries between arguments - if your original arguments had embedded spaces, something like $0 $* would result in a different command.
printf %q also protects against other shell metacharacters (e.g., |) embedded in arguments.
printf %q quotes the given argument for reuse as a single argument in a shell command, applying the necessary quoting; e.g.:
$ printf %q 'a |b'
a\ \|b
a\ \|b is equivalent to single-quoted string 'a |b' from the shell's perspective, but this example shows how the resulting representation is not necessarily the same as the input representation.
Incidentally, ksh and zsh also support printf %q, and ksh actually outputs 'a |b' in this case.
If you're prepared to modify how your script is invoked, you can pass $BASH_COMMANDas an extra argument: $BASH_COMMAND contains the raw[1]
command line of the currently executing command.
For simplicity of processing inside the script, pass it as the first argument (note that the double quotes are required to preserve the value as a single argument):
my_script "$BASH_COMMAND" --option1 value --option2
Inside your script:
# The *first* argument is what "$BASH_COMMAND" expanded to,
# i.e., the entire (alias-expanded) command line.
myInvocation=$1 # Save the command line in a variable...
shift # ... and remove it from "$#".
# Now process "$#", as you normally would.
Unfortunately, there are only two options when it comes to ensuring that your script is invoked this way, and they're both suboptimal:
The end user has to invoke the script this way - which is obviously tricky and fragile (you could however, check in your script whether the first argument contains the script name and error out, if not).
Alternatively, provide an alias that wraps the passing of $BASH_COMMAND as follows:
alias my_script='/path/to/my_script "$BASH_COMMAND"'
The tricky part is that this alias must be defined in all end users' shell initialization files to ensure that it's available.
Also, inside your script, you'd have to do extra work to re-transform the alias-expanded version of the command line into its aliased form:
# The *first* argument is what "$BASH_COMMAND" expanded to,
# i.e., the entire (alias-expanded) command line.
# Here we also re-transform the alias-expanded command line to
# its original aliased form, by replacing everything up to and including
# "$BASH_COMMMAND" with the alias name.
myInvocation=$(sed 's/^.* "\$BASH_COMMAND"/my_script/' <<<"$1")
shift # Remove the first argument from "$#".
# Now process "$#", as you normally would.
Sadly, wrapping the invocation via a script or function is not an option, because the $BASH_COMMAND truly only ever reports the current command's command line, which in the case of a script or function wrapper would be the line inside that wrapper.
[1] The only thing that gets expanded are aliases, so if you invoked your script via an alias, you'll still see the underlying script in $BASH_COMMAND, but that's generally desirable, given that aliases are user-specific.
All other arguments and even input/output redirections, including process substitutiions <(...) are reflected as-is.
"$0" contains the script's name, "$#" contains the parameters.
Do you mean something like echo $0 $*?
I've written a small bash script to invoke a make command.
The makefile expects args='<arguments to be passed to program>' argument. (i.e. make a args='--aaa 5 --bbb 6')
The problem is when I use the script to invoke the make command, the argument is truncated on its first white space.
For example, when executing make a args='--aaa 5' through the script, the args variable is '--aaa and not '--aaa 5'
I've also tried adding quotes and single quotes but the result is the same.
When I invoke the make command manually through terminal the, the args variable gets all the arguments and their values as expected.
Here is the script call :
args="args='--aaa 5 --bbb 6'"
make a ${args}
You have to double-quote $args to preserve the whitespace.
make a "$args"
I'm assuming the Makefile correctly handles the value once it receives it.