Edit: This question was originally bash specific. I'd still rather have a bash solution, but if there's a good way to do this in another shell then that would be useful to know as well!
Okay, top level description of the problem. I would like to be able to add a hook to bash such that, when a user enters, for example $cat foo | sort -n | less, this is intercepted and translated into wrapper 'cat foo | sort -n | less'. I've seen ways to run commands before and after each command (using DEBUG traps or PROMPT_COMMAND or similar), but nothing about how to intercept each command and allow it to be handled by another process. Is there a way to do this?
For an explanation of why I'd like to do this, in case people have other suggestions of ways to approach it:
Tools like script let you log everything you do in a terminal to a log (as, to an extent, does bash history). However, they don't do it very well - script mixes input with output into one big string and gets confused with applications such as vi which take over the screen, history only gives you the raw commands being typed in, and neither of them work well if you have commands being entered into multiple terminals at the same time. What I would like to do is capture much richer information - as an example, the command, the time it executed, the time it completed, the exit status, the first few lines of stdin and stdout. I'd also prefer to send this to a listening daemon somewhere which could happily multiplex multiple terminals. The easy way to do this is to pass the command to another program which can exec a shell to handle the command as a subprocess whilst getting handles to stdin, stdout, exit status etc. One could write a shell to do this, but you'd lose much of the functionality already in bash, which would be annoying.
The motivation for this comes from trying to make sense of exploratory data analysis like procedures after the fact. With richer information like this, it would be possible to generate decent reporting on what happened, squashing multiple invocations of one command into one where the first few gave non-zero exits, asking where files came from by searching for everything that touched the file, etc etc.
Run this bash script:
#!/bin/bash
while read -e line
do
wrapper "$line"
done
In its simplest form, wrapper could consist of eval "$LINE". You mentioned wanting to have timings, so maybe instead have time eval "$line". You wanted to capture exit status, so this should be followed by the line save=$?. And, you wanted to capture the first few lines of stdout, so some redirecting is in order. And so on.
MORE: Jo So suggests that handling for multiple-line bash commands be included. In its simplest form, if eval returns with "syntax error: unexpected end of file", then you want to prompt for another line of input before proceeding. Better yet, to check for proper bash commands, run bash -n <<<"$line" before you do the eval. If bash -n reports the end-of-line error, then prompt for more input to add to `$line'. And so on.
Binfmt_misc comes to mind. The Linux kernel has a capability to allow arbitrary executable file formats to be recognized and passed to user application.
You could use this capability to register your wrapper but instead of handling arbitrary executable, it should handle all executable.
Related
Often when I run a command that writes to stdout, and that command fails, I have to scroll up (using uncomfortable key-bindings) looking for the place where I pressed Enter, to see what the first error was (out of hundreds others, across many screens of text). This is both annoying and time-consuming. I wish there was a feature which allowed me to pin my current terminal to the place where I am now, then start the command, see only the first lines of the output (as many as fits below my cursor) and let the rest of the output be written but not displayed. In other words I would like a feature to allow me automatically scroll up to the place where I gave the command, to see the first lines of the output (where usually the origin of the failure is displayed).
I searched for it but I didn't find it. Do you know if such feature exists? Or have an idea how to implement it with some tricks or workarounds?
If you have a unique shell prompt you could bind a key to jump between shell prompts, for example something like this will make C-b S jump to the previous shell prompt then S subsequent ones:
bind S copy-mode \; send -X search-backward 'nicholas#myhost:'
bind -Tcopy-mode S send -X search-backward 'nicholas#myhost:'
Or similarly you could search for error strings if they have a recognisable prefix. If you install the tmux 3.1 release candidate, you can search for regular expressions.
Alternatively, you could use capture-pane to load the entire history into an editor with key bindings you prefer, for example:
$ tmux capturep -S- -E- -p|vim -
Or pipe to grep or whatever. Note you will need to use a temporary file for this to work with emacs.
Or try to get into the habit of teeing commands with lots of output to a file to start with.
This is a simple question.
Sometimes on a Terminal when you make a small mistake the console asks ¿Did you mean ...? - ¿Is there a way to quicky accept the suggestion?.
For example:
$ git add . -all
error: did you mean `--all` (with two dashes ?)
Is there a command that repeats the last line, but with the two dashes?
If you forget to write sudo, you can just do sudo !! and it will solve your problem. I want to know if there is something similar but for the error: did you mean case.
In the case of...
$ git add . -all
error: did you mean `--all` (with two dashes ?)
...the message is written by git directly to the terminal. This means that bash has no way of knowing what message was written; it would be literally impossible to implement anything in the shell that could automate putting that correction in place without making programs run under the shell considerably less efficient (by routing their output through the shell rather than directly to the terminal) and changing their behavior (if they ever call isatty() on their stdout or stderr).
That said, you can certainly run
^-all^--all
...if you haven't turned history expansion off, as with set +H (if off, it can be reenabled with set -H). I typically do turn this functionality off, myself; it's often more trouble than it's worth (making commands which would work perfectly well in scripts break in interactive shells when they use characters that history expansion is sensitive to, particularly !).
So I have this existing command that accepts a single argument, but I need something that accepts the argument over stdin instead.
A shell script wrapper like the following works, but as I will be allowing untrusted users to pass arbitrary strings on stdin, I'm wondering if there's potential for someone to execute arbitary commands on the shell.
#!/bin/sh
$CMD "`cat`"
Obviously if $CMD has a vulnerability in the way it processes the argument there's nothing I can do, so I'm concerned stuff like this:
Somehow allow the user to escape the double quotes and pass input into argument #2 of $CMD
Somehow cause another arbitary command to run
The parameter looks fine to me, but the command might be a bit shaky, if it can have a space in it. Also, if you're looking to get just one line from the user then you might prefer this:
#!/bin/bash
read line
exec "$CMD" "$line"
A lot of code would be broken if "$(cmd)" could expand to multiple words.
Mostly, I will not use | less for each and every command from the shell.
Pipe to less is used only when I actually run the command without is and find out that it does not fit on the page. That costs me two runs of the same shell command.
Is there a way so that every time a command result is more than a display page, it automatically gets piped to less?
Pipe it to less -F aka --quit-if-one-screen:
Causes less to automatically exit if the entire file can be dis-
played on the first screen.
The most significant problem with trying to do that is how to get it to turn off when running programs that need a tty.
What I would recommend is that, for programs and utilities you frequently use, create shell functions that wrap them and pipe to less -F. In some cases, you can name the function the same as the program and it will take precedence, but can be overridden.
Here is an example wrapper function which would need testing and perhaps some additional code to handle edge cases, etc.
#!/bin/bash
foo () {
if [[ -p /dev/stdout ]] # you don't want to pipe to less if you're piping to something else
then
command foo "$#" | less -F
else
command foo "$#"
fi
}
If you use the same name as I have in the example, it could break things that expect different behavior. To override the function to run the underlying program directly precede it with command:
command foo
will run foo without using the function of the same name.
You could always pipe to less -E (this will cause less to automatically quit at the end of the file). For commands with short output it would do what you want. I don't think you can automatically pipe to less when there is a lot of output.
In general, automatically piping to less requires the shell to be prescient about the output that will be produced by the commands it runs - and it is hard enough for humans to predict that without trying to make programs do so.
You could write a shell that does it for you - that captures the output (but what about stderr?) and paginates if necessary, but it would most certainly not be a standard shell.
I wrote this wrapper function and put it in my .profile. You can use this before a command and it will automatically pipe it to less if it is longer than 1 page.
lcmd ()
{
echo "$("$#")" | less -F;
};
So 'lcmd ls' would ls the current directory and pipe that output to less.
I have a for loop that I pass through ssh to a server that is formatted to look kind of like this...
i=0
for cmd in cmd_list;
do ${cmd} | sed "s/^/OUTPUT_${i}_: /" &
(( i++ ));
done;
wait
The idea is that the for loop will run a list of commands that I give it and pipe each one to sed where sed prepends each line of output with a command number (0, 1, 2, 3, etc...). Then it backgrounds so as to allow parallel execution. This allows me to keep track of which command the output was associated with since the data could come back simultaneously and all mixed up. This works really well. Depending on the timing of when the commands print out information and when they are complete, the output might look something like this...
OUTPUT_0_: some_data_string_from_a_command
OUTPUT_2_: some_data_string_from_a_command
OUTPUT_0_: some_data_string_from_a_command
OUTPUT_3_: some_data_string_from_a_command
OUTPUT_1_: some_data_string_from_a_command
OUTPUT_1_: some_data_string_from_a_command
However, what I really want to do is this...
do ${cmd} 2>&1 | sed "s/^/OUTPUT_${i}_${PIPESTATUS[0]}: /" &
So I can get back this...
OUTPUT_0_0: some_data_string_from_a_command
OUTPUT_2_1: some_error_message_from_a_command
OUTPUT_0_0: some_data_string_from_a_command
OUTPUT_3_1: some_error_message_from_a_command
OUTPUT_1_0: some_data_string_from_a_command
OUTPUT_1_0: some_data_string_from_a_command
This works fine on the first command if it errors out. I will usually get the non-zero exit code from ${PIPESTATUS[0]}. However, when I have purposefully planted commands further along in the list that I know will fail (i.e. cat /tmp/some_non_existent_file), PIPESTATUS does not give me the proper exit code of the command in the pipe chain. I will sometimes get 0 instead of whatever the real exit code is.
Any idea why this is?
Commands in a pipeline are executed in parallel. This means that they may not have exited by the time you evaluate PIPESTATUS, particularly since PIPESTATUS is expanded by the shell before the sed command is actually run.
I don't really understand what it is you are trying to do, because you appear to want to violate causality. Have you thought about it carefully? You seem to want the exit value of a command with its output, but if it is outputting data it clearly hasn't exited. You are relying on a small timing window always being open. It may be for some short-run-time commands, but that is even questionable since coordinating parallel execution without synchronisation operations is prone to "random" failure.
If you want to capture the data along with the output status of a command, you'll need to save the output away somewhere (in a file for instance), and when the process exits, then output the data with the exit status.