Beyond Compare provides "Select for compare" and "Compare to Selected" by using two nautilus scripts (stored in /home/user/.gnome2/nautilus-scripts).
Script 1: Select for compare
#!/bin/sh
quoted=$(echo "$NAUTILUS_SCRIPT_SELECTED_FILE_PATHS" | awk 'BEGIN { FS = "\n" } { printf "\"%s\" ", $1 }' | sed -e s#\"\"##)
echo "$quoted" > $HOME/.beyondcompare/nautilus
Script 2: Compare to Selected
#!/bin/sh
arg2=$(cat $HOME/.beyondcompare/nautilus)
arg1=$(echo "$NAUTILUS_SCRIPT_SELECTED_FILE_PATHS" | awk 'BEGIN { FS = "\n" } { printf "\"%s\" ", $1 }' | sed -e s#\"\"##)
bcompare $arg1 $arg2
I am trying to do similar scripts for Meld, but it is not working.
I am not familiar with shell scripts. Can anyone help me understand this:
quoted=$(echo "$NAUTILUS_SCRIPT_SELECTED_FILE_PATHS" | awk 'BEGIN { FS = "\n" } { printf "\"%s\" ", $1 }' | sed -e s#\"\"##)
so that I can adapt to meld.
If you are not rolling your own solution for the sake of learning, I would suggest installing the diff-ext extension to nautilus. It is cross platform and if you are running Debian/Ubuntu installing it should be as simple as sudo apt-get install diff-ext.
Check out some screenshots here - http://diff-ext.sourceforge.net/screenshots.shtml
The quoted=$( ...) assigns whatever output there is to the variable named quoted, and can be used later in the script as $quoted OR ${quoted} OR "${quoted}" OR "$quoted"
The '|' char is called a 'pipe' in unix/linux and it connects the output of the preceding command to feed into the following command.
So you just take the script apart 1 piece at a time and see what it does,
quoted=$(
# I would execute below by itself first
echo "$NAUTILUS_SCRIPT_SELECTED_FILE_PATHS"
# then add on this piped program to see how data gets transformed
| awk 'BEGIN { FS = "\n" } { printf "\"%s\" ", $1 }'
# then add this
| sed -e s#\"\"##
# the capturing of the output to the var 'quoted' is the final step of code
)
# you **cannot** copy paste this whole block of code and expect it to work ;-)
I don't know what is supposed to be in $NAUTILUS_SCRIPT_SELECTED_FILE_PATHS, so it is hard to show you here. AND, that variable is not defined in any of the code you specify here, so you may only get a blank line when you echo its value. Be prepared to do some research on how that value get set AND what are the correct values.
Also I notice that your code is 'prefixed' as #!/bin/sh. If it is truly /bin/sh then command substitution like quoted=$(....) will not work and should generate an error message. Persumably your system is really using bash for /bin/sh. You can eliminate any possible confusion in the future (when changing to a system where /bin/sh = bourne shell), by changing the 'shebang' to #! /bin/bash.
I hope this helps.
I just discovered diff-ext thanks to this post, excellent!
The first try I did failed: by default diff-ext does not handle backup files (*~ and *.bak). To enable this, run:
$ diff-ext-setup
and in the Mime types pane, check application/x-trash.
Now you can compare a file and its backup.
Related
I am currently working with a vendor-provided software that is trying to handle sending attachment files to another script that will text-extract from the listed file. The script fails when we receive files from an outside source that contain spaces, as the vendor-supplied software does not surround the filename in quotes - meaning when the text-extraction script is run, it receives a filename that will split apart on the space and cause an error on the extractor script. The vendor-provided software is not editable by us.
This whole process is designed to be an automated transfer, so having this wrench that could be randomly thrown into the gears is an issue.
What we're trying to do, is handle the spaced name in our text extractor script, since that is the piece we have some control over. After a quick Google, it seems like changing the IFS value for the script would be the quick solution, but unfortunately, that script would take effect after the extensions have already mutilated the incoming data.
The script I'm using takes in a -e value, a -i value, and a -o value. These values are sent from the vendor supplied script, which I have no editing control over.
#!/bin/bash
usage() { echo "Usage: $0 -i input -o output -e encoding" 1>&2; exit 1; }
while getopts ":o:i:e:" o; do
case "${o}" in
i)
inputfile=${OPTARG}
;;
o)
outputfile=${OPTARG}
;;
e)
encoding=${OPTARG}
;;
*)
usage
;;
esac
done
shift $((OPTIND-1))
...
...
<Uses the inputfile, outputfile, and encoding variables>
I admit, there may be pieces to this I don't fully understand, and it could be a simple fix, but my end goal is to be able to extract -o, -i, and -e that all contain 1 value, regardless of the spaces within each section. I can handle quoting the script after I can extract the filename value
The script fragment that you have posted does not have any issues with spaces in the arguments.
The following, for example, does not need quoting (since it's an assignment):
inputfile=${OPTARG}
All other uses of $inputfile in the script should be double quoted.
What matters is how this script is called.
This would fail and would assign only hello to the variable inputfile:
$ ./script.sh -i hello world.txt
The string world.txt would prompt the getopts function to stop processing the command line and the script would continue with the shift (world.txt would be left in $1 afterwards).
The following would correctly assign the string hello world.txt to inputfile:
$ ./script.sh -i "hello world.txt"
as would
$ ./script.sh -i hello\ world.txt
The following script uses awk to split the arguments while including spaces in the file names. The arguments can be in any order. It does not handle multiple consecutive spaces in an argument, it collapses them to one.
#!/bin/bash
IFS=' '
str=$(printf "%s" "$*")
istr=$(echo "${str}" | awk 'BEGIN {FS="-i"} {print $2}' | awk 'BEGIN {FS="-o"} {print $1}' | awk 'BEGIN {FS="-e"} {print $1}')
estr=$(echo "${str}" | awk 'BEGIN {FS="-e"} {print $2}' | awk 'BEGIN {FS="-o"} {print $1}' | awk 'BEGIN {FS="-i"} {print $1}')
ostr=$(echo "${str}" | awk 'BEGIN {FS="-o"} {print $2}' | awk 'BEGIN {FS="-e"} {print $1}' | awk 'BEGIN {FS="-i"} {print $1}')
inputfile=""${istr}""
outputfile=""${ostr}""
encoding=""${estr}""
# call the jar
There was an issue when calling the jar where Java threw a MalformedUrlException on a filename with a space.
So after reading through the commentary, we decided that although it may not be the right answer for every scenario, the right answer for this specific scenario was to extract the pieces manually.
Because we are building this for a pre-built script passing to it, and we aren't updating that script any time soon, we can accept with certainty that this script will always receive a -i, -o, and -e flag, and there will be spaces between them, which causes all the pieces passed in to be stored in different variables in $*.
And we can assume that the text after a flag is the response to the flag, until another flag is referenced. This leaves us 3 scenarios:
The variable contains one of the flags
The variable contains the first piece of a parameter immediately after the flag
The variable contains part 2+ of a parameter, and the space in the name was interpreted as a split, and needs to be reinserted.
One of the other issues I kept running into was trying to get string literals to equate to variables in my IF statements. To resolve that issue, I pre-stored all relevant data in array variables, so I could test $variable == $otherVariable.
Although I don't expect it to change, we also handled what to do if the three flags appear in a different order than we anticipate (Our assumption was that they list as i,o,e... but we can't see excatly what is passed). The parameters are dumped into an array in the order they were read in, and a parallel array tracks whether the items in slots 0,1,2 relate to i,o,e.
The final result still has one flaw: if there is more than one consecutive space in the filename, the whitespace is trimmed before processing, and I can only account for one space. But saying as we processed over 4000 files before encountering one with a space, I find it unlikely with the naming conventions that we would encounter something with more than one space.
At that point, we would have to be stepping in for a rare intervention anyways.
Final code change is as follows:
#!/bin/bash
IFS='|'
position=-1
ioeArray=("" "" "")
previous=""
flagArr=("-i" "-o" "-e" " ")
ioePattern=(0 1 2)
#echo "for loop:"
for i in $*; do
#printf "%s\n" "$i"
if [ "$i" == "${flagArr[0]}" ] || [ "$i" == "${flagArr[1]}" ] || [ "$i" == "${flagArr[2]}" ]; then
((position += 1));
previous=$i;
case "$i" in
"${flagArr[0]}")
ioePattern[$position]=0
;;
"${flagArr[1]}")
ioePattern[$position]=1
;;
"${flagArr[2]}")
ioePattern[$position]=2
;;
esac
continue;
fi
if [[ $previous == "-"* ]]; then
ioeArray[$position]=${ioeArray[$position]}$i;
else
ioeArray[$position]=${ioeArray[$position]}" "$i;
fi
previous=$i;
done
echo "extracting (${ioeArray[${ioePattern[0]}]}) to (${ioeArray[${ioePattern[1]}]}) with (${ioeArray[${ioePattern[2]}]}) encoding."
inputfile=""${ioeArray[${ioePattern[0]}]}"";
outputfile=""${ioeArray[${ioePattern[1]}]}"";
encoding=""${ioeArray[${ioePattern[2]}]}"";
I am currently writing a awkscript which looks like this :
#!/usr/bin/awk -f
BEGIN {
print "Starting extracting data"
}
{
print $0
}
END {
print "End of file"
}
My script works fine like that on my computer but it is not portable... I'd like to do something like
#!/usr/bin/env awk -f
...
but the Debian shell does not accept several parameters in a single shebang. I get "awk -f" no such file or directory. Is there any workaround I could use or is it completely impossible ?
There is no easy way to do this cleanly; as mentioned in glenn jackman's comment, shebangs are implemented by execve(), which only accepts an interpreter and a single optional argument.
You'll need a workaround.
The comments to the question imply you don't want a wrapper script, but what if it's contained inside the final ~awk script?
#!/bin/sh
awk -f - "$#" << 'EOF'
#!/usr/bin/awk -f
BEGIN {
print "Starting extracting data"
}
{
print $0
}
END {
print "End of file"
}
This uses portable POSIX shell in its shebang and then immediately invokes awk with awk code from a heredoc on standard input (-f -), passing the further arguments and options ("$#") to awk as files. The heredoc is quoted (<< 'EOF') so things like $0 aren't interpreted by the POSIX shell. Since a line consisting solely of EOF is not present, the heredoc ends with the file.
(The second shebang is not read by anything. It's purely cosmetic for people who read the code. If you name this file with the .awk suffix, editors like vim will default their syntax highlighting to awk despite the contents of the first shebang.)
That code won't work for piping content into the file because the script is itself piped into awk. If you want to support piping, it needs to be a little uglier, using bash's input process substitution (<(…)):
#!/bin/bash
exec awk -f <(awk 'NR > 2' "$0") "$#"
#!/usr/bin/awk -f
BEGIN {
print "Starting extracting data"
}
{
print $0
}
END {
print "End of file"
}
This tells bash to execute awk on a named pipe created from that second awk command, which reads the full script (bash interprets $0 as the name of the file it is running) and prints lines 3 and higher. Again, the second shebang is purely cosmetic and is therefore just a comment.
There's really no reason to invoke awk in the shebang. Just invoke it via sh:
#!/bin/sh
exec awk '
BEGIN {
print "Starting extracting data"
}
{
print $0
}
END {
print "End of file"
}
' "$#"
I have a shell script of more than 1000 lines, i would like to check if all the commands used in the script are installed in my Linux operating system.
Is there any tool to get the list of Linux commands used in the shell script?
Or how can i write a small script which can do this for me?
The script runs successfully on the Ubuntu machine, it is invoked as a part of C++ application. we need to run the same on a device where a Linux with limited capability runs. I have identified manually, few commands which the script runs and not present on Device OS. before we try installing these commands i would like to check all other commands and install all at once.
Thanks in advance
I already tried this in the past and got to the conclusion that is very difficult to provide a solution which would work for all scripts. The reason is that each script with complex commands has a different approach in using the shells features.
In case of a simple linear script, it might be as easy as using debug mode.
For example: bash -x script.sh 2>&1 | grep ^+ | awk '{print $2}' | sort -u
In case the script has some decisions, then you might use the same approach an consider that for the "else" cases the commands would still be the same just with different arguments or would be something trivial (echo + exit).
In case of a complex script, I attempted to write a script that would just look for commands in the same place I would do it myself. The challenge is to create expressions that would help identify all used possibilities, I would say this is doable for about 80-90% of the script and the output should only be used as reference since it will contain invalid data (~20%).
Here is an example script that would parse itself using a very simple approach (separate commands on different lines, 1st word will be the command):
# 1. Eliminate all quoted text
# 2. Eliminate all comments
# 3. Replace all delimiters between commands with new lines ( ; | && || )
# 4. extract the command from 1st column and print it once
cat $0 \
| sed -e 's/\"/./g' -e "s/'[^']*'//g" -e 's/"[^"]*"//g' \
| sed -e "s/^[[:space:]]*#.*$//" -e "s/\([^\\]\)#[^\"']*$/\1/" \
| sed -e "s/&&/;/g" -e "s/||/;/g" | tr ";|" "\n\n" \
| awk '{print $1}' | sort -u
the output is:
.
/
/g.
awk
cat
sed
sort
tr
There are many more cases to consider (command substitutions, aliases etc.), 1, 2 and 3 are just beginning, but they would still cover 80% of most complex scripts.
The regular expressions used would need to be adjusted or extended to increase precision and special cases.
In conclusion if you really need something like this, then you can write a script as above, but don't trust the output until you verify it yourself.
Add export PATH='' to the second line of your script.
Execute your_script.sh 2>&1 > /dev/null | grep 'No such file or directory' | awk '{print $4;}' | grep -v '/' | sort | uniq | sed 's/.$//'.
If you have a fedora/redhat based system, bash has been patched with the --rpm-requires flag
--rpm-requires: Produce the list of files that are required for the shell script to run. This implies -n and is subject to the same limitations as compile time error checking checking; Command substitutions, Conditional expressions and eval builtin are not parsed so some dependencies may be missed.
So when you run the following:
$ bash --rpm-requires script.sh
executable(command1)
function(function1)
function(function2)
executable(command2)
function(function3)
There are some limitations here:
command and process substitutions and conditional expressions are not picked up. So the following are ignored:
$(command)
<(command)
>(command)
command1 && command2 || command3
commands as strings are not picked up. So the following line will be ignored
"/path/to/my/command"
commands that contain shell variables are not listed. This generally makes sense since
some might be the result of some script logic, but even the following is ignored
$HOME/bin/command
This point can however be bypassed by using envsubst and running it as
$ bash --rpm-requires <(<script envsubst)
However, if you use shellcheck, you most likely quoted this and it will still be ignored due to point 2
So if you want to use check if your scripts are all there, you can do something like:
while IFS='' read -r app; do
[ "${app%%(*}" == "executable" ] || continue
app="${app#*(}"; app="${app%)}";
if [ "$(type -t "${app}")" != "builtin" ] && \
! [ -x "$(command -v "${app}")" ]
then
echo "${app}: missing application"
fi
done < <(bash --rpm-requires <(<"$0" envsubst) )
If your script contains files that are sourced that might contain various functions and other important definitions, you might want to do something like
bash --rpm-requires <(cat source1 source2 ... <(<script.sh envsubst))
Based #czvtools’ answer, I added some extra checks to filter out bad values:
#!/usr/bin/fish
if test "$argv[1]" = ""
echo "Give path to command to be tested"
exit 1
end
set commands (cat $argv \
| sed -e 's/\"/./g' -e "s/'[^']*'//g" -e 's/"[^"]*"//g' \
| sed -e "s/^[[:space:]]*#.*\$//" -e "s/\([^\\]\)#[^\"']*\$/\1/" \
| sed -e "s/&&/;/g" -e "s/||/;/g" | tr ";|" "\n\n" \
| awk '{print $1}' | sort -u)
for command in $commands
if command -q -- $command
set -a resolved (realpath (which $command))
end
end
set resolved (string join0 $resolved | sort -z -u | string split0)
for command in $resolved
echo $command
end
I'm having trouble trying to wrap this command inside the other command.
# Target Command:
/bin/df / | awk END' { gsub(/\%/, ""); print $5} '
# What I want:
/bin/sh -c " [command above goes here]"
I'm running into a problem with the awk and all the quotes...
I've tried:
bin/sh -c "/bin/df / | awk END' { gsub(/\%/, "'"''"'"); print $5} '"
But the problem is that awk doesn't seem to print only column $5 in this instance.
How can I fix the above command awk command to print only column 5 (of the last line)?
PS: what I'm trying to do is to get the percentage of disk used (excluding the % sign). Since I'm calling it from a program that doesn't support pipes in an easy way, I'm using sh -c.
You can use:
/bin/sh -c "/bin/df | awk 'END{gsub(/%/, \"\", \$5); print \$5}'"
$ also needs to be escaped.
As per #Ed's comment below if $5 is not available in END block in some versions of awk then use:
/bin/sh -c "/bin/df | awk '{p=\$5} END{sub(/%/, \"\", p); print p}'"
You need to escape the " and the $:
/bin/sh -c "/bin/df / | awk END' { gsub(/\%/, \"\"); print \$5} '"
The reason why you did not need to escape the $ in your original line is: because it was guarded by ''. But when you quote the ticks (with the double quotes) they lose that protective property.
The " need to be escaped because they would terminate your outermost double quotes too early.
But it is most likely easier to put everything into a shell script and start this. This also allows you some more lines like setting up the PATH and IFS and maybe doing platform specific searching of (g)awk and so on.
BTW: and I would add -x nfs (Linux) or -t nonfs,nullfs (BSD) if possible. Monitoring scripts are known to kill a system if started repeatingly (while the NFS server is unabaulable). Of course this asumes, you dont want to monitor NFS.
You can let bash do the heavy lifting with exported functions:
myfunc() {
/bin/df / | awk 'END { gsub(/\%/, ""); print $5} '
}
export -f myfunc
bash -c "myfunc"
This way, you don't have to mess around with unreadable escaping.
I've a bash script that need to use some variables included in a separate file.
Normally, for including the entire file I would use source otherfile.sh in the main script.
In this case I need to use just a portion of this file. I can't use (include) the rest of the variables included in the rest of the file.
To filter the content of the config file (let's say just as example from the tag "# start" to "# end") I use awk, but I can't redirect the output to the soruce command.
Below my code:
awk ' /'"# start"'/ {flag=1;next} /'"# end"'/{flag=0} flag { print }' config.sh
Do you know any way to redirect the output of this command into source? Is there any other way to include the output in my bash script at run-time?
By the way, I wouldn't like to create temporaty files (it seems me too dirty...)..and I've tried something found on this site, but for some reasons it doesn't work. Below there's the solution I've found.
awk ' /'"# start"'/ {flag=1;next} /'"# end"'/{flag=0} flag { print }' config.sh | source /dev/stdin
Thank you,
Luca
source can read from a process substitution in bash:
source <( awk ' ... ' config.sh )
sed allows a simpler way to get the subset of lines:
sed -n '/#start/,/#end/p' config.sh
UPDATE: It appears that this may only work in bash 4 or later.
A correct way of doing it provided by a friend today. Here's the code:
source /dev/stdin <<EOF
$(awk ' /'"# 10.216.33.133 - start"'/ {flag=1;next} /'"# 10.216.33.133 - end"'/{flag=0} flag { print }' testbed.sh)
EOF
Working perfectly...thanks Andrea! :) (and of course everyone tried to answer)