Unchanged shell script arguments propagation - linux

The primary problem running either 32 or 64 bit binaries depending on the current platform. I have predefined set of platforms (precisely 2: 32- and 64-bit Debian, both running 64-bit kernel, but all user-space is 32- or 64-bit respectively) and >100 platform-specific binaries. All these binaries are called from my wrapper script, which is identical for all of them.
I have 2 directories: bin and tmp. In bin there are links to wrapper script and script itself. In tmp there are 2 directories: 32 and 64. In these directories executables and libraries are placed.
I've wrote the following script that relies on exported by Bash environment variable HOSTTYPE:
#!/bin/bash
executable=`basename $0`
tmp_path=$(readlink -f $(dirname $(readlink -f $0))/../tmp)
case "$HOSTTYPE" in
i?86 )
export LD_LIBRARY_PATH="$tmp_path/32:$LD_LIBRARY_PATH"
exec "$tmp_path/32/$executable" $#
;;
x86_64 )
export LD_LIBRARY_PATH="$tmp_path/64:$LD_LIBRARY_PATH"
exec "$tmp_path/64/$executable" $#
;;
* )
echo "Unknown host type: '$HOSTTYPE'"
;;
esac
Most of the time it works as expected but it gets broken when the argument that should be quoted is passed. For example one calls
./some_binary --argument="\t" filename
where \t is a tab character. The script receives the pre-interpreted arguments list as:
./some_binary --argument=\t filename
Then it is reinterpreted once again in exec as
../tmp/32/some_binary --argument= filename
The tab characted get lost. How can I prevent this? Or should I solve my primary problem in a drastically different way?

Related

How to use Sass with NetBeans on Linux / macOS

I used to be able to install and use Sass with NetBeans 8 as described in the top answer on How to use SASS with Netbeans 8.0.1
Now, with the current version of Sass (1.14.1), installing is different. Basically just download and untar. That's done and I've pointed NetBeans to the correct location. But this current version of Sass won't run correctly from NetBeans:
"/opt/dart-sass/sass" "--cache-location"
"/home/jasper/.cache/netbeans/8.2/sass-compiler"
"path_to_my.scss" "path_to_my.css"
Could not find an option named "cache-location".
This error is also covered by Sass output error in Netbeans 8.2 where they are using Windows.
I tried to add the cache location parameter (similar to the solution for Windows) to this line in the sass file:
exec "$path/src/dart" --no-preview-dart-2 "-Dversion=1.14.1" "$path/src/sass.dart.snapshot" "$#"
but I could not get it working (same error keeps appearing).
Anybody any ideas on how to get Sass 1.14.1 working from NetBeans 8.2 on Linux (Ubuntu)?
The issue is that --cache-location is no longer supported and should be removed. All of the original parameters are used by "$#". To remove the first two parameters, you should be able to use "${#:3}" (see Process all arguments except the first one (in a bash script)), but somehow that resulted into a "Bad substitution" error for me. So I opted to use shift 2 to remove them:
#!/bin/sh
# Copyright 2016 Google Inc. Use of this source code is governed by an MIT-style
# license that can be found in the LICENSE file or at
# https://opensource.org/licenses/MIT.
# This script drives the standalone Sass package, which bundles together a Dart
# executable and a snapshot of Sass. It can be created with `pub run grinder
# package`.
follow_links() {
file="$1"
while [ -h "$file" ]; do
# On Mac OS, readlink -f doesn't work.
file="$(readlink "$file")"
done
echo "$file"
}
# Unlike $0, $BASH_SOURCE points to the absolute path of this file.
path=`dirname "$(follow_links "$0")"`
shift 2
exec "$path/src/dart" --no-preview-dart-2 "-Dversion=1.14.1" "$path/src/sass.dart.snapshot" "${#}"
Make sure to keep the original file and create a copy to only be used with NetBeans and make the change there.
macOS (Home Brew)
If you are looking for the Dart Sass install location (after installing it with Home Brew), it is located here:
/usr/local/Cellar/sass/{version}/bin
macOS (node.js)
When using node.js, you will run into the "env: node: No such file or directory" issue.
To work around that I created (make sure you make it executable (chmod a+x)):
/usr/local/lib/node_modules/sass/sass_nb.sh
and added:
#!/bin/zsh
export PATH="$PATH:"/usr/local/bin/
shift 3
sass ${#}
NetBeans 11+
On NetBeans 11 and 12 I had to use shift 3 instead of shift 2.
My response is based heavily on Jasper de Vries'one:
It seems that Netbeans simply adds some additional parameters that are no longer supported by sass compiler.
In my case the complete command issued by Netbeans was:
"/home/alex/tools/dart-sass/sass" "--cache-location" "/home/alex/snap/netbeans/common/cache/12.0/sass-compiler" "--debug-info" "/home/alex/projects/alexgheorghiu.com/web/aaa.scss" "/home/alex/projects/alexgheorghiu.com/web/aaa.css"
So the first 3 parameters
"--cache-location" "/home/alex/snap/netbeans/common/cache/12.0/sass-compiler" "--debug-info"
must be "deleted" or ignored.
So you need to either alter the sass file or make a copy of it (safest way)
and add
shift 3
instruction.
So if you start from original version like:
#!/bin/sh
# This script drives the standalone dart-sass package, which bundles together a
# Dart executable and a snapshot of dart-sass.
follow_links() {
file="$1"
while [ -h "$file" ]; do
# On Mac OS, readlink -f doesn't work.
file="$(readlink "$file")"
done
echo "$file"
}
# Unlike $0, $BASH_SOURCE points to the absolute path of this file.
path=`dirname "$(follow_links "$0")"`
exec "$path/src/dart" "$path/src/sass.snapshot" "$#"
You need to end up with something like:
#!/bin/sh
# This script drives the standalone dart-sass package, which bundles together a
# Dart executable and a snapshot of dart-sass.
follow_links() {
file="$1"
while [ -h "$file" ]; do
# On Mac OS, readlink -f doesn't work.
file="$(readlink "$file")"
done
echo "$file"
}
# Unlike $0, $BASH_SOURCE points to the absolute path of this file.
path=`dirname "$(follow_links "$0")"`
shift 3
exec "$path/src/dart" "$path/src/sass.snapshot" "$#"
An interesting aspect is that this bug is known by Netbeans developers (See: Could not find an option named "cache-location") but I was not able to achieve that because under my Xubuntu 18 the Netbeans is a "snap" and therefore it's netbeans.conf file is read only.
But in case you CAN modify that file it might be a cleaner solution.

How to check the OS with automake

I have a project that uses automake to create the configure and all related files (I'm using autoreconf command to make all this stuff). So, I'm trying to set some conditional files to compile when the project is compiling for macOS (OS X), Windows or Linux. But it fails with the following:
$ autoreconf -i ..
src/Makefile.am:30: error: LINUX does not appear in AM_CONDITIONAL
autoreconf: automake failed with exit status: 1
And the part containing the error in that Makefile.am is the following:
if OSX
butt_SOURCES += CurrentTrackOSX.h CurrentTrackOSX.m
endif
if LINUX
butt_SOURCES += currentTrack.h currentTrackLinux.cpp
endif
if WINDOWS
butt_SOURCES += currentTrack.h currentTrack.cpp
endif
My question is, how can I check if the OS is Linux? And if it's possible, is there a better way to check the OS in automake?
You can detect it directly in the Makefile, or define the conditionals in the configure source file (probably configure.ac), since you are using autoreconf:
# AC_CANONICAL_HOST is needed to access the 'host_os' variable
AC_CANONICAL_HOST
build_linux=no
build_windows=no
build_mac=no
# Detect the target system
case "${host_os}" in
linux*)
build_linux=yes
;;
cygwin*|mingw*)
build_windows=yes
;;
darwin*)
build_mac=yes
;;
*)
AC_MSG_ERROR(["OS $host_os is not supported"])
;;
esac
# Pass the conditionals to automake
AM_CONDITIONAL([LINUX], [test "$build_linux" = "yes"])
AM_CONDITIONAL([WINDOWS], [test "$build_windows" = "yes"])
AM_CONDITIONAL([OSX], [test "$build_mac" = "yes"])
Note: host_os refers to the target system, so if you are cross-compiling it sets the OS conditional of the system you are compiling to.
Have you tried using the lsb_release command to determine the type and version of operating system ?

Making Sublime Text 2 command on linux behave as it does on MacOS X

There are many questions asking about accessing the Sublime Text 2 editor from the command line. The responses, in summary, are to make a symlink, alias or simple shell script to run the appropriate sublime_text command. I can do that. What I want is to make the linux version behave like the MacOS version.
On MacOS, I have the following:
ln -s /Applications/Sublime\ Text\ 2.app/Contents/SharedSupport/bin/subl ~/bin/subl
Then in my .zshrc:
alias subl="$HOME/bin/subl -n"
export EDITOR="$HOME/bin/subl -n -w"
This does two things. It gives me a subl command that opens any files given on the command line in a new window. The subl command does not block the terminal. It also sets up my editor to open sublime text to edit the arguments, but this time it does block. In particular, $EDITOR blocks until its arguments are closed. It does not block on unrelated sublime text windows.
I can achieve a similar effect on linux with the following:
In ~/bin/subl:
#! /bin/zsh
$HOME/Sublime\ Text\ 2/sublime_text -n $# &
and then in ~/bin/subl_wait: (think mate_wait for TextMate users)
#! /bin/zsh
exec $HOME/Sublime\ Text\ 2/sublime_text -n -w $#
I can then set EDITOR to subl_wait, and things almost work. subl opens files for editing and doesn't block. subl_wait opens files for editing and does block.
The problem is that subl_wait is waiting until all open files are closed, not just its arguments.
Is it possible to get this working perfectly?
Looks like I've found the issue. (Thanks to this post: http://www.sublimetext.com/forum/viewtopic.php?f=2&t=7003 )
Basic point: sublime behaves differently depending upon whether an instance is already running!
If an instance is already running then sublime on linux behaves similarly to MacOS. If no instance is running then the terminal blocks until you exit sublime.
With that in mind, we just need to modify the scripts to make sure sublime is running:
in ~/bin/subl_start:
#! /bin/zsh
if [ ! "$(pidof sublime_text)" ] ; then
# start sublime text main instance
# echo "Starting Sublime Text 2"
$HOME/Sublime\ Text\ 2/sublime_text &
sleep 1 # needed to avoid a race condition
fi
in ~/bin/subl:
#! /bin/zsh
. $HOME/bin/subl_start
exec $HOME/Sublime\ Text\ 2/sublime_text -n $#
in ~/bin/subl_wait:
#! /bin/zsh
. $HOME/bin/subl_start
exec $HOME/Sublime\ Text\ 2/sublime_text -n -w $#
Note that I've used the -n flags everywhere. This might not be your cup of tea. If you are using -n then you possibly also want to look at your close_windows_when_empty setting.
Inspired by the OP's answer, I've created a bash wrapper script for Sublime Text that incorporates all your findings and runs on both OSX and Linux.
Its purpose is threefold:
provide a unified subl CLI that works like ST's own subl on OSX: invoke ST without blocking, unless waiting is explicitly requested.
encapsulate a workaround for the waiting-related bug on Linux.
when saved or symlinked to as sublwait, provide a sublwait CLI that automatically applies the --wait and --new-window options so as to make it suitable for use with $EDITOR (note that some programs, e.g. npm, require the $EDITOR to contain the name of an executable only - executables + options are not supported); also makes sure that at least one file is specified.
The only open question is whether the OP's approach to avoiding the race condition - sleep 1 - is robust enough.
Update:
Note that subl on OSX is by default NOT placed in the $PATH - you normally have to do that manually. If you haven't done so, the script will now locate subl inside ST's application bundle; (it tries app names in the following sequence: 'Sublime Text', 'Sublime Text 2', 'Sublime Text 3', first in /Applications, then in ~/Applications.)
Here's the output from running the script with -h:
Multi-platform (OSX, Linux) wrapper script for invocation of Sublime Text (ST)
from the command line.
Linux:
Works around undesired blocking of the shell (unless requested)
and a bug when waiting for specific files to be edited.
Both platforms:
When invoked as `sublwait`, automatically applies the
--wait --new-window
options to make it suitable for use with $EDITOR.
Therefore, you can to the following:
- Name this script `subl` for a CLI that supports ALL options.
(On OSX, this will simply defer to the `subl` CLI that came with ST.)
- Place the script in a directory in your $PATH.
- In the same directory, create a symlink to the `subl` script named
`sublwait`:
ln -s subl sublwait
and, if desired, add
export EDITOR=sublwait
to your shell profile.
Note that if you only use OSX, you can make do with ST's own subl and just save this script directly as sublwait.
Script source:
#!/usr/bin/env bash
# Multi-platform (OSX, Linux) wrapper script for invocation of Sublime Text (ST)
# from the command line. Invoke with -h for details.
[[ $1 == '-h' || $1 == '--help' ]] && showHelpOnly=1 || showHelpOnly=0
[[ $(basename "$BASH_SOURCE") == 'sublwait' ]] && invokedAsSublWait=1 || invokedAsSublWait=0
[[ $(uname) == 'Darwin' ]] && isOsX=1 || isOsX=0
# Find the platform-appropriate ST executable.
if (( isOsX )); then # OSX: ST comes with a bona-fide CLI, `subl`.
# First, try to find the `subl` CLI in the $PATH.
# Note: This CLI is NOT there by default; it must be created by symlinking it from
# its location inside the ST app bundle.
# Find the `subl` executable, ignoring this script, if named subl' as well, or a
# script by that name in the same folder as this one (when invoked via symlink 'sublwait').
stExe=$(which -a subl | fgrep -v -x "$(dirname "$BASH_SOURCE")/subl" | head -1)
# If not already in the path, look for it inside the application bundle. Try several locations and versions.
if [[ -z $stExe ]]; then
for p in {,$HOME}"/Applications/Sublime Text"{,' 2',' 3'}".app/Contents/SharedSupport/bin/subl"; do
[[ -f $p ]] && { stExe=$p; break; }
done
fi
[[ -x $stExe ]] || { echo "ERROR: Sublime Text CLI 'subl' not found." 1>&2; exit 1; }
else # Linux: `sublime_text` is the only executable - the app itself.
stExe='sublime_text'
which "$stExe" >/dev/null || { echo "ERROR: Sublime Text executable '$stExe' not found." 1>&2; exit 1; }
fi
# Show command-line help, if requested.
# Add preamble before printing ST's own help.
# Note that we needn't worry about blocking the
# shell in this case - ST just outputs synchronously
# to stdout, then exits.
if (( showHelpOnly )); then
bugDescr=$(
cat <<EOF
works around a bug on Linux (as of v2.0.2), where Sublime Text,
if it is not already running, mistakenly blocks until it is exited altogether.
EOF
)
if (( invokedAsSublWait )); then
# We provide variant-specific help here.
cat <<EOF
Wrapper script for Sublime Text suitable for use with the \$EDITOR variable.
Opens the specified files for editing in a new window and blocks the
invoking program (shell) until they are closed.
In other words: the --wait and --new-window options are automatically
applied.
Aside from encapsulating this functionality without the need for options
- helpful for tools that require \$EDITOR to be an executable name only -
$bugDescr
Usage: sublwait file ...
EOF
# Note: Adding other options doesn't make sense in this scenario
# (as of v2.0.2), so we do NOT show ST's own help here.
else
cat <<EOF
Multi-platform (OSX, Linux) wrapper script for invocation of
Sublime Text (ST) from the command line.
Linux:
Works around undesired blocking of the shell (unless requested)
and a bug when waiting for specific files to be edited.
Both platforms:
When invoked as \`sublwait\`, automatically applies the
--wait --new-window
options to make it suitable for use with \$EDITOR.
Therefore, you can to the following:
- Name this script \`subl\` for a CLI that supports ALL options.
(On OSX, this will simply defer to the \`subl\` CLI that came with ST.)
- Place the script in a directory in your \$PATH.
- In the same directory, create a symlink to the \`subl\` script named
\`sublwait\`:
ln -s subl sublwait
and, if desired, add
export EDITOR=sublwait
to your shell profile.
Sublime Text's own help:
------------------------
EOF
# Finally, print ST's own help and exit.
exec "$stExe" "$#"
fi
exit 0
fi
# Invoked as `sublwait`? -> automatically apply --wait --new-window options.
if (( invokedAsSublWait )); then
# Validate parameters.
# - We expect NO options - to keep things simple and predictable, we do NOT allow
# specifying additional options (beyond the implied ones).
# - We need at least 1 file argument.
# - As a courtesy, we ensure that no *directories* are among the arguments - ST doesn't support
# that properly (always waits for ST exit altogether); beyond that, however, we leave input
# validation to ST.
if [[ "$1" =~ ^-[[:alnum:]]+$ || "$1" =~ ^--[[:alnum:]]+[[:alnum:]-]+$ ]]; then # options specified?
{ echo "ERROR: Unexpected option specified: '$1'. Use -h for help." 1>&2; exit 1; }
elif (( $# == 0 )); then # no file arguments?
{ echo "ERROR: Missing file argument. Use -h for help." 1>&2; exit 1; }
else # any directories among the arguments?
# Note: We do NOT check for file existence - files could be created on demand.
# (Things can still go wrong - e.g., /nosuchdir/mynewfile - and ST doesn't
# handle that gracefully, but we don't want to do too much here.)
for f in "$#"; do
[[ ! -d "$f" ]] || { echo "ERROR: Specifying directories is not supported: '$f'. Use -h for help." 1>&2; exit 1; }
done
fi
# Prepend the implied options.
set -- '--wait' '--new-window' "$#"
fi
# Finally, invoke ST:
if (( isOsX )); then # OSX
# `subl` on OSX handles all cases correctly; simply pass parameters through.
exec "$stExe" "$#"
else # LINUX: `sublime_text`, the app executable itself, does have a CLI, but it blocks the shell.
# Determine if the wait option was specified.
mustWait=0
if (( invokedAsSublWait )); then
mustWait=1
else
# Look for the wait option in the parameters to pass through.
for p in "$#"; do
[[ $p != -* ]] && break # past options
[[ $p == '--wait' || $p =~ ^-[[:alnum:]]*w[[:alnum:]]*$ ]] && { mustWait=1; break; }
done
fi
if (( mustWait )); then # Invoke in wait-for-specified-files-to-close mode.
# Quirk on Linux:
# If sublime_text isn't running yet, we must start it explicitly first.
# Otherwise, --wait will wait for ST *as a whole* to be closed before returning,
# which is undesired.
# Thanks, http://stackoverflow.com/questions/14598261/making-sublime-text-2-command-on-linux-behave-as-it-does-on-macos-x
if ! pidof "$stExe" 1>/dev/null; then
# Launch as BACKGROUND task to avoid blocking.
# (Sadly, the `--background` option - designed not to activate the Sublime Text window
# on launching - doesn't actually work on Linux (as of ST v2.0.2 on Ubuntu 12.04).)
("$stExe" --background &)
# !! We MUST give ST some time to start up, otherwise the 2nd invocation below will be ignored.
# ?? Does a fixed sleep time of 1 second work reliably?
sleep 1
fi
# Invoke in blocking manner, as requested.
exec "$stExe" "$#"
else # Ensure invocation in NON-blocking manner.
if ! pidof "$stExe" 1>/dev/null; then # ST isn't running.
# If ST isn't running, invoking it *always* blocks.
# Therefore, we launch it as a background taks.
# Invocation via a subshell (parentheses) suppresses display of the
# background-task 'housekeeping' info.
("$stExe" "$#" &)
else # ST is already running, we can safely invoke it directly without fear of blocking.
exec "$stExe" "$#"
fi
fi
fi
On Ubuntu Gnu/Linux 13.04 64-bit:
I just keep subl running pretty much all the time. So my git config has:
core.editor=/usr/bin/subl -n -w
And that's all I need. I save the git commit file with ctrl-s, close the window with ctrl-w and I'm done. But I then have to really close the window by hitting the X in the upper corner... 96% perfect.

Meaning of UTS in UTS_RELEASE

UTS_RELEASE defines the kernel version in Linux. It's defined in generated/utsrelease.h, which is created by the main Makefile like so:
# KERNELRELEASE can change from a few different places, meaning version.h
# needs to be updated, so this check is forced on all builds
uts_len := 64
define filechk_utsrelease.h
if [ `echo -n "$(KERNELRELEASE)" | wc -c ` -gt $(uts_len) ]; then \
echo '"$(KERNELRELEASE)" exceeds $(uts_len) characters' >&2; \
exit 1; \
fi; \
(echo \#define UTS_RELEASE \"$(KERNELRELEASE)\";)
endef
I was wondering what UTS stands for, here?
I will do a bet : it comes from unix history age.
Unix Time Sharing
http://en.wikipedia.org/wiki/Time-sharing
( with another link to give more weight to my guess : http://www.linuxmisc.com/9-unix-programmer/515225795f89ebf5.htm )
Additionally if you search for UTS on Wikipedia you'll find this as evidence too:
UTS is a three-letter abbreviation which may describe:
Time-sharing, known as Unix Time-sharing System (UTS) when abbreviated in the source code of many Unix-like operating systems
Maybe https://lwn.net/Articles/531114/ and https://lwn.net/Articles/179345/ are the right(tm) answer :-)
For example, if KERNELRELEASE value is:
3.18.31-g18e453b
Also in the file
*utsrelease.h
Will be:
#define UTS_RELEASE "3.18.31-g18e453b"
In Android, it goes here:
Settings > About Phone > Kernel Version

LD_LIBRARY_PATH : how to find a shared object

I have a shared object ( libxyz.so ). Given LD_LIBRARY_PATH, how can find the exact location of this shared object? If i had a binary that depends on this lib, i would have used ldd on that.
Here is the reason why i ask:
I have a cgi script which works when using LD_LIBRARY_PATH set to say VALUE1. It does not work when the path is set to VALUE2. I would like to find the exact location of the library as specified by the path in VALUE1 ( Note that VALUE1 has almost 20+ different locations )
Platform: Linux
Put this in a file:
#!/bin/bash
IFS=:
for p in ${LD_LIBRARY_PATH}; do
if [ -e ${p}/libxyz.so ]; then
echo ${p}
fi
done
and run it.
You can also use ldd. To do this, you would:
Set LD_LIBRARY_PATH to the value when it works (i.e. export LD_LIBRARY_PATH=VALUE1)
Run ldd /path/to/prog | grep libxyz.so
put a sleep(30); in your cgi, launch it from a browser, then look into /proc/$(pidof mycgi)/maps for actual libs used by your program.

Resources