I use a combination of stat and touch for getting/setting timestamps on files and repertories. But I need different set-ups if on mac os x or GNU/Linux:
touch on mac os x does not know the -d option described there
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/touch.html
which allows things like
touch -d "2007-11-12 10:15:30.002Z" ajosey
I am seemingly constrained to -t [[CC]YY]MMDDhhmm[.SS].
stat also differs, for example on a Linux account of mine, it does not recognize the -t format from the stat on mac os x.
Thus on the Linux I currently do something like
stat --format 'touch -d "%y" "%n"' index.html
to create a command line of the type
touch -d "2015-04-08 00:38:51.940365000 +0200" "index.html"
whereas on the mac os x I have
stat -f "touch -t %Sm \"%N\"" -t %Y%m%d%H%M.%S index.html
which gives me something (this is not the same index.html as prior) like:
touch -t 201503281339.42 "index.html"
How could handle this in a unified way ? Perhaps with some sed in between ?
I need to produce a sequence of touch commands in a format working on both platforms. The creation of this sequence must work on both platforms.
I am open to other scripting than bash, with the constraint that on the Linux side I am with a system with no admin rights. perl there is This is perl, v5.10.1 (*) built for x86_64-linux-thread-multi.
Short of a better method, I will temporarily adopt the following, which is based on these observations:
touch -t works the same on my mac os x and the Linux I have access too.
On the Linux side, I can use date -d to transform a date as produced by stat -c %y to the YYYYMMDDHHMM.SS format I can use on input to touch -t, and on the Mac OS X side I can use directly stat with suitable options for this result.
For batch processing of files in a repertory, where I was using stat with * shell expansion, I can replace that with a for shell loop.
Putting these things together I end with the following script:
#!/bin/sh
case `uname -s` in
"Linux" )
MYDATEFORTOUCH() {
date -d"$(stat -c %y "$1")" +%Y%m%d%H%M.%S
}
;;
"Darwin" )
MYDATEFORTOUCH() {
stat -f %Sm -t %Y%m%d%H%M.%S "$1"
}
;;
* )
MYDATEFORTOUCH() {
197001010000.00
}
;;
esac
echo "#!/bin/sh" > fichierTEMPA
for file in *
do echo "touch -ch -t $(MYDATEFORTOUCH "$file") \"$file\"" >> fichierTEMPA
done
Executing this in a repertory produces a file (with silly name here fichierTEMPA) which is a series of touch -t commands. The -h is for not following symbolic links, on mac os x, it implies the -c which is to not create a file which didn't exist, I am not sure if -c is also implied by -h on GNU/Linux.
Install the GNU Coreutils on your Mac and you can stop bothering about incompatibilities. It is explained here how to do it.
Related
I have a bash script that works fine in linux, but when I run it on my Mac terminal it fails, as the options for the splitcommand are slightly different in Mac terminal. My script is:
## Merge and half final two segments
last_file=`ls temp_filt.snplist_* | tail -n 1`
penultimate_file=`ls temp_filt.snplist_* | tail -n 2 | head -1`
cat $penultimate_file $last_file > temp && mv temp $penultimate_file
split -n l/2 $penultimate_file && mv xaa $penultimate_file; mv xab $last_file
The script fails at the final line, since the -n l/2 doesn't exist in tcsh (default shel environment in Mac OS 10.x.x). I was wondering what is the equivalent script in tcsh.
Is there a generic way to run linux script in Mac OS terminal, without the need to change the script?
It's not the MacOS terminal that's doing the split. It's a programm called split. MacOS is built on the FreeBSD userland tools, which behave differently from the GNU utils.
There are two options:
Install the FreeBSD tools on your Linux boxes to make them compatible with FreeBSD.
Install the GNU utils on your MacOS machine. If you have brew you can do this with brew install coreutils
An option is to use the language built-ins and limit external commands
Note the script contains several flaws: ls is useless and parsing ls output is not safe
array=(temp_filt.snplist_*)
last_file=${array[ -1]}
penultimate_file=${array[ -2]}
If the files are big bash read built-in will be very slow.
A simple solution in this case using cat, wc, head and tail which are compatible between systems. Note when passed in a command variables must be double quoted to avoid word splitting.
cat "$penultimate_file" "$last_file" > temp || exit 1
nb_lines=$(wc -l < temp)
((half_nb_lines=nb_lines/2))
head "-$half_nb_lines" temp > "$penultimate_file" || exit 1
tail "+$((half_nb_lines+1))" temp > "$last_file" || exit 1
rm temp
Note in the last line
command1 && command2 ; command3
the command3 is executed whatever the first exit status, { ; } may be used for grouping commands
command1 && { command2 ; command3; }
I have been monitoring the performance of my Linux server with ioping (had some performance degradation last year). For this purpose I created a simple script:
echo $(date) | tee -a ../sb-output.log | tee -a ../iotest.txt
./ioping -c 10 . 2>&1 | tee -a ../sb-output.log | grep "requests completed in\|ioping" | grep -v "ioping statistics" | sed "s/^/IOPing I\/O\: /" | tee -a ../iotest.txt
./ioping -RD . 2>&1 | tee -a ../sb-output.log | grep "requests completed in\|ioping" | grep -v "ioping statistics" | sed "s/^/IOPing seek rate\: /" | tee -a ../iotest.txt
etc
The script calls ioping in the folder /home/bench/ioping-0.6. Then it saves the output in readable form in /home/bench/iotest.txt. It also adds the date so I can compare points in time.
Unfortunately I am no experienced programmer and this version of the script only works if you first enter the right directory (/home/bench/ioping-0.6).
I would like to call this script from anywhere. For example by calling
sh /home/bench/ioping.sh
Googling this and reading about path variables was a bit over my head. I kept up ending up with different version of
line 3: ./ioping: No such file or directory
Any thoughts on how to upgrade my scripts so that it works anywhere?
The trick is the shell's $0 variable. This is set to the path of the script.
#!/bin/sh
set -x
cd $(dirname $0)
pwd
cd ${0%/*}
pwd
If dirname isn't available for some reason, like some limited busybox distributions, you can try using shell parameter expansion tricks like the second one in my example.
Isn't it obvious? ioping is not in . so you can't use ./ioping.
Easiest solution is to set PATH to include the directory where ioping is. perhaps more robust - figure out the path to $0 and use that path as the location for ioping (assing your script sits next to ioping).
If iopinf itself depend on being ruin in a certain directory, you might have to make your script cd to the ioping directory before running.
On Cygwin, the cygpath application translates between Windows and Unix-style paths.
Consider the following examples:
$ cygpath -u "c:/"
/cygdrive/c
$ cygpath -u "c:/cygwin64/bin"
/usr/bin
Is there any way to get /cygdrive/c/cygwin64/bin from the second command?
I need this because sometimes Cygwin gets confused about where its root is, so I want an absolute path in order to be clear.
No, Cygwin's cygpath doesn't support this. The best you can do is manually fix it up with your own conversion tool; something along the lines of:
#!/usr/bin/env bash
if [[ "$1" == -z ]]; then
# Invoked with -z, so skip normal cygpath processing and convert the path
# here.
#
# The sed command replaces "c:" with "/cygdrive/c", and switches any
# back slashes to forward slashes.
shift
printf "%s\n" "$*" | sed -r 's!(.):([\\\/].*)$!/cygdrive/\1\2!;s!\\!/!g'
else
# Not invoked with -z, so just call cygpath with the arguments this script
# was called with.
exec cygpath "$#"
fi
If you store the above script as mycygpath.sh then it will behave exactly as cygpath does unless you give it the -z argument, in which case it will simply convert n:/ to /cygdrive/n/:
$ ./mycygpath.sh -u "c:/"
/cygdrive/c
$ ./mycygpath.sh -u "c:/cygwin64/bin"
/usr/bin
$ ./mycygpath.sh -z "c:/cygwin64/bin"
/cygdrive/c/cygwin64/bin
Of course, there's the obvious question of why "Cygwin gets confused about where it root is"; that's not something that should happen at all and implies there's something wrong with your Cygwin setup. But that's not the question you asked and you've not given enough detail to be able to start making suggestions.
How to get all process ids (pid) (similar to: $ ps aux) but without using ps.
One example of when this would be used is when developing a dotnet 5 application to run on a docker host. The dotnet runtime image is a very cut-down Linux image, with bash, but without ps. When diagnosing an issue with the application, it's sometimes useful to see what processes are running and if separate processes have been spawned correctly. ps is unavailable on this image. Is there an alternative?
On Linux, all running process have "metadata" stored in the /proc filesystem.
All running process ids:
shopt -s extglob # assuming bash
(cd /proc && echo +([0-9]))
Further to the comment by #FelixJongleur42, the command
ls -l /proc/*/exe
yields a parseable output with additional info such as the process user, start time and command.
This one-liner will give you the pid and the cmd with args:
for prc in /proc/*/cmdline; { (printf "$prc "; cat -A "$prc") | sed 's/\^#/ /g;s|/proc/||;s|/cmdline||'; echo; }
Based on Ivan's example with some filtering:
for prc in /proc/*/cmdline; {
(printf "$prc "; cat -A "$prc") | sed 's/\^#/ /g;s|/proc/||;s|/cmdline||' | grep java ; echo -n;
}
I have an ubuntu machine with default shell set to bash and both ways to the binary in $PATH:
$ which bash
/bin/bash
$ which sh
/bin/sh
$ ll /bin/sh
lrwxrwxrwx 1 root root 4 Mar 6 2013 /bin/sh -> bash*
But when I try to call a script that uses the inline file descriptor (that only bash can handle, but not sh) both calls behave differently:
$ . ./inline-pipe
reached
$ bash ./inline-pipe
reached
$ sh ./inline-pipe
./inline-pipe: line 6: syntax error near unexpected token `<'
./inline-pipe: line 6: `done < <(echo "reached")'
The example-script I am referring to looks like that
#!/bin/sh
while read line; do
if [[ "$line" == "reached" ]]; then echo "reached"; fi
done < <(echo "reached")
the real one is a little bit longer:
#!/bin/sh
declare -A elements
while read line
do
for ele in $(echo $line | grep -o "[a-z]*:[^ ]*")
do
id=$(echo $ele | cut -d ":" -f 1)
elements["$id"]=$(echo $ele | cut -d ":" -f 2)
done
done < <(adb devices -l)
echo ${elements[*]}
When bash is invoked as sh, it (mostly) restricts itself to features found in the POSIX standard. Process substitution is not one of those features, hence the error.
Theoretically, it is a feature of bash: if you call as "sh", it by default switches off all of its features. And the root shell is by default "/bin/sh".
Its primary goal is the security. Secondary is the produce some level of compatibility between some shells of the system, because it enables the system scripts to run in alternate (faster? more secure?) environment.
This is the theory.
Practically goes this so, that there are always people in a development team, who want to reduce and eliminate everything with various arguments (security, simplicity, safety, stability - but these arguments are going somehow always to the direction of the removal, deletion, destroying).
This is because the bash in debian doesn't have network sockets, this is because debian wasn't able in 20 years to normally integrate the best compressors (bz2, xz) - and this is because the root shell is by default so primitive, as of the PDP11 of the eighties.
I believe sh on ubuntu is actually dash which is smaller than bash with fewer features.