I'm trying to create a shell script called sv that will prepend to a file, but the solution I'm using seems to only be good for one use, and them the temporary file is deleted. Is there a way I can make a shell script that will be go to use over and over?
Here's the questions:
"Suppose we wish to maintain a list of all the dates when we logged on to our UNIX system. It would be easy to do this by adding the following to the
.login file:
date >> logdates
Unfortunately, the latest date comes at the end of file logdates. I want it at the front; that is, the file should contain login dates from latest to earliest. Write a C shell script sv that will be used as follows:
date | sv logdates
(This way, the script is quite general, and I can use it for other cases when I want to add things to the front of a file.)"
Here's the script I've come up with:
"#!/bin/sh
cat - logdates /tmp/out && mv /tmp/out logdates"
This will work once, when I try again the system tells me that /tmp/out doesn't exist.
Does anyone have any suggestions?
Thank you!
Using sponge utility:
#!/bin/sh
cat - "$1" | sponge "$1"
Your shell script sv could contain the following, where newline is your data read from the | and $1 is the filename passed to sv:
#!/bin/sh
read newline
(echo "$newline"; cat $1) > tmp; mv tmp $1
And then you could use it like:
$ date > logdates
$ date >> logdates
$ cat logdates
Tue Mar 11 22:14:34 CDT 2014
Tue Mar 11 22:14:37 CDT 2014
$ date | ./sv logdates
$ cat logdates
Tue Mar 11 22:14:50 CDT 2014
Tue Mar 11 22:14:34 CDT 2014
Tue Mar 11 22:14:37 CDT 2014
This will only work for one-line appends though, as read is terminated by the newline (/n) character.
Simply use tac ( reverse of cat ) to output a file in reverse
#!/bin/sh
tac logdates > /tmp/out && mv /tmp/out logdates
Related
I would like to get filenames, creation dates, modification dates and file mime-types from directory structure. I've made a script which reads as follows :
#!/bin/bash
output="file_list.csv"
## columns
echo '"File name";"Creation date";"Modification date";"Mime type"' > $output
## content
find $1 -type f -printf '"%f";"%Tc";"%Cc";"no idea!"\n' >> $output
which gives me encouraging results :
"File name";"Creation date";"Modification date";"Mime type"
"Exercice 4 Cluster.xlsx";"ven. 27 mars 2020 10:35:46 CET";"mar. 17 mars 2020 19:14:18 CET";"no idea!"
"Exercice 5 Bayes.xlsx";"ven. 27 mars 2020 10:36:30 CET";"ven. 20 mars 2020 16:18:54 CET";"no idea!"
"Exercice 3 Régression.xlsx";"ven. 27 mars 2020 10:36:46 CET";"mer. 28 août 2019 17:21:10 CEST";"no idea!"
"Archers et Clustering.xlsx";"ven. 27 mars 2020 10:37:34 CET";"lun. 16 mars 2020 14:12:05 CET";"no idea!"
...
but I'm missing a capital thing : how do I get the files mime-types ? It would be great if I could chain the command 'mimetype -b' on each file found with 'find' command, and write it in the convenient column.
Thanks in advance,
Cyril
You might try using the -exec option of the find command, in which the brackets {} represent the name of the current file.
Then, you could remove the new line when appending to an existing file: AFAIK default behavior automatically appends new content to a new line, so the \n should not be necessary.
Last, you want to have a closing quote after your mimetype, so you should not only use the -b option, but the --output-format one, which will give you more control over what you want to display.
Hence the third command of your script should look like this:
find $1 -type f -printf '"%f";"%Tc";"%Cc";"' -exec mimetype --output-format %m\" {} \; >> $output
This is what I came up with:
for entry in *; do stat --printf='"%n";"%z";"%y";"' $entry; file -00 --mime-type $entry | cut -d $'\0' -f2; echo '"'; done
Uses a shell "for loop", to perform a stat on the directory entries in the current directory. Then uses file to get the mime type, and pipes that to cut to get only the mime type (by excluding the file name which is also printed by file).
The format for stat is what I believe was requested -- the file name, the last change date, the last modification date (both in ISO format, but could easily be made to UNIX seconds-since-epoch by upper-casing Z and Y).
Availability:
file: probably its own package if you are on Linux? But should be preinstalled on macOS I'm guessing.
bash/zsh: easily accessible both on Linux and macOS.
stat and cut: part of coreutils so should be preinstalled on most systems.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 3 years ago.
Improve this question
I am trying to list the files in a directory along with date and time.
I tried using the below two commands:
ls
getting output as below:
abc.txt testFile.txt
Then
ls -ltr
getting output as below:
-rw-r--r-- 1 xxxxxxx domain users 23 Aug 22 09:00 RCS
-rw-r--r-- 1 xxxxxxx domain users 0 Sep 12 06:09 testFile.txt
I expect an output as:
Aug 22 09:00 RCS
Sep 12 06:09 testFile.txt
You can use below command to get the output as you needed :
ls -l|awk '{print $6,$7,$8,$9}'
We are using -l as long listing which will give you most of the information and then we will use awk command to fetch what we want. You can tweak the print statement as per your requirement.
As pointed out by F Hauri, it would be better to use a process substitution with stat and read from there, e.g.
while read time file;do
printf "%(%b %e %T)T %s\n" $time "$file"
done < <(stat -c '%Z %n' *)
Example Use/Output
$ while read time file;do
printf "%(%b %e %T)T %s\n" $time "$file"
done < <(stat -c '%Z %n' *)
Apr 12 15:36:38 comboboxfocus.c
Apr 12 15:36:38 comboboxfocus2.c
Nov 3 17:43:51 createonclick.c
Feb 28 19:01:54 cw-drawinput.c
Nov 28 07:08:19 debug.c
Apr 12 15:36:38 debugsig.c
Jun 20 16:35:32 evboxstruct.c
Jun 20 16:35:32 evboxstruct2.c
Aug 23 08:50:38 ex00-window.c
Aug 23 08:50:38 ex01-hello.c
Aug 23 08:50:38 ex02-packing.c
Aug 23 08:50:38 ex03-builder.c
Feb 28 19:01:54 exampleappmain.c
Feb 28 19:01:54 exampleappwin_final.c
Feb 28 19:01:54 exampleappwin_wsrch.c
Dec 11 03:56:00 examplewindow.c
Dec 16 10:11:15 file_dialog_new.c
Dec 11 03:56:00 infobarex.c
Dec 7 14:03:20 poppler_page.c
Either will work, but awk will likely be quite a bit faster.
Why not use stat command ?
stat -c "%z %n" *
or you can use find command with its "-printf" switch to get this :
find -maxdepth 1 -type -f -printf '%t %f\n'
Similar output can be fetched using the find command.
Intro
After reading all this thread, I think rprakash's answer is the more accurate, as they point to two standard indicated command: stat and find -maxdepth....
Another way, using ls | sed
You could write:
ls -og|sed -re 's/\S*\s+\S+\s+\S+\s+//'
But as bash could be very efficient and because I prefer to be able to choose time ouput format (strftime) as I want:
Clean bash way to list date and files:
based on rprakash's answer stat command, with pretty formatting, as David C. Rankin' purpose, but without useless forks:
Short oneliner:
while read tm fl;do printf "%(%b %e %T)T %s\n" $tm "$fl";done< <(stat -c %Z\ %n *)
More readable:
while read time file;do
printf "%(%b %e %T)T %s\n" $time "$file"
done < <(
stat -c '%Z %n' *
)
And if you want sorted list, you could add sort in this way:
while read time file;do
printf "%(%b %e %T)T %s\n" $time "$file"
done < <(
stat -c '%Z %n' * |
sort -n
)
Some explanations:
From man stat:
%x time of last access, human-readable
%X time of last access, seconds since Epoch
%y time of last data modification, human-readable
%Y time of last data modification, seconds since Epoch
%z time of last status change, human-readable
%Z time of last status change, seconds since Epoch
From man bash
while read ... do; ... ; done < <(stat ... | sort ..)
Process Substitution
Process substitution allows a process's input or output to be referred to
using a filename. It takes the form of <(list) or >(list). The process list
is run asynchronously, and its input or output appears as a filename. This
filename is passed as an argument to the current command as the result of the
expansion. If the >(list) form is used, writing to the file will provide
input for list. If the <(list) form is used, the file passed as an argument
should be read to obtain the output of list. Process substitution is sup‐
ported on systems that support named pipes (FIFOs) or the /dev/fd method of
naming open files.
When available, process substitution is performed simultaneously with parame‐
ter and variable expansion, command substitution, and arithmetic expansion.
printf "%(...)T" $UNIXTIME
printf [-v var] format [arguments]
...
%(datefmt)T
causes printf to output the date-time string resulting from
using datefmt as a format string for strftime(3). The corre‐
sponding argument is an integer representing the number of sec‐
onds since the epoch. Two special argument values may be used:
-1 represents the current time, and -2 represents the time the
shell was invoked. If no argument is specified, conversion
behaves as if -1 had been given. This is an exception to the
usual printf behavior.
Was just wondering because I whipped this up last month.
#!/usr/bin/bash
# Collects all of the args, make sure to seperate with ','
IN="$*"
# Takes everything before a ',' and places them each on a single line of tmp file
echo $IN | sed 's/,/\n/g' > /tmp/pick.a.random.word.or.phrase
# Obvious vars are obvious
WORDFILE="/tmp/pick.a.random.word.or.phrase"
# Pick only one of the vars
NUMWORDS=1
## Picks a random line from tmp file
#Number of lines in $WORDFILE
tL=`awk 'NF!=0 {++c} END {print c}' $WORDFILE`
# Expand random
RANDOM_CMD='od -vAn -N4 -tu4 /dev/urandom'
for i in `seq $NUMWORDS`
do
rnum=$((`${RANDOM_CMD}`%$tL+1))
sed -n "$rnum p" $WORDFILE | tr '\n' ' '
done
printf "\n"
rm /tmp/pick.a.random.word.or.phrase
Mainly I ask:
Do I need to have a tmp file?
Is there a way to do this in one line with another program?
How to condense as much as possible?
The command-line argument handling is, to my mind, bizarre. Why not just use normal command line arguments? That makes the problem trivial:
#!/usr/bin/bash
shuf -en1 "$#"
Of course, you could just use shuf -en1, which is only nine keystrokes:
$ shuf -en1 word another_word "random phrase"
another_word
$ shuf -en1 word another_word "random phrase"
word
$ shuf -en1 word another_word "random phrase"
another_word
$ shuf -en1 word another_word "random phrase"
random phrase
shuf command-line flags:
-e Shuffle command line arguments instead of lines in a file/stdin
-n1 Produce only the first random line (or argument in this case)
If you really insist on running the arguments together and then separating them with commas, you can use the following. As with your original, it will exhibit unexpected behaviour if some word in the arguments could be glob-expanded, so I really don't recommend it:
#!/usr/bin/bash
IFS=, read -ra args <<<"$*"
echo $(shuf -en1 "${args[#]}")
The first line combines the arguments and then splits the result at commas into the array args. (The -a option to read.) Since the string is split at commas, spaces (such as though automatically inserted by the argument concatenation) are preserved; to remove the spaces, I word-split the result of shuf by not quoting the command expansion.
You could use shuff to shorten your script and remove temporary file.
#!/usr/bin/bash
# Collects all of the args, make sure to seperate with ','
IN="$*"
# Takes everything before a ',' and places them in an array
words=($(echo $IN | sed 's/,/ /g'))
# Get random indexi in range: 0, length of array: words
index=$(shuf -i 0-"${#words[#]}" -n 1)
# Print the random index
echo ${words[$index]}
If you don't want to use shuff, you could also use $RANDOM:
#!/usr/bin/bash
# Collects all of the args, make sure to seperate with ','
IN="$*"
# Takes everything before a ',' and places them in an array
words=($(echo $IN | sed 's/,/ /g'))
# Print the random index
echo ${words[$RANDOM % ${#words[#]}]}
shuf in coreutils does exactly this, but with multiple command arguments instead of a single comma separated argument.
shuf -n1 -e arg1 arg2 ...
The -n1 option says to choose just one element. The -e option indicates that elements will be passed as arguments (as opposed to through standard input).
Your script then just needs to replace commas with spaces in $*. We can do this using bash parameter substitution:
#!/usr/bin/bash
shuf -n1 -e ${*//,/ }
This won't work with elements with embedded spaces.
Isn't it as simple as generating a number at random between 1 and $# and simply echo the corresponding argument? It depends on what you have; your comment about 'collect arguments; make sure to separate with commas' isn't clear, because the assignment does nothing with commas — and you don't show how you invoke your command.
I've simply cribbed the random number generation from the question: it works OK on my Mac, generating the values 42,405,691 and 1,817,261,076 on successive runs.
n=$(( $(od -vAn -N4 -tu4 /dev/urandom) % $# + 1 ))
eval echo "\${$n}"
You could even reduce that to a single line if you were really determined:
eval echo "\${$(( $(od -vAn -N4 -tu4 /dev/urandom) % $# + 1 ))}"
This use of eval is safe as it involves no user input. The script should check that it is provided at least one argument to prevent a division-by-zero error if $# is 0. The code does an absolute minimum of data movement — in contrast to solutions which shuffle the data in some way.
If that's packaged in a script random_selection, then I can run:
$ bash random_selection Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Feb
$ bash random_selection Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Oct
$ bash random_selection Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Nov
$
If the total number of arguments is big enough that you run out of argument space, then you need to think again, but that restriction is present in the existing code.
The selection is marginally biassed towards the earlier entries in the list; you have to do a better job of rejecting random numbers that are very near the maximum value in the range. For a random 32-bit unsigned value, if it is larger than $# * (0xFFFFFFFF / $#) you should generate another random number.
I have a command that keeps printing on the screen. I have to collect its print to do something, while, I have to supervise the print.
How could I do that in bash?
Use tee:
VAR=$(my_cmd --args ... | tee /dev/stderr)
tee outputs to the screen and a file at the same time. But, here we specify stderr, which is another stream that goes to the screen. The variable captures stdout, and tee puts a copy of that output on stderr which goes to your screen.
You can duplicate your stdout:
exec 9>&1
variable=$(date |tee >(cat - >&9))
echo "Variable contains: $variable"
prints:
Thu Oct 2 21:21:52 CEST 2014 #normal output from the date
Variable contains: Thu Oct 2 21:21:52 CEST 2014 #the echo...
Can somebody tell me how to print variable and not executing it in bash? I mean:
bash#bash $ cat script.sh
#!/bin/bash
echo $1
bash#bash $ ./script.sh "`date`"
Sat Sep 20 18:42:19 CEST 2014
I don't want to get:
Sat Sep 20 18:35:37 CEST 2014
I want to get output:
date
I'm interested in how to prevent executing sent commands to script.
It looks like you are trying to prevent code from being injected into your script. The problem with echo $1 is that the contents of $1 are being evaluated by the shell. In order to avoid that, you need to wrap $1 in double quotes in your script:
#!/bin/bash
echo "$1"
Testing it out:
$ ./script.sh '`date`'
`date`
The problem in your question is that you are using double quotes around "date", so the expansion has occurred before your script is run. You can use set -x to see the difference:
$ set -x
$ ./script '`date`'
+ ./script '`date`'
`date`
$ ./script "`date`"
++ date # date is being run
+ ./script 'Sat Sep 20 18:01:32 BST 2014' # result is passed to your script
Sat Sep 20 18:01:32 BST 2014
There is nothing you can do about this.
I think the following section of my original answer is still relevant, so I'll leave it in.
Different types of quotes in bash
Backticks (which you have used in your question) are an old-fashioned way of capturing the output of executing a command. The more modern syntax is $( ), so if you wanted to store the current date in a variable you could do d=$(date). Single quotes are used for literal strings, so echo '$d' would output $d, not the value of the variable. Variables inside double quotes are expanded, so echo "$d" would output the value of the variable $d. It is always a good idea to wrap your variables in double quotes to prevent word splitting and glob expansion.
Replace the backticks from var1 with single quotes:
var1='date'
var2="echo $var1"
echo $var2