`sed` pattern matching? - linux

Permissions links Owner Group Size Date Time Directory or file
-rwxr--r-- 1 User1 root 26 2012-04-12 19:51 MyFile.txt
drwxrwxr-x 3 User2 csstf 4096 2012-03-15 00:12 MyDir
I have problem for pattern match to get certain details using the above details. I actually need to write down the shell script to get the following details.
I need to use pipe in this question. When I do ls -la | prog.sh it need to show the details below.
The major part I don't get is how to use sed pattern matching.
1. Total number of lines read.
2. Total number of different users (owners).
3. Total number of files with execute permission for the owner.
4. The top 3 largest directory.
This is what I have tried so far
#!/bin/bash
while read j
do
B=`sed -n '$=' $1`
echo "total number of lines read = $B"
done

The while loop reads the output of ls -la line by line and you need to process each line and maintain variables for the information you need.
Here is a sample script to get you started:
#!/bin/bash
declare -i lineCount=0
declare -i executePermissionCount=0
# an array to keep track of owners
declare -a owners=()
# read each line into an array called lineFields
while read -r -a lineFields
do
# the owner is the third element in the array
owner="${lineFields[2]}"
# check if we have already seen this owner before
found=false
for i in "${owners[#]}"
do
if [[ $i == $owner ]]
then
found=true
fi
done
# if we haven't seen this owner, add it to the array
if ! $found
then
owners+=( "$owner" )
fi
# check if this file has owner execute permission
permission="${lineFields[0]}"
# the 4th character should be x
if [[ ${permission:3:1} == "x" ]]
then
(( executePermissionCount++ ))
fi
# increment line count
(( lineCount++ ))
done
echo "Number of lines: $lineCount"
echo "Number of different owners: ${#owners[#]}"
echo "Number of files with execute permission: $executePermissionCount"

Related

Loop to compare numbers within file name

So I've written a code to compare if a certain number within the file name is bigger than 11 and if it is than it should make a directory.
-->Mainfolder
-->Jan
-->Huistaak1-HelloWorld_Jonas.De Preter.s.ua_poging_2019-11-12
-->Feremans
-->Huistaak1-HelloWorld_Len.Feremans.s.ua_poging_2019-11-10
...
The code needs to get the day of the provided date
and if it's above 11 it creates a directory "late_inzending"
So it should look like this
-->Mainfolder
-->Jan
-->Huistaak1-HelloWorld_Jonas.De Preter.s.ua_poging_2019-11-12
-->late_inzending
...
My code doesn't seem to work
for dir in */
do
cut1=$(echo "$dir" | cut -f4 -d '_')
cut2=$(echo "$cut1" | cut -f3 -d '-')
declare -i x="$cut2"
declare -i y=11
if (( x > y))
then
mkdir late_inzending
fi
done
Something like.
#!/usr/bin/env bash
for d in ./*/*/; do #: From main plus one level down
[[ ! -d "$d" ]] && continue #: If it is not a directory, skip it.
end="${d##*-}" #: To remain only the last 2 strings and a /, e.g. 12/
(( ${end%/} > 11 )) && #: Remove the trailing `/`, to remain only 12 and compare.
echo mkdir -p "$d"late_inzending #: Append the desired string to the directory and create it.
done
Execute from within main
Remove the echo if you're ok with the ouput.
Resources for the answer above
Parameter Expansion
Shell Arithmetic
Conditional Constructs
help test

How to search string references in specified location?

I'm trying to find the occurence of elements list(from a text file) in a directory.
Below is the Bash code I'm using ,but I'm unable to get the output of grep command on to console.
!/bin/bash
FILENAME=$1
count=0
while read LINE
do
let count++
echo "$count $LINE"
grep -r $LINE /home/user/vaishnavi
done < $FILENAME
echo -e "\nTotal $count Lines read"
Output:
1 ASK
2 TELL
3 ORDER
4 NUMBER
5 SIZE
6 BASKET
7 FRUIT
8 VEGGIES
Total 8 Lines read
I'm getting only the list of elements but not their occurences in the specified location.
Is there anything wrong with my code?
Thanks.
You need to echo the result of the grep, for example:
echo $(grep -r $LINE /home/user/vaishnavi)

Export variables to another script

I am making 2 scripts. The first script is going to take a file, and then move it to a directory named "Trash". The second script will recover this file and send it back to it's original directory. So far I have the first script moving the file correctly.
Here is my code so far:
For delete.sh
FILE=$1
DATEDELETED=$(date)
DIRECTORY=$(pwd)
mv $1 Trash
echo $FILE
echo $DATEDELETED
echo $DIRECTORY
Output:
trashfile
Sun Mar 2 21:37:21 CST 2014
/home/user
For undelete.sh:
PATH=/home/user/Trash
for file in $PATH
do
$file | echo "deleted on" | date -r $file
done
echo "Enter the filename to undelete from the above list:"
EDIT: So I realized that I don't need variables, I can just list all the files in the Trash directory and edit the output to what I want. I'm having a little trouble with my for statement though, I'm getting these two errors: ./undelete.sh: line 6: date: command not found
./undelete.sh: line 6: /home/user/Trash: Is a directory. So I'm not exactly sure what I'm doing wrong in my for statement.
Here is the expected output:
file1 deleted on Tue Mar 16 17:15:34 CDT 2010
file2 deleted on Tue Mar 16 17:15:47 CDT 2010
Enter the filename to undelete from the above list:
Well I've accomplished what could be a workaround for what your scenario is trying to accomplish.
Basically you can enter echo "script2variable=$script1variable" >> script2.sh from script1.sh. Then use the source command to call that script later from any script you desire. Might have to just play with the theories involved.
Good Luck!
Delete Script file
#!/bin/bash
# delete.sh file
# Usage: ./delete.sh [filename]
#DATEDELETED=$(date) #not best solution for this kind of application
DIR=$(pwd)
fn=$1
#Specify your trash directory or partition
trash=~/trash
#Set path and filename for simple use in the script
trashed=$trash/$fn.tgz
#Send variables to new manifest script.
echo "#!/bin/bash" > $1.mf.sh
echo "DIR=$DIR" >> $1.mf.sh
# Questionable need?
#echo "DATEDELETED=$DATEDELETED" >> $1.mf.sh
echo "mv $1 $DIR" >> $1.mf.sh
echo Compressing
tar -cvpzf $trashed $1 $1.mf.sh
if [ $? -ne 0 ]; then
echo Compression Failed
else
echo completed trash compression successfully
echo Trashbin lists the file as $trashed
rm $1 -f
rm $1.mf.sh -f
echo file removed successfully
fi
exit 0
Restore Script File
#!/bin/bash
# restore.sh
# Usage: ./restore.sh
# filename not required for this script, use index selection
fn=$1
#SET LOCATION OF TRASH DIRECTORY
trash=~/trash
listfile=($(ls $trash))
#SET COUNTER FOR LISTING FILES = 0
i=0
#THIS IS JUST A HEADER FOR YOUR OUTPUT.
clear #cleanup your shell
echo -e INDEX '\t' Filename '\t' '\t' '\t' Date Removed
#Echo a list of files from the array with the counter.
for FILES in "${listfile[#]}"
do
echo -e $i '\t' $FILES '\t' "deleted on $(date -r $trash/$FILES)"
let "i += 1"
done
#Output total number of items from the ls directory.
echo -e '\n' $i file\(s\) found in the trash!
# 1 Offset for 1 = 0, 2 = 1, and so on.
let "i -= 1"
#Require input of a single, double, or triple digit number only.
#Loop back prompt if not a number.
while true;
do
read -p "Enter an index number for file restoration: " indx
case $indx in
[0-9]|[0-9][0-9]|[0-9][0-9][0-9] ) break ;;
* ) read -p "Please enter a valid number 0-$i: " indx ;;
esac
done
#
script=$(echo ${listfile[$indx]}|sed 's/\.tgz/\.mf.sh/g')
tar -xvpzf $trash/${listfile[$indx]}
rm $trash/${listfile[$indx]}
sleep 2
chmod +x $script
source $script
rm $script
Run the script with source
source <yourscript>
or
. ./<yourscript>
In your case
. ./delete.sh && ./undelete.sh
Hope this will help

Bash script won't work: command not found

I'm trying to create a simple shell script to list the first input 6 times, a line, then report the size of the second input. Here is my script:
#!/bin/sh
# script1.sh
#
#
# $1=filename $2=number
i=0
while [$i -lt 7] #line 11
do
i=$(($i + 1))
echo $1
done
printf "\n"
if [$2 -gt 1000] #line 19
then
echo 'This is a big number!'
else
echo 'This is a small number.'
fi
Here is the error I receive when trying to use:
./script1.sh test 131234
./script1.sh: line 11: [0: command not found
./script1.sh: line 19: [131234: command not found
This is a small number.
I suppose it partially works but something about the command -lt and -gt is causing an error. Running on both Linux and Terminal (OS X) provide the same error.
You need spaces on your [] condition:
if [ $2 -gt 1000 ] #line 19
^ ^
and
while [ $i -lt 7 ] #line 11
^ ^
instead of
if [$2 -gt 1000] #line 19
and
while [$i -lt 7] #line 11
Note that otherwise it will not consider these expressions as it should. Instead, it understand them as a string called [$2 or [$i.
You need spaces around []:
while [ $i -lt 7 ] #line 11
And the other lines with [].
[ is a command like anything else, so when you use [$i, it's trying to execute [$i, in this case [0 and [121234.
$ \[.exe --help
Usage: test EXPRESSION
or: test
or: [ EXPRESSION ]
or: [ ]
or: [ OPTION
Exit with the status determined by EXPRESSION.
--help display this help and exit
--version output version information and exit
An omitted EXPRESSION defaults to false. Otherwise,
EXPRESSION is true or false and sets exit status. It is one of:
( EXPRESSION ) EXPRESSION is true
! EXPRESSION EXPRESSION is false
EXPRESSION1 -a EXPRESSION2 both EXPRESSION1 and EXPRESSION2 are true
EXPRESSION1 -o EXPRESSION2 either EXPRESSION1 or EXPRESSION2 is true
-n STRING the length of STRING is nonzero
STRING equivalent to -n STRING
-z STRING the length of STRING is zero
STRING1 = STRING2 the strings are equal
STRING1 != STRING2 the strings are not equal
INTEGER1 -eq INTEGER2 INTEGER1 is equal to INTEGER2
INTEGER1 -ge INTEGER2 INTEGER1 is greater than or equal to INTEGER2
INTEGER1 -gt INTEGER2 INTEGER1 is greater than INTEGER2
INTEGER1 -le INTEGER2 INTEGER1 is less than or equal to INTEGER2
INTEGER1 -lt INTEGER2 INTEGER1 is less than INTEGER2
INTEGER1 -ne INTEGER2 INTEGER1 is not equal to INTEGER2
FILE1 -ef FILE2 FILE1 and FILE2 have the same device and inode numbers
FILE1 -nt FILE2 FILE1 is newer (modification date) than FILE2
FILE1 -ot FILE2 FILE1 is older than FILE2
-b FILE FILE exists and is block special
-c FILE FILE exists and is character special
-d FILE FILE exists and is a directory
-e FILE FILE exists
-f FILE FILE exists and is a regular file
-g FILE FILE exists and is set-group-ID
-G FILE FILE exists and is owned by the effective group ID
-h FILE FILE exists and is a symbolic link (same as -L)
-k FILE FILE exists and has its sticky bit set
-L FILE FILE exists and is a symbolic link (same as -h)
-O FILE FILE exists and is owned by the effective user ID
-p FILE FILE exists and is a named pipe
-r FILE FILE exists and read permission is granted
-s FILE FILE exists and has a size greater than zero
-S FILE FILE exists and is a socket
-t FD file descriptor FD is opened on a terminal
-u FILE FILE exists and its set-user-ID bit is set
-w FILE FILE exists and write permission is granted
-x FILE FILE exists and execute (or search) permission is granted
Except for -h and -L, all FILE-related tests dereference symbolic links.
Beware that parentheses need to be escaped (e.g., by backslashes) for shells.
INTEGER may also be -l STRING, which evaluates to the length of STRING.
NOTE: [ honors the --help and --version options, but test does not.
test treats each of those as it treats any other nonempty STRING.
NOTE: your shell may have its own version of test and/or [, which usually supersedes
the version described here. Please refer to your shell's documentation
for details about the options it supports.
Report [ bugs to bug-coreutils#gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
For complete documentation, run: info coreutils '[ invocation'
Work with Cygwin+Bash, should work in any environment ,IMHO.

How to add line number for output, prompt for line, then act based on input?

I wrote a shell script like this:
#! /bin/sh
...
ls | grep "android"
...
and the output is :
android1
android2
xx_android
...
I want to add a number in each file, like this:
1 android1
2 android2
3 XX_android
...
please choose your dir number:
and then wait for the user input line number x, the script reads the line number back then process the corresponding dir. How can we do this in shell ? Thanks !
nl prints line numbers:
ls | grep android | nl
If you pipe the result into cat, you can use the -n option to number each line like so:
ls | grep "android" | cat -n
Pass -n to grep, as follows:
ls | grep -n "android"
From the grep man-page:
-n, --line-number
Prefix each line of output with the line number within its input file.
Instead of implementing the interaction, you can use built-in command select.
select d in $(find . -type d -name '*android*'); do
if [ -n "$d" ]; then
# put your command here
echo "$d selected"
fi
done
The other answers on this page actually don't answer the question 100%. They don't show how to let the user interactively choose the file from another script.
The following approach will allow you to do this, as can be seen in the example. Note that the select_from_list script was pulled from this stackoverflow post
$ ls
android1 android4 android7 mac2 mac5
android2 android5 demo.sh mac3 mac6
android3 android6 mac1 mac4 mac7
$ ./demo.sh
1) android1 3) android3 5) android5 7) android7
2) android2 4) android4 6) android6 8) Quit
Please select an item: 3
Contents of file selected by user: 2.3 Android 1.5 Cupcake (API 3)
Here's the demo.sh and the script it uses to select an item from a list, select_from_list.sh
demo.sh
#!/usr/bin/env bash
# Ask the user to pick a file, and
# cat the file contents if they select a file.
OUTPUT=$(\ls | grep android | select_from_list.sh | xargs cat)
STATUS=$?
# Check if user selected something
if [ $STATUS == 0 ]
then
echo "Contents of file selected by user: $OUTPUT"
else
echo "Cancelled!"
fi
select_from_list.sh
#!/usr/bin/env bash
prompt="Please select an item:"
options=()
if [ -z "$1" ]
then
# Get options from PIPE
input=$(cat /dev/stdin)
while read -r line; do
options+=("$line")
done <<< "$input"
else
# Get options from command line
for var in "$#"
do
options+=("$var")
done
fi
# Close stdin
0<&-
# open /dev/tty as stdin
exec 0</dev/tty
PS3="$prompt "
select opt in "${options[#]}" "Quit" ; do
if (( REPLY == 1 + ${#options[#]} )) ; then
exit 1
elif (( REPLY > 0 && REPLY <= ${#options[#]} )) ; then
break
else
echo "Invalid option. Try another one."
fi
done
echo $opt
This works for me:
line-number=$(ls | grep -n "android" | cut -d: -f 1)
I use this in a script to remove sections of my sitemap.xml which I don't want Googlebot to crawl. I search for the URL (which is unique) and then find the line number using the above. Using simple maths the script then calculates the range of numbers required to delete the entire entry in the XML file.
I agree with jweyrich regarding updating your question to get better answers.

Resources