The task is to read a file containing letters and numbers. Then put the letters in a text file inside a Letters directory and the numbers in a different text file in a numbers directory. The problem I'm having is I don't know how I would identify strings and integers within a file.
The file I'm using contains:
723
Xavier
Cat
323
Zebra
This is the code I've come up with.
#!/bin/bash
file = $1
mkdir Numbers
mkdir Letters
touch Numbers/Numbers.txt
touch Letters/Letters.txt
for x in $file; do
if [x == string];
then
echo x >> Letters/Letters.txt;
fi
if [x == integer];
then
echo x >> Numbers/Numbers.txt;
fi
Your original code has a lot of syntactical errors. I suggest going forward please submit a code which is at least syntactically correct.
Anyways, assuming you are new to unix shell scripting, here a quick solution to your problem
#!/bin/bash
file=$1
mkdir Numbers
mkdir Letters
touch Numbers/Numbers.txt
touch Letters/Letters.txt
while read line
do
if [[ $line =~ [^0-9] ]]
then
echo $line >> Letters/Letters.txt
else
echo $line >> Numbers/Numbers.txt
fi
done < $file
The question is vague.
This solves the question as posted, and handles lines with mixed
numbers and strings, (e.g.: "a1b2c3".), using tee and bash
process substitution, with tr to delete all undesired chars:
mkdir Numbers Letters
tee >(tr -d '[0-9]' > Letters/Letters.txt) \
>(tr -d '[a-zA-Z]' > Numbers/Numbers.txt) \
> /dev/null < inputfile
If the goal is to remove whole lines that are either all numbers
or all letters, try grep:
mkdir Numbers Letters
tee >(grep -v '^[0-9]*$' > Letters/Letters.txt) \
>(grep '^[0-9]*$' > Numbers/Numbers.txt) \
> /dev/null < inputfile
Related
I have a text file named creating.txt. In this file, each line has a path.
If line starts with D it means that I have to create directory.
If line starts with F, I have to create a file.
For example:
D:/root/F1/Cat1
F:/root/F1/file1.txt
D:/root/F1/Cat1/Cat2
I need to go line by line, first check if first letter is D or F, then ignore the : and the first letter and accordingly create a file or directory based on given path. If first letter in line is not D or F, just ignore the line.
Also if the file already exists I'm not allowed to create another one.
I tried using awk:
path=`awk { print } creating.txt`
touch $path
But the problem is this reads the whole file (not line by line) so there have to be only one line. Also I don't know how to ignore D: and F:.
It can be done with two 2 one-liners
I reproduced your input :
$ cat creating.txt
D:/root/F1/Cat1
F:/root/F1/file1.txt
D:/root/F1/Cat1/Cat2
You can use this to create the Directories
$ for path in `grep ^D creating.txt | cut -d: -f2`; do mkdir -p "$path";done
Then, for files
$ for file in `grep ^F creating.txt | cut -d: -f2`; do touch "$file";done
Explanation:
The grep, will select the lines beginning [as I used ^] by D (or F)
The cut, will catch the 2nd field using ':' as delimiter
Here is a bash script that will do as you asked:
#!/bin/bash
while IFS= read -r line; do
[[ $line =~ (.):(.*) ]]
[[ ${BASH_REMATCH[1]} == D ]] && mkdir -p "${BASH_REMATCH[2]}"
[[ ${BASH_REMATCH[1]} == F ]] && [[ ! -d ${BASH_REMATCH[2]%/*} ]] && mkdir -p "${BASH_REMATCH[2]%/*}"
[[ ${BASH_REMATCH[1]} == F ]] && touch "${BASH_REMATCH[2]}"
done < creating.txt
I am newbie to shell scripting. I have a requirement to read a file by line and match for specific string. If it matches, print x and if it doesn't match, print y.
Here is what I am trying. But,I am getting unexpected results. I am getting 700 lines of result where my /tmp/l1.txt has 10 lines only. Somewhere, I am going through the loop. I appreciate your help.
for line in `cat /tmp/l3.txt`
do
if echo $line | grep "abc.log" ; then
echo "X" >>/tmp/l4.txt
else
echo "Y" >>/tmp/l4.txt
fi
done
I don't understand the urge to do looping ...
awk '{if($0 ~ /abc\.log/){print "x"}else{print "y"}}' /tmp/13.txt > /tmp/14.txt
EDIT after inquiry ...
Of course, your spec wasn't overly precise, and I'm jumping to conclusions regarding your lines format ... we basically take the whole line that matched abc.log, replace everything up to the directory abc and from /log to the end of line with nothing, which leaves us with clusterX/xyz.
awk '{if($0 ~ /abc\.log/){print gensub(/.+\/abc\/(.+)\/logs/, "\\1", 1)}else{print "y"}}' /tmp/13.txt > /tmp/14.txt
cat /tmp/l3.txt | while read line # read the entire line into the variable "line"
do
if [ -n `echo "$line" | grep "abc.log"` ] # If there is a value "-n"
then
echo "X" >> /tmp/l4.txt # Echo "X" or the value of the variable "line" into l4.txt
else
echo "Y" >> /tmp/l4.txt # If empty echo "Y" into l4.txt
fi
done
While read statement will read either the entire line if only one variable is given, in this case "line" or if you have a fixed amount of fields you can specify a variable for each field, I.E. "| while read field1 field2" etc... The -n tests for if their is a value or not. -z will test if it's empty.
Why worry about cat and the rest before grep, you can simply test the return of grep and append all matching lines to /tmp/14.txt or append "Y":
[ -f "/tmpfile.tmp" ] && :> /tmpfile.tmp # test for existing tmpfile & truncate
if grep "abc.log" /tmp/13.txt >>tmpfile.tmp ; then # write all matching lines to tmpfile
cat tmpfile.tmp /tmp/14.txt # if grep matched append to /tmp/14.txt
else
echo "Y" >> /tmp/14.txt # write "Y" to /tmp/14.txt
fi
rm tmpfile.tmp # cleanup
Note: if you don't want the result of the grep appended to /tmp/14.txt, then just replace cat tmpfile.tmp /tmp/14.txt with echo "X" >> /tmp/14.txt and you can remove the 1st and last lines.
I think the "awk" answer above is better. However, if you really need to interact using a bash loop, you can use:
PATTERN="abc.log"
OUTPUTFILE=/tmp/14.txt
INPUTFILE=/tmp/13.txt
while read line
do
grep -q "$PATTERN" <<< "$line" > /dev/null 2>&1 && echo X || echo Y
done < $INPUTFILE >> $OUTPUTFILE
I have a script running that is checking multiples directories and comparing them to expanded tarballs of the same directories elsewhere.
I am using diff -r -q and what I would like is that when diff finds any difference in the recursive run it will stop running instead of going through more directories in the same run.
All help appreciated!
Thank you
#bazzargh I did try it like you suggested or like this.
for file in $(find $dir1 -type f);
do if [[ $(diff -q $file ${file/#$dir1/$dir2}) ]];
then echo differs: $file > /tmp/$runid.tmp 2>&1; break;
else echo same: $file > /dev/null; fi; done
But this only works with files that exist in both directories. If one file is missing I won't get information about that. Also the directories I am working with have over 300.000 files so it seems to be a bit of overhead to do a find for each file and then diff.
I would like something like this to work, with and elif statement that checks if $runid.tmp contains data and breaks if it does. I added 2> after the first if statement so stderr is sent to the $runid.tmp file.
for file in $(find $dir1 -type f);
do if [[ $(diff -q $file ${file/#$dir1/$dir2}) ]] 2> /tmp/$runid.tmp;
then echo differs: $file > /tmp/$runid.tmp 2>&1; break;
elif [[ -s /tmp/$runid.tmp ]];
then echo differs: $file >> /tmp/$runid.tmp 2>&1; break;
else echo same: $file > /dev/null; fi; done
Would this work?
You can do the loop over files with 'find' and break when they differ. eg for dirs foo, bar:
for file in $(find foo -type f); do if [[ $(diff -q $file ${file/#foo/bar}) ]]; then echo differs: $file; break; else echo same: $file; fi; done
NB this will not detect if 'bar' has directories that do not exist in 'foo'.
Edited to add: I just realised I overlooked the really obvious solution:
diff -rq foo bar | head -n1
It's not 'diff', but with 'awk' you can compare two files (or more) and then exit when they have a different line.
Try something like this (sorry, it's a little rough)
awk '{ h[$0] = ! h[$0] } END { for (k in h) if (h[k]) exit }' file1 file2
Sources are here and here.
edit: to break out of the loop when two files have the same line, you may have to do the loop in awk. See here.
You can try the following:
#!/usr/bin/env bash
# Determine directories to compare
d1='./someDir1'
d2='./someDir2'
# Loop over the file lists and diff corresponding files
while IFS= read -r line; do
# Split the 3-column `comm` output into indiv. variables.
lineNoTabs=${line//$'\t'}
numTabs=$(( ${#line} - ${#lineNoTabs} ))
d1Only='' d2Only='' common=''
case $numTabs in
0)
d1Only=$lineNoTabs
;;
1)
d2Only=$lineNoTabs
;;
*)
common=$lineNoTabs
;;
esac
# If a file exists in both directories, compare them,
# and exit if they differ, continue otherwise
if [[ -n $common ]]; then
diff -q "$d1/$common" "$d2/$common" || {
echo "EXITING: Diff found: '$common'" 1>&2;
exit 1; }
# Deal with files unique to either directory.
elif [[ -n $d1Only ]]; then # fie
echo "File '$d1Only' only in '$d1'."
else # implies: if [[ -n $d2Only ]]; then
echo "File '$d2Only' only in '$d2."
fi
# Note: The `comm` command below is CASE-SENSITIVE, which means:
# - The input directories must be specified case-exact.
# To change that, add `I` after the last `|` in _both_ `sed commands`.
# - The paths and names of the files diffed must match in case too.
# To change that, insert `| tr '[:upper:]' '[:lower:]' before _both_
# `sort commands.
done < <(comm \
<(find "$d1" -type f | sed 's|'"$d1/"'||' | sort) \
<(find "$d2" -type f | sed 's|'"$d2/"'||' | sort))
The approach is based on building a list of files (using find) containing relative paths (using sed to remove the root path) for each input directory, sorting the lists, and comparing them with comm, which produces 3-column, tab-separated output to indicated which lines (and therefore files) are unique to the first list, which are unique to the second list, and which lines they have in common.
Thus, the values in the 3rd column can be diffed and action taken if they're not identical.
Also, the 1st and 2nd-column values can be used to take action based on unique files.
The somewhat complicated splitting of the 3 column values output by comm into individual variables is necessary, because:
read will treat multiple tabs in sequence as a single separator
comm outputs a variable number of tabs; e.g., if there's only a 1st-column value, no tab is output at all.
I got a solution to this thanks to #bazzargh.
I use this code in my script and now it works perfectly.
for file in $(find ${intfolder} -type f);
do if [[ $(diff -q $file ${file/#${intfolder}/${EXPANDEDROOT}/${runid}/$(basename ${intfolder})}) ]] 2> ${resultfile}.tmp;
then echo differs: $file > ${resultfile}.tmp 2>&1; break;
elif [[ -s ${resultfile}.tmp ]];
then echo differs: $file >> ${resultfile}.tmp 2>&1; break;
else echo same: $file > /dev/null;
fi; done
thanks!
I need to find strings matching some regexp pattern and represent the search result as array for iterating through it with loop ), do I need to use sed ? In general I want to replace some strings but analyse them before replacing.
Using sed and diff:
sed -i.bak 's/this/that/' input
diff input input.bak
GNU sed will create a backup file before substitutions, and diff will show you those changes. However, if you are not using GNU sed:
mv input input.bak
sed 's/this/that/' input.bak > input
diff input input.bak
Another method using grep:
pattern="/X"
subst=that
while IFS='' read -r line; do
if [[ $line = *"$pattern"* ]]; then
echo "changing line: $line" 1>&2
echo "${line//$pattern/$subst}"
else
echo "$line"
fi
done < input > output
The best way to do this would be to use grep to get the lines, and populate an array with the result using newline as the internal field separator:
#!/bin/bash
# get just the desired lines
results=$(grep "mypattern" mysourcefile.txt)
# change the internal field separator to be a newline
IFS=$'/n'
# populate an array from the result lines
lines=($results)
# return the third result
echo "${lines[2]}"
You could build a loop to iterate through the results of the array, but a more traditional and simple solution would just be to use bash's iteration:
for line in $lines; do
echo "$line"
done
FYI: Here is a similar concept I created for fun. I thought it would be good to show how to loop a file and such with this. This is a script where I look at a Linux sudoers file check that it contains one of the valid words in my valid_words array list. Of course it ignores the comment "#" and blank "" lines with sed. In this example, we would probably want to just print the Invalid lines only but this script prints both.
#!/bin/bash
# -- Inspect a sudoer file, look for valid and invalid lines.
file="${1}"
declare -a valid_words=( _Alias = Defaults includedir )
actual_lines=$(cat "${file}" | wc -l)
functional_lines=$(cat "${file}" | sed '/^\s*#/d;/^\s*$/d' | wc -l)
while read line ;do
# -- set the line to nothing "" if it has a comment or is empty line.
line="$(echo "${line}" | sed '/^\s*#/d;/^\s*$/d')"
# -- if not set to nothing "", check if the line is valid from our list of valid words.
if ! [[ -z "$line" ]] ;then
unset found
for each in "${valid_words[#]}" ;do
found="$(echo "$line" | egrep -i "$each")"
[[ -z "$found" ]] || break;
done
[[ -z "$found" ]] && { echo "Invalid=$line"; sleep 3; } || echo "Valid=$found"
fi
done < "${file}"
echo "actual lines: $actual_lines funtional lines: $functional_lines"
I'm trying to write a script to add the name of a file and the directory path to a text file on a single line.
E.g.
Filename /root/folder/folder/
I've tried:
ls "$1" >> /root/folder/folder/file.txt
pwd >> /root/folder/folder/file.txt
But it shows up on seperate lines.
I did try
ls "$1" && pwd >> ......
but it only pasted the pwd to the text file.
I'm new to Linux so any help would be greatly appreciated.
How about this:
echo "$1 $(pwd)" >> outputfile
Use:
p1=`ls $1`
p2=`pwd`
echo $p1 $p2 >> /root/folder/folder/file.txt
Try:
echo "$1$(pwd)" >> /root/folder/folder/file.txt
Or you can create a new variable with the concatenated string and then echo that:
NewVar="$1$(pwd)"
echo "$NewVar" >> /root/folder/folder/file.txttxt
In the first example you have above, each command inserts a newline. In the second example, the && that you have breaks up the two commands so that the output of the first is not redirected. You can't distribute redirection across the &&, in other words.
A common technique is to use printf (or echo -n, but that's less portable) to avoid writing the newline:
exec >> /root/folder/folder/file.txt
printf "%s " "$1"
pwd
In your example, you use a command to generate the initial data, and a good technique in that case is to use tr to trim the newline
ls "$1" | tr -d '\012'
printf " " # Insert a space
pwd