I need to remove a line in a specified file if it has more than one word in it using a bash script in linux.
e.g. file:
$ cat testfile
This is a text
file
This line should be deleted
this-should-not.
awk 'NF<=1{print}' testfile
a word being a run of non-whitespace.
Just for fun, here's a pure bash version which doesn't call any other executable (since you asked for it in bash):
$ while read a b; do if [ -z "$b" ]; then echo $a;fi;done <testfile
awk '!/[ \t]/{print $1}' testfile
This reads "print the first element of lines that don't contain a space or a tab".
Empty lines will be output (since they don't contain more than one word).
Easy enough:
$ egrep -v '\S\s+\S' testfile
$ sed '/ /d' << EOF
> This is a text
> file
>
> This line should be deleted
> this-should-not.
> EOF
file
this-should-not.
If you want to edit files in-place (without any backups), you may also use man ed:
cat <<-'EOF' | ed -s testfile
H
,g/^[[:space:]]*/s///
,g/[[:space:]]*$/s///
,g/[[:space:]]/.d
wq
EOF
This should satisfy your needs:
cat filename | sed -n '/^\S*$/p'
Related
I am trying to append a text from a file to the another file in Linux using the grep command .
I have a file named "temp1buildVersion.properties" which contain the data
like
Project version: 1.0.5
also, I have another file named buildversion.properties which contain data
VERSION_BUILD=
I want to fetch content from temp1buildVersion.properties" after "Project version:" and append it to an existing file named "buildversion.properties"
so that output of the buildversion.properties will be
VERSION_BUILD=1.0.5
currently, I am doing using the grep command to fetch data and appending output to file " buildversion.properties "
grep 'Project version: ' /tmp/tempbuildVersion.properties | cut -d\ -f3 >> /tmp/buildversion.properties
it comes in two-line How can I append to the same line /or a specific line?
VERSION_BUILD =
1.0.5
You may use this awk:
awk -F ': ' 'FNR==NR {ver=$2; next} /^VERSION_BUILD=/ {print $0 ver}' temp1buildVersion.properties buildversion.properties > _tmp && mv _tmp buildversion.properties
VERSION_BUILD=1.0.5
Another option is using sed to append to the end of the line, e.g.
sed "/VERSION_BUILD/s/\$/$(grep 'Project version: ' /tmp/tempbuildVersion.properties | cut -d\ -f3)/" buildversion.properties
Above your command is simply placed as a command substituion in sed "/VERSION_BUILD/s/\$/$(your_cmd)/" file. You would add sed -i to update the file in place.
You can eliminate the pipeline and cut by simply using awk to isolate the version number and shorten the command a bit, e.g.
sed "/VERSION_BUILD/s/\$/$(awk '/^Project version:/{printf "%s", $NF; exit}' /tmp/tempbuildVersion.properties)/" buildversion.properties
If ed is available/acceptable.
printf '%s\n' 'r temp1buildVersion.properties' 's/^Project version: //' '1,$j' ,p Q | ed -s buildversion.properties
Change Q to w if you're ok with the output and to edit the file buildversion.properties
The script.
#!/usr/bin/env bash
ed -s "$1" <<-EOF
r $2
s/^Project version: //
1,\$j
,p
Q
EOF
You can execute with the files as the arguments.
./myscript buildversion.properties temp1buildVersion.properties
This might work for you (GNU sed):
sed -i '/VERSION_BUILD=/{x;s/.*/cat fileVersion/e;x;G;s/\n.*:\s*//}' fileBuild
Process the build file until a match on a line VERSION_BUILD=.
Swap to the hold space and insert the version file line.
Append the line from the version file to the current line and using pattern matching manipulate the line into the desired format.
I need to append a UUID ( newly generated unique for each line) to each line of a file. I would prefer to use SED or AWK for this activity and take advantage of UUIDGEN executable on my linux box. I cannot figure out how to generate the the UUID for each line and append it.
I have tried:
awk '{print system(uuidgen) $1} myfile.csv
sed -i -- 's/^/$(uuidgen)/g' myfile.csv
And many other variations that didn't work. Can this be done with SED or AWK, or should I be investigating another solution that is not shell script based?
Sincerely,
Stephen.
Using bash, this will create a file outfile.txt with a concatenated uuid:
NOTE: Please run which bash to verify the location of your copy of bash on your system. It may not be located in the same location used in the script below.
#!/usr/local/bin/bash
while IFS= read -r line
do
uuid=$(uuidgen)
echo "$line $uuid" >> outfile.txt
done < myfile.txt
myfile.txt:
john,doe
mary,jane
albert,ellis
bob,glob
fig,newton
outfile.txt
john,doe 46fb31a2-6bc5-4303-9783-85844a4a6583
mary,jane a14bb565-eea0-47cd-a999-90f84cc8e1e5
albert,ellis cfab6e8b-00e7-420b-8fe9-f7655801c91c
bob,glob 63a32fd1-3092-4a72-8c24-7b01c400820c
fig,newton 63d38ad9-5553-46a4-9f24-2e19035cc40d
Just tweaking the syntax on your attempt, something like this should work:
awk '("uuidgen" | getline uuid) > 0 {print uuid, $0} {close("uuidgen")}' myfile.csv
For example:
$ cat file
a
b
c
$ awk '("uuidgen" | getline uuid) > 0 {print uuid, $0} {close("uuidgen")}' file
52a75bc9-e632-4258-bbc6-c944ff51727a a
24c97c41-d0f4-4cc6-b0c9-81b6d89c5b77 b
76de9987-a60f-4e3b-ba5e-ae976ab53c7b c
The right solution is to use other shell commands though since the awk isn't buying you anything:
$ xargs -n 1 printf "%s %s\n" $(uuidgen) < file
763ed28c-453f-47f4-9b1b-b2f972b2cc7d a
763ed28c-453f-47f4-9b1b-b2f972b2cc7d b
763ed28c-453f-47f4-9b1b-b2f972b2cc7d c
Try this
awk '{ "uuidgen" |& getline u; print u, $1}' myfile.csv
if you want to append instead of prepend change the order of print.
Using xargs is simpler here:
paste -d " " myfile.csv <(xargs -I {} uuidgen {} < myfile.csv)
This will call uuidgen for each line of myfile.csv
You can use paste and GNU sed:
paste <(sed 's/.*/uuidgen/e' file) file
This uses the GNU execute extension e to generate a UUID per line, then paste pastes the text back together. Use the -d paste flag to change the delimiter from the default tab, to whatever you want.
I'm trying to add a line at the beginning of a file, using
echo 'time/F:x1:x2' | cat - file.txt>newfile.txt
But this produces line breaks at each line in the new file (except for after the added 'time/F:x1:x2' line). Any ideas on how to avoid this?
Use -n to disable the trailing newline:
echo -n 'time/F:x1:x2' | cat - file.txt > newfile.txt
There are other ways, too:
sed '1s|^|time/F:x1:x2|' file.txt > newfile.txt
How about
{ echo 'time/F:x1:x2'; cat file.txt; } >newfile.txt
or
sed '1i\
time/F:x1:x2' file.txt > newfile.txt
Actually you don't even need the echo and pipe if you're using bash. Just use a herestring:
<<< 'time/F:x1:x2' cat - file.txt > newfile.txt
I have file.txt with names one per line as shown below:
ABCB8
ABCC12
ABCC3
ABCC4
AHR
ALDH4A1
ALDH5A1
....
I want to grep each of these from an input.txt file.
Manually i do this one at a time as
grep "ABCB8" input.txt > output.txt
Could someone help to automatically grep all the strings in file.txt from input.txt and write it to output.txt.
You can use the -f flag as described in Bash, Linux, Need to remove lines from one file based on matching content from another file
grep -o -f file.txt input.txt > output.txt
Flag
-f FILE, --file=FILE:
Obtain patterns from FILE, one per line. The empty file
contains zero patterns, and therefore matches nothing. (-f is
specified by POSIX.)
-o, --only-matching:
Print only the matched (non-empty) parts of a matching line, with
each such part on a separate output line.
for line in `cat text.txt`; do grep $line input.txt >> output.txt; done
Contents of text.txt:
ABCB8
ABCC12
ABCC3
ABCC4
AHR
ALDH4A1
ALDH5A1
Edit:
A safer solution with while read:
cat text.txt | while read line; do grep "$line" input.txt >> output.txt; done
Edit 2:
Sample text.txt:
ABCB8
ABCB8XY
ABCC12
Sample input.txt:
You were hired to do a job; we expect you to do it.
You were hired because ABCB8 you kick ass;
we expect you to kick ass.
ABCB8XY You were hired because you can commit to a rational deadline and meet it;
ABCC12 we'll expect you to do that too.
You're not someone who needs a middle manager tracking your mouse clicks
If You don't care about the order of lines, the quick workaround would be to pipe the solution through a sort | uniq:
cat text.txt | while read line; do grep "$line" input.txt >> output.txt; done; cat output.txt | sort | uniq > output2.txt
The result is then in output.txt.
Edit 3:
cat text.txt | while read line; do grep "\<${line}\>" input.txt >> output.txt; done
Is that fine?
Say I have a script with a number of lines beginning foobar
I would like to move all of the lines to the end of the document while keeping their order
e.g. go from:
# There's a Polar Bear
# In our Frigidaire--
foobar['brangelina'] <- 2
# He likes it 'cause it's cold in there.
# With his seat in the meat
foobar['billybob'] <- 1
# And his face in the fish
to
# There's a Polar Bear
# In our Frigidaire--
# He likes it 'cause it's cold in there.
# With his seat in the meat
# And his face in the fish
foobar['brangelina'] <- 2
foobar['billybob'] <- 1
This is as far as I have gotten:
grep foobar file.txt > newfile.txt
sed -i 's/foobar//g' foo.txt
cat newfile.txt > foo.txt
This might work:
sed '/^foobar/{H;$!d;s/.*//};$G;s/\n*//' input_file
EDIT: Amended for the corner case when foobar is on the last line
This will do:
grep -v ^foobar file.txt > tmp1.txt
grep ^foobar file.txt > tmp2.txt
cat tmp1.txt tmp2.txt > newfile.txt
rm tmp1.txt tmp2.txt
The -v option returns all the lines which do not match the given pattern. The ^ marks the beginning of a line, so ^foobar matches lines beginning with foobar.
grep -v ^foobar file.txt > file1.txt
grep ^foobar file.txt > file2.txt
cat file2.txt >> file1.txt
grep -v ^foobar file.txt >newfile.txt
grep ^foobar file.txt >>newfile.txt
no need for temporary file
You can also do:
vim file.txt -c 'g/^foobar/m$' -c 'wq'
The -c switch means an Ex command follows, the g commands operates on all lines containing the pattern given, and the action is here m$ which means “move to end of file” (it preserves order). wq weans “save and exit vim”.
If this is too slow you can also prevent vim from reading vimrc:
vim -u NONE file.txt -c 'g/^foobar/m$' -c 'wq'