I would like to achieve this procedure.
I have this command:
sed -i -e 's/few/asd/g' /usr/local/sbin/.myappenv
This command works if in the file .myappenv there is a few string text. But this command fails to simply create the asd text wheter or not the few is found.
So, if there is the few text, it should replace it, if few text is missing, just create it on the first line.
EDIT_:
It should create the string on the first line of the document .myappenv, it should only be created if there is no coincidence.
The file .myappenv should contain =>
asd
If the file is already populated with few just replace it =>
asd
if there is the few text, it should replace it, if few text is missing, just create it on the first line.
That's an if.
if <file has text>; then <replace text>; else <add text to first line>; fi
or in bash:
file=/usr/local/sbin/.myappenv
if grep -q few "$file"; then
sed 's/few/asd/' "$file"
else
{
echo asd
cat "$file"
} > "$file".tmp
mv "$file".tmp "$file"
fi
How to test if string exists in file with Bash? https://unix.stackexchange.com/questions/99350/how-to-insert-text-before-the-first-line-of-a-file and https://mywiki.wooledge.org/BashGuide/TestsAndConditionals . You might interest yourself in some automation methods, like ansible lineinfile or chezmoi depending on the goal.
Using a single awk you can use this:
awk 'FNR == NR {if (/few/) {n = 1; nextfile}; next}
FNR == 1 && !n {print "asd"} n {gsub(/few/, "asd")} 1' file file
This might work for you (GNU sed):
sed 'H;$!d;x;/few/!s/^/asd/;t;s//asd/;s/.//' file
Make a copy of the file in the hold space (first line will have a prepended newline).
At the end of the file, if few is not matched, insert the literal asd.
Otherwise, replace the first occurrence of few with asd and remove the introduced newline at the start of the file.
To replace all occurrences of few with asd use:
sed 'H;$!d;x;/few/!s/^/asd/;t;s//asd/g;s/.//' file
Alternative:
grep -q few file && sed 's/few/asd/' file || (echo asd; cat file)
Related
I'm making a script (in vim) that goes through a file and looks for a specific line. If the line matches, I want it to be moved to the top of the file.
The file looks something like this:
6596628c9cbab49b80d6a07d0304377768f5114e7f8b21edffa820aab1c508be, ./favicon.ico
150a04dd76f733e5ef05ece49de115d05f71efa8e73025e015dd4e0fb3217553, ./about.php
28acfc4b0d378c22a2c0e4913cae3d15aef9b21938de81be92f74aef85b0cc0e, ./info.php
67976bbbd1b62a00d454da3a9f95e72d97d0fd156d4c65a12707f2602cbea582, ./missing.php
6ed318718f4cc617c82121db7cde54188eac6f89c355f0bfe9d198218de7fffc, ./browse.php
abd277cc3453be980bb48cbffe9d1f7422ca1ef4bc0b7d035fda87cea4d55cbc, ./composer.phar
73ac79eccac12120dc601cd6cce1282a1d8a920d440d3d1141d257db1ed4b0f0, ./search.php
f412aabd74f4c99bd32c5e534132c565f52c2bd32fbf7f629eb5a4495ac46351, ./index.php
c2d49a4873088fbe635d8653494f7f1425b6ad9f55d63ee4de52170d8a8d01b8, ./content/style.css
18e7d61367d80bc125b309ac002bb3946c5e7ba419ef59537afc939eff799dfd, ./content/logo.png
d8da15f62d55641320f7e7c21d9be86db6d81f7667bbd35c738b4c917cad3ce9, ./robots.txt
How would I be able to move the content on line 8 (index.php) to the top so it looks like this:
f412aabd74f4c99bd32c5e534132c565f52c2bd32fbf7f629eb5a4495ac46351, ./index.php
6596628c9cbab49b80d6a07d0304377768f5114e7f8b21edffa820aab1c508be, ./favicon.ico
150a04dd76f733e5ef05ece49de115d05f71efa8e73025e015dd4e0fb3217553, ./about.php
28acfc4b0d378c22a2c0e4913cae3d15aef9b21938de81be92f74aef85b0cc0e, ./info.php
67976bbbd1b62a00d454da3a9f95e72d97d0fd156d4c65a12707f2602cbea582, ./missing.php
6ed318718f4cc617c82121db7cde54188eac6f89c355f0bfe9d198218de7fffc, ./browse.php
abd277cc3453be980bb48cbffe9d1f7422ca1ef4bc0b7d035fda87cea4d55cbc, ./composer.phar
73ac79eccac12120dc601cd6cce1282a1d8a920d440d3d1141d257db1ed4b0f0, ./search.php
c2d49a4873088fbe635d8653494f7f1425b6ad9f55d63ee4de52170d8a8d01b8, ./content/style.css
18e7d61367d80bc125b309ac002bb3946c5e7ba419ef59537afc939eff799dfd, ./content/logo.png
d8da15f62d55641320f7e7c21d9be86db6d81f7667bbd35c738b4c917cad3ce9, ./robots.txt
(The file contains about 9000 lines)
How can this be done most efficiently?
I think can use sed to do the operations needed:
This command will be used to get the content in the file at specific line
sed -n "$LINE_NUMBER"p "$FILENAME"
This command will be used to delete the content of the file at specified line
sed -i.bak -e "$LINE_NUMBER"'d' "$FILENAME"
And finally, this command will be used to append a string to the top of a file
sed -i -e "1i${LINE_CONTENT}" "$FILENAME"
Combine the 3 commands together, you can perform the operation mentioned in your question (test.log is the file that contain your sample input)
FILENAME="test.log"
LINE_NUMBER=8
LINE_CONTENT=$(sed -n "$LINE_NUMBER"p "$FILENAME")
sed -i.bak -e "$LINE_NUMBER"'d' "$FILENAME"
sed -i -e "1i${LINE_CONTENT}" "$FILENAME"
With ed, this will do in-place editing as well:
printf '8m0\nwq\n' | ed -s ip.txt -
Or, if you don't know line number:
printf '/index\.php/m0\nwq\n' | ed -s ip.txt -
Supposing your file name is test.txt, you can try this:
(Please backup your file first)
grep "./index.php" test.txt > out && grep -v "./index.php" test.txt >> out && mv out test.txt
To do this job reading the input file just once without reading the whole file into memory, and using any awk in any shell on every Unix box all you need is either of these:
awk 'f{print; next} NR==8{print $0 buf; f=1} {buf=buf ORS $0}' file
awk 'f{print; next} $2=="./index.php"{print $0 buf; f=1} {buf=buf ORS $0}' file
If you want to move the 8th line to the top:
$ awk 'f{print; next} NR==8{print $0 buf; f=1} {buf=buf ORS $0}' file
f412aabd74f4c99bd32c5e534132c565f52c2bd32fbf7f629eb5a4495ac46351, ./index.php
6596628c9cbab49b80d6a07d0304377768f5114e7f8b21edffa820aab1c508be, ./favicon.ico
150a04dd76f733e5ef05ece49de115d05f71efa8e73025e015dd4e0fb3217553, ./about.php
28acfc4b0d378c22a2c0e4913cae3d15aef9b21938de81be92f74aef85b0cc0e, ./info.php
67976bbbd1b62a00d454da3a9f95e72d97d0fd156d4c65a12707f2602cbea582, ./missing.php
6ed318718f4cc617c82121db7cde54188eac6f89c355f0bfe9d198218de7fffc, ./browse.php
abd277cc3453be980bb48cbffe9d1f7422ca1ef4bc0b7d035fda87cea4d55cbc, ./composer.phar
73ac79eccac12120dc601cd6cce1282a1d8a920d440d3d1141d257db1ed4b0f0, ./search.php
c2d49a4873088fbe635d8653494f7f1425b6ad9f55d63ee4de52170d8a8d01b8, ./content/style.css
18e7d61367d80bc125b309ac002bb3946c5e7ba419ef59537afc939eff799dfd, ./content/logo.png
d8da15f62d55641320f7e7c21d9be86db6d81f7667bbd35c738b4c917cad3ce9, ./robots.txt
If you want to move the line containing "index" to the top:
$ awk 'f{print; next} $2=="./index.php"{print $0 buf; f=1} {buf=buf ORS $0}' file
f412aabd74f4c99bd32c5e534132c565f52c2bd32fbf7f629eb5a4495ac46351, ./index.php
6596628c9cbab49b80d6a07d0304377768f5114e7f8b21edffa820aab1c508be, ./favicon.ico
150a04dd76f733e5ef05ece49de115d05f71efa8e73025e015dd4e0fb3217553, ./about.php
28acfc4b0d378c22a2c0e4913cae3d15aef9b21938de81be92f74aef85b0cc0e, ./info.php
67976bbbd1b62a00d454da3a9f95e72d97d0fd156d4c65a12707f2602cbea582, ./missing.php
6ed318718f4cc617c82121db7cde54188eac6f89c355f0bfe9d198218de7fffc, ./browse.php
abd277cc3453be980bb48cbffe9d1f7422ca1ef4bc0b7d035fda87cea4d55cbc, ./composer.phar
73ac79eccac12120dc601cd6cce1282a1d8a920d440d3d1141d257db1ed4b0f0, ./search.php
c2d49a4873088fbe635d8653494f7f1425b6ad9f55d63ee4de52170d8a8d01b8, ./content/style.css
18e7d61367d80bc125b309ac002bb3946c5e7ba419ef59537afc939eff799dfd, ./content/logo.png
d8da15f62d55641320f7e7c21d9be86db6d81f7667bbd35c738b4c917cad3ce9, ./robots.txt
To modify the original file if you have GNU awk:
awk -i inplace '...script...' file
or with any awk:
awk '...script...' file > tmp && mv tmp file
This might work for you (GNU sed):
sed -E '1,8{H;8!d;p;x;s/^(.)(.*)\1.*/\2/}' file
Create a range of lines in the hold space from the first to the line number to be inserted at the start of the file (in the example above line 8).
When the current line is the desired line, print it and then swap to the hold space, remove the newline introduced at the start of the range and the last line added (the desired line) and print the remainder too.
All other lines will be printed as normal.
Alternative:
sed -E '1{:a;N;8!ba;s/(.*)(\n)(.*)/\3\2\1/}' file
Yet another quirky solution:
sed '1,7{H;d};8{p;x;D}' file
I would like to match all lines from a file containing a word, and take all lines under until coming two two newline characters in a row.
I have the following sed code to cut and paste specific lines, but not subsequent lines:
sed 's|.*|/\\<&\\>/{w results\nd}|' teststring | sed -file.bak -f - testfile
How could I modify this to take all subsequent lines?
For example, say I wanted to match lines with 'dog', the following should take the first 3 lines of the 5:
The best kind of an animal is a dog, for sure
-man's best friend
-related to wolves
Racoons are not cute
Is there a way to do this?
This should do:
awk '/dog/ {f=1} /^$/ {f=0} f {print > "new"} !f {print > "tmp"}' file && mv tmp file
It will set f to true if word dog is found, then if a blank line is found set f to false.
If f is true, print to new file.
If f is false, print to tmp file.
Copy tmp file to original file
Edit: Can be shorten some:
awk '/dog/ {f=1} /^$/ {f=0} {print > (f?"new":"tmp")}' file && mv tmp file
Edit2: as requested add space for every section in the new file:
awk '/dog/ {f=1;print ""> "new"} /^$/ {f=0} {print > (f?"new":"tmp")}' file && mv tmp file
If the original files does contains tabs or spaces instead of just a blank line after each dog section, change from /^$/ to /^[ \t]*$/
This might work for you (GNU sed):
sed 's|.*|/\\<&\\>/ba|' stringFile |
sed -f - -e 'b;:a;w resultFile' -e 'n;/^$/!ba' file
Build a set of regexps from the stringFile and send matches to :a. Then write the matched line and any further lines until an empty line (or end of file) to the resultFile.
N.B. The results could be sent directly to resultFile,using:
sed 's#.*#/\\<&\\>/ba#' stringFile |
sed -nf - -e 'b;:a;p;n;/^$/!ba' file > resultFile
To cut the matches from the original file use:
sed 's|.*|/\\<&\\>/ba|' stringFile |
sed -f - -e 'b;:a;N;/\n\s*$/!ba;w resultFile' -e 's/.*//p;d' file
Is this what you're trying to do?
$ awk -v RS= '/dog/' file
The best kind of an animal is a dog, for sure
-man's best friend
-related to wolves
Could you please try following.
awk '/dog/{count="";found=1} found && ++count<4' Input_file > temp && mv temp Input_file
What's the best way to find and replace the values of all keys in a file without precising the keys ?
Example :
originalFile.txt ------------------------------------> newFile.txt
key1=a key1=$key1
key2=b key2=$key2
key3=c key3=$key3
I came up with this but I'm not able to do it dynamically for any new added key :
sed '/^key1/s/=.*$/= $key1/' newFile.txt
awk -vFS="=" -v OFS="=" '$2="$"$1' originalFile.txt
Using sed: using back referencing capture the left of = in \1 and right of = in \2:
sed -r 's/(^[^=]+)*=(.*)/\1=$\1/g' originalFile.txt
key1=$key1
key2=$key2
key3=$key3
This should do it:
sed -re 's/^(key[[:digit:]]+)=.*$/\1=\$\1/' originalFile.txt
It works because (since the -r options is used) a pair of parentheses captures its match. In the substitution pattern, the \1 reproduces the text matched.
this script extract the password from env and include it in the new file:
while read line; do
password=$(echo $line | sed "s/\(.*\)=.*/\1/g")
newpassword="$(printenv $password)"
if [ -n "$newpassword" ]; then
echo "$password=$newpassword"
else
echo $line
fi
done <originalFile.txt >newFile.txt
I want to add a text to the end of the first line of a file using a bash script.
The file is /etc/cmdline.txt which does not allow line breaks and needs new commands seperated by a blank, so text i want to add realy needs to be in first line.
What i got so far is:
line=' bcm2708.w1_gpio_pin=20'
file=/boot/cmdline.txt
if ! grep -q -x -F -e "$line" <"$file"; then
printf '%s' "$line\n" >>"$file"
fi
But that appends the text after the line break of the first line, so the result is wrong.
I either need to trim the file contend, add my text and a line feed or somehow just add it to first line of file not touching the rest somehow, but my knowledge of bash scripts is not good enough to find a solution here, and all the examples i find online add beginning/end of every line in a file, not just the first line.
This sed command will add 123 to end of first line of your file.
sed ' 1 s/.*/&123/' yourfile.txt
also
sed '1 s/$/ 123/' yourfile.txt
For appending result to the same file you have to use -i switch :
sed -i ' 1 s/.*/&123/' yourfile.txt
This is a solution to add "ok" at the first line on /etc/passwd, I think you can use this in your script with a little bit of 'tuning' :
$ awk 'NR==1{printf "%s %s\n", $0, "ok"}' /etc/passwd
root:x:0:0:root:/root:/bin/bash ok
To edit a file, you can use ed, the standard editor:
line=' bcm2708.w1_gpio_pin=20'
file=/boot/cmdline.txt
if ! grep -q -x -F -e "$line" <"$file"; then
ed -s "$file" < <(printf '%s\n' 1 a "$line" . 1,2j w q)
fi
ed's commands:
1: go to line 1
a: append (this will insert after the current line)
We're in insert mode and we're inserting the expansion of $line
.: stop insert mode
1,2j join lines 1 and 2
w: write
q: quit
This can be used to append a variable to the first line of input:
awk -v suffix="$suffix" '{print NR==1 ? $0 suffix : $0}'
This will work even if the variable could potentially contain regex formatting characters.
Example:
suffix=' [first line]'
cat input.txt | awk -v suffix="$suffix" '{print NR==1 ? $0 suffix : $0}' > output.txt
input.txt:
Line 1
Line 2
Line 3
output.txt:
Line 1 [first line]
Line 2
Line 3
I am new to linux programming
I have a file which contains extra line in file.
amit>more file_20131130111111.ctl
file_20131130111111.psv|224
amit>
I want to move this type of files to another location.
And also if i can remove those line after moving it to reprocess.
With bash, you can test if the last line of a file is empty like this:
[[ -z $(tail -1 filename) ]] && echo empty
And you can remove an empty last line with:
sed -i '${/^$/d}' filename
Another possibility with awk, we check if the last line is an empty line or an empty line with spaces:
awk 'END{if($0 ~ /^$|^[ ]+$/) print "empty"}' your_file
An example of use
for file in $files
do
result="$(awk 'END{if($0 ~ /^$|^[ ]+$/) print "empty"}' $file)"
if [[ "$result" == "empty" ]]
then
mv $file /tmp/$file
sed -i '${/^$/d};${/^[ ]*$/d}' /tmp/$file
fi
done
EDIT: It seems like you want to move all files in current folder which contain a blank line to a new folder WITHOUT that blank line. Am I correct? Do do that, you can use this:
for i in $(grep -lE '[[:space:]]' *); do awk 'NF > 0' $i > newlocationfolder/$i; done