I am trying to insert a line of text under a line of existing text in a file as per the following excerpt:
#!/bin/bash
MATCH='Text_To_Look_For'
INSERT='Text_To_Add'
FILE='/home/user/test.txt'
echo "Modifying file..."
sed -i 's/"$MATCH/$MATCH"\n"$INSERT"/' $FILE
echo "Done."
I have tried all sorts of combinations of / ' and " but cant seem to make it work. The script will run now but not insert anything. In other formats I keep getting errors relating to:
unterminated s command
sed: -e expression #1, char 35: unknown option to `s'
You didn't say what version of sed you're using, I've seen something similar on MacOS, where the standard sed is rather poor... Worked around by installing a GNU Sed (invoked later as gsed).
Anyway, can you give a try with sed a command (which stands for append):
#!/bin/bash
MATCH='Text_To_Look_For'
INSERT='Text_To_Add'
FILE='/tmp/xxx'
echo "Modifying file..."
sed -i "/$MATCH/a $INSERT" $FILE
echo "Done."
Best regards,
Jarek
The -i option to sed should be followed by a file extension, see man sed: -i extension - Edit files in-place, saving backups with the specified extension.
It is trying to use your s command as a filename extension!
Try this:
sed -i .bak s/$MATCH/$MATCH\n$INSERT/ "$FILE"
Note that the -i option is non-standard and sed versions vary. For example you might have to remove the space:
sed -i.bak s/$MATCH/$MATCH\n$INSERT/ "$FILE"
An alternative to the -i is:
if sed s/$MATCH/$MATCH\n$INSERT/ "$FILE" > "$FILE.bak"
then
mv "$FILE.bak" "$FILE"
fi
Related
I've 95 files that looks like :
2019-10-29-18-00/dev/xx;512.00;0.4;/var/x/xx/xxx
2019-10-29-18-00/dev/xx;512.00;0.68;/xx
2019-10-29-18-00/dev/xx;512.00;1.84;/xx/xx/xx
2019-10-29-18-00/dev/xx;512.00;80.08;/opt/xx/x
2019-10-29-18-00/dev/xx;20480.00;83.44;/var/x/x
2019-10-29-18-00/dev/xx;3584.00;840.43;/var/xx/x
2019-10-30-00-00/dev/xx;2048.00;411.59;/
2019-10-30-00-00/dev/xx;7168.00;6168.09;/usr
2019-10-30-00-00/dev/xx;3072.00;1036.1;/var
2019-10-30-00-00/dev/xx;5120.00;348.72;/tmp
2019-10-30-00-00/dev/xx;20480.00;2033.19;/home
2019-10-30-12-00;/dev/xx;5120.00;348.72;/tmp
2019-10-30-12-00;/dev/hd1;20480.00;2037.62;/home
2019-10-30-12-00;/dev/xx;512.00;0.43;/xx
2019-10-30-12-00;/dev/xx;3584.00;794.39;/xx
2019-10-30-12-00;/dev/xx;512.00;0.4;/var/xx/xx/xx
2019-10-30-12-00;/dev/xx;512.00;0.68;/xx
2019-10-30-12-00;/dev/xx;512.00;1.84;/var/xx/xx
2019-10-30-12-00;/dev/xx;512.00;80.08;/opt/xx/x
2019-10-30-12-00;/dev/xx;20480.00;83.44;/var/xx/xx
2019-10-30-12-00;/dev/x;3584.00;840.43;/var/xx/xx
For some lines I've 2019-10-29-18-00/dev and for some other lines, I've 2019-10-30-12-00;/dev/
I want to add the ; before the /dev/ where it is missing, so for that I use this sed command :
sed 's/\/dev/\;\/dev/'
But How I can apply this command for each lines where the ; is missing ? I try this :
for i in $(cat /home/xxx/xxx/xxx/*.txt | grep -e "00/dev/")
do
sed 's/\/dev/\;\/dev/' $i > $i
done
But it doesn't work... Can you help me ?
Could you please try following with GNU awkif you are ok with it.
awk -i inplace '/00\/dev\//{gsub(/00\/dev\//,"/00;/dev/")} 1' *.txt
sed solution: Tested with GNU sed for few files and it worked fine.
sed -i.bak '/00\/dev/s/00\/dev/00\;\/dev/g' *.txt
This might work for you (GNU sed & parallel):
parallel -q sed -i 's#;*/dev#;/dev#' ::: *.txt
or if you prefer:
sed -i 's#;*/dev#;/dev#' *.txt
Ignore lines with ;/dev.
sed '/;\/dev/{p;d}; s^/dev^;/dev^'
The /;\/dev/ check if the line has ;/dev. If it has ;/dev do: p - print the current line and d - start from the beginning.
You can use any character with s command in sed. Also, there is no need in escaping \;, just ;.
How I can apply this command for each lines where the ; is missing ? I try this
Don't edit the same file redirecting to the same file $i > $i. Think about it. How can you re-write and read from the same file at the same time? You can't, the resulting file will be in most cases empty, as the > $i will "execute" first making the file empty, then sed $i will start running and it will read an empty file. Use a temporary file sed ... "$i" > temp.txt; mv temp.txt "$i" or use gnu extension -i sed option to edit in place.
What you want to do really is:
grep -l '00/dev/' /home/xxx/xxx/xxx/*.txt |
xargs -n1 sed -i '/;\/dev/{p;d}; s^/dev^;/dev^'
grep -l prints list of files that match the pattern, then xargs for each single one -n1 of the files executes sed which -i edits files in place.
grep for filtering can be eliminated in your case, we can accomplish the task with a single sed command:
for f in $(cat /home/xxx/xxx/xxx/*.txt)
do
[[ -f "$f" ]] && sed -Ei '/00\/dev/ s/([^;])(\/dev)/\1;\2/' "$f"
done
The easiest way would be to adjust your regex so that it's looking a bit wider than '/dev/', e.g.
sed -i -E 's|([0-9])/dev|\1;/dev|'
(note that I'm taking advantage of sed's flexible approach to delimiters on substitute. Also, -E changes the group syntax)
Alternatively, sed lets you filter which lines it handles:
sed -i '/[0-9]\/dev/ s/\/dev/;/dev/'
This uses the same substitution you already have but only applied on lines that match the filter regex
I am attempting to use sed to delete a line, read from user input, from a file whose name is stored in a variable. Right now all sed does is print the line and nothing else.
This is a code snippet of the command I am using:
FILE="/home/devosion/scripts/files/todo.db"
read DELETELINE
sed -e "$DELETELINE"'d' "$FILE"
Is there something I am missing here?
Edit: Switching out the -e option with -i fixed my woes!
You need to delimit the search.
#!/bin/bash
read -r Line
sed "/$Line/d" file
Will delete any line containing the typed input.
Bear in mind that sed matches on regex though and any special characters will be seen as such.
For example searching for 1* will actually delete lines containing any number of 1's not an actual 1 and a star.
Also bear in mind that when the variable expands, it cannot contain the delimiters or the command will break or have unexpexted results.
For example if "$Line" contained "/hello" then the sed command will fail with
sed: -e expression #1, char 4: extra characters after command.
You can either escape the / in this case or use different delimiters.
Personally i would use awk for this
awk -vLine="$Line" '!index($0,Line)' file
Which searches for an exact string and has none of the drawbacks of the sed command.
You might have success with grep instead of sed
read -p "Enter a regex to remove lines: " filter
grep -v "$filter" "$file"
Storing in-place is a little more work:
tmp=$(mktemp)
grep -v "$filter" "$file" > "$tmp" && mv "$tmp" "$file"
or, with sponge (apt install moreutils)
grep -v "$filter" "$file" | sponge "$file"
Note: try to get out of the habit of using ALLCAPSVARS: one day you'll accidentally use PATH=... and then wonder why your script is broken.
I found this, it allows for a range deletion with variables:
#!/bin/bash
lastline=$(whatever you need to do to find the last line)` //or any variation
lines="1,$lastline"
sed -i "$lines"'d' yourfile
keeps it all one util.
Please try this :
sed -i "${DELETELINE}d" $FILE
I am writing a shell script for linux which takes argument as port no.
inside file following is a line which needs to be updated:
define('NO_OF_PORTS',10);
I need to replace that 10 by the argument passed.
But this should be dynamic, like next time I pass new port no it must get updated.
Using sed:
s="define('NO_OF_PORTS',10);"
n=25
sed "s/\('NO_OF_PORTS',\)[0-9]*/\1$n/" <<< "$s"
define('NO_OF_PORTS',25);
To change inline in the file use:
sed -i.bak "s/\('NO_OF_PORTS',\)[0-9]*/\1$n/" file
You can use sed in the script to edit the file
sed -i s/NO_OF_PORTS\',[0-9]*/NO_OF_PORTS\',$1/ $2
1.txt has
define('NO_OF_PORTS',19)
shell script
#!/bin/sh
echo $1
sed -i -r '/NO_OF_PORTS/ s/'[0-9]+'/'$1'/g' 1.txt
run
linux:/home/test # ./replace_port.sh 78
linux:/home/test # cat 1.txt
define('NO_OF_PORTS',78)
I have the following bash code which loops through a text file, line by line .. im trying to prefix the work 'prefix' to each line but instead am getting this error:
rob#laptop:~/Desktop$ ./appendToFile.sh stusers.txt kp
stusers.txt
kp
./appendToFile.sh: line 11: /bin/sed: Argument list too long
115000_210org#house.com,passw0rd
This is the bash script ..
#!/bin/bash
file=$1
string=$2
echo "$file"
echo "$string"
for line in `cat $file`
do
sed -e 's/^/prefix/' $line
echo "$line"
done < $file
What am i doing wrong here?
Update:
Performing head on file dumps all the lines onto a single line of the terminal, probably related?
rob#laptop:~/Desktop$ head stusers.txt
rob#laptop:~/Desktop$ ouse.com,passw0rd
a one-line awk command should do the trick also:
awk '{print "prefix" $0}' file
Concerning your original error:
./appendToFile.sh: line 11: /bin/sed: Argument list too long
The problem is with this line of code:
sed -e 's/^/prefix/' $line
$line in this context is file name that sed is running against. To correct your code you should fix this line as such:
echo $line | sed -e 's/^/prefix/'
(Also note that your original code should not have the < $file at the end.)
William Pursell addresses this issue correctly in both of his suggestions.
However, I believe you have correctly identified that there is an issue with your original text file. dos2unix will not correct this issue, as it only strips the carriage returns Windows sticks on the end of lines. (However, if you are attempting to read a Linux file in Windows, you would get a mammoth line with no returns.)
Assuming that it is not an issue with the end of line characters in your text file, William Pursell's, Andy Lester's, or nullrevolution's answers will work.
A variation on the while read... suggestion:
while read -r line; do echo "PREFIX " $line; done < $file
This could be run directly from the shell (no need for a batch / script file):
while read -r line; do echo "kp" $line; done < stusers.txt
The entire loop can be replaced by a single sed command that operates on the entire file:
sed -e 's/^/prefix/' $file
A Perl way to do it would be:
perl -p -e's/^/prefix' filename
or
perl -p -e'$_ = "prefix $_"' filename
In either case, that reads from filename and prints the prefixed lines to STDOUT.
If you add a -i flag, then Perl will modify the file in place. You can also specify multiple filenames and Perl will magically do all of them.
Instead of the for loop, it is more appropriate to use while read...:
while read -r line; do
do
echo "$line" | sed -e 's/^/prefix/'
done < $file
But you would be much better off with the simpler:
sed -e 's/^/prefix/' $file
Use sed. Just change the word prefix.
sed -e 's/^/prefix/' file.ext
If you want to save the output in another file
sed -e 's/^/prefix/' file.ext > file_new.ext
You don't need sed, just concatenate the strings in the echo command
while IFS= read -r line; do
echo "prefix$line"
done < filename
Your loop iterates over each word in the file:
for line in `cat file`; ...
sed -i '1a\
Your Text' file1 file2 file3
A solution without sed/awk and while loops:
xargs -n1 printf "$prefix%s\n" < "$file"
Is it possible to search in a file using shell and then replace a value? When I install a service I would like to be able to search out a variable in a config file and then replace/insert my own settings in that value.
Sure, you can do this using sed or awk. sed example:
sed -i 's/Andrew/James/g' /home/oleksandr/names.txt
You can use sed to perform search/replace. I usually do this from a bash shell script, and move the original file containing values to be substituted to a new name, and run sed writing the output to my original file name like this:
#!/bin/bash
mv myfile.txt myfile.txt.in
sed -e 's/PatternToBeReplaced/Replacement/g' myfile.txt.in > myfile.txt.
If you don't specify an output, the replacement will go to stdout.
sed -i 's/variable/replacement/g' *.conf
You can use sed to do this:
sed -i 's/toreplace/yoursetting/' configfile
sed is probably available on every unix like system out there. If you want to replace more than one occurence you can add a g to the s-command:
sed -i 's/toreplace/yoursetting/g' configfile
Be careful since this can completely destroy your configfile if you don't specify your toreplace-value correctly. sed also supports regular expressions in searching and replacing.
Look at the UNIX power tools awk, sed, grep and in-place edit of files with Perl.
filepath="/var/start/system/dir1"
searchstring="test"
replacestring="test01"
i=0;
for file in $(grep -l -R $searchstring $filepath)
do
cp $file $file.bak
sed -e "s/$searchstring/$replacestring/ig" $file > tempfile.tmp
mv tempfile.tmp $file
let i++;
echo "Modified: " $file
done
Generally a tool like awk or sed are used for this.
$ sed -i 's/ugly/beautiful/g' /home/bruno/old-friends/sue.txt