$ cat example.txt
Yields:
example
test
example
I want to remove 'test' string from this file.
$ grep -v test example.txt > example.txt
$ cat example.txt
$
The below works, but I have a feeling there is a better way!
$ grep -v test example.txt > example.txt.tmp;mv example.txt.tmp example.txt
$ cat example.txt
example
example
Worth noting that this is going to be on a file with over 10,000 lines.
Cheers
You could use sed,
sed -i '/test/d' example.txt
-i saves the changes made to that file. so you don't need to use a redirection operator.
-i[SUFFIX], --in-place[=SUFFIX]
edit files in place (makes backup if SUFFIX supplied)
You're doing it the right way but use an && before the mv to make sure the grep succeeded or you'll zap your original file:
grep -F -v test example.txt > example.txt.tmp && mv example.txt.tmp example.txt
I also added the -F options since you said you want to remove a string, not a regexp.
You COULD use sed -i but then you need to worry about figuring out and/or escaping sed delimiters and sed does not support searching for strings so you'd need to try to escape every possible combination of regexp characters in your search string to try to make sed treat them as literal chars (a process you CANNOT automate due to the position-sensitive nature of regexp chars) and all it'd save you is manually naming your tmp file since sed uses one internally anyway.
Oh, one other option - you could use GNU awk 4.* with "inplace editing". It also uses a tmp file internally like sed does but it does support string operations so you don't need to try to escape RE metacharacters and it doesn't have delimiters as part of the syntax to worry about:
awk -i inplace -v rmv="test" '!index($0,rmv)' example.txt
Any grep/sed/awk solution will run the in blink of an eye on a 10,000 line file.
Related
Specify the command that removes multiple spaces from a text file, leaving a single space in their place. Extra requirements : Original file to be modified.
Managed to pull out those 3 commands:
awk '{$2=$2};1' filename.txt
tr -s '[:space:]' < filename.txt > filename.new && mv filename.new filename.txt
sed -i 's/\s\+/ /g' filename.txt
Not sure if using a 'temporary file' is the best way to do the trick. Is there any more efficient way to do the problem ? Doesn't matter if it is tr / sed / awk or anything else, you can post all of them.
Example input:
I'm just giving spaces
Output :
I'm just giving spaces
Edit: Still looking for more answers
I'd use ed over the non-standard sed -i (And non-portable RE in your example) if you want to alter the original file:
printf "%s\n" '1,$s/[[:space:]]\{2,\}/ /g' w | ed -s filename.txt
or with perl:
perl -pi -e 's/\s{2,}/ /g' filename.txt
The {2,} regular expression construct (\{2,\} for POSIX Basic Regular Expressions like sed and ed use) matches 2 or more of the previous token.
Both of these match any whitespace characters, not just space, because that's how your examples work. If the goal is to only compress multiple spaces, not spaces + tabs, switch out the [[:space:]] and \s for just a single space.
(Anything that modifies a file "in place", be it ed, sed -i, perl -i, or a regular editor, has a good chance that it's going to be using a temporary file under the hood, by the way. They just handle it for you so you don't have to do it manually like with your tr example.)
I need to replace first 4 header lines of only selected 250 erlang files (with extension .erl), but there are 400 erlang files in total in the directory+subdirectories, I need to avoid modifying the files which doesn't need the change.
I've the list of file names that are to be modified, but don't know how to make my linux command to make use of them.
sed -i '1s#.*#%% This Source Code Form is subject to the terms of the Mozilla Public#' *.erl
sed -i '2s#.*#%% License, v. 2.0. If a copy of the MPL was not distributed with this file,#' *.erl
sed -i '3s#.*#%% You can obtain one at http://mozilla.org/MPL/2.0/.#' *.erl
sed -i '4s#.*##' *.erl
in the above commands instead of passing *.erl I want to pass those list of file names which I need to modify, doing that one by one will take me more than 3 days to complete it.
Is there any way to do this?
Iterate over the shortlisted file names using awk and use xargs to execute the sed. You can execute multiple sed commands to a file using -e option.
awk '{print $1}' your_shortlisted_file_lists | xargs sed -i -e first_sed -e second_sed $1
xargs gets the file name from awk in a $1 variable.
Try this:
< file_list.txt xargs -1 sed -i -e 'first_cmd' -e 'second_cmd' ...
Not answering your question but a suggestion for improvement. Four sed commands for replacing header is inefficient. I would instead write the new header into a file and do the following
sed -i -e '1,3d' -e '4{r header' -e 'd}' file
will replace the first four lines of the file with header.
Another concern with your current s### approach is you have to watch for special chars \, & and your delimiter # in the text you are replacing.
You can apply the sed c (for change) command to each file of your list :
while read file; do
sed -i '1,4 c\
%% This Source Code Form is subject to the terms of the Mozilla Public\
%% License, v. 2.0. If a copy of the MPL was not distributed with this file,\
%% You can obtain one at http://mozilla.org/MPL/2.0/.\
' "$file"
done < filelist
Let's say you have a file called file_list.txt with all file names as content:
file1.txt
file2.txt
file3.txt
file4.txt
You can simply read all lines into a variable (here: files) and then iterate through each one:
files=`cat file_list.txt`
for file in $files; do
echo "do something with $file"
done
I am attempting to use sed to delete a line, read from user input, from a file whose name is stored in a variable. Right now all sed does is print the line and nothing else.
This is a code snippet of the command I am using:
FILE="/home/devosion/scripts/files/todo.db"
read DELETELINE
sed -e "$DELETELINE"'d' "$FILE"
Is there something I am missing here?
Edit: Switching out the -e option with -i fixed my woes!
You need to delimit the search.
#!/bin/bash
read -r Line
sed "/$Line/d" file
Will delete any line containing the typed input.
Bear in mind that sed matches on regex though and any special characters will be seen as such.
For example searching for 1* will actually delete lines containing any number of 1's not an actual 1 and a star.
Also bear in mind that when the variable expands, it cannot contain the delimiters or the command will break or have unexpexted results.
For example if "$Line" contained "/hello" then the sed command will fail with
sed: -e expression #1, char 4: extra characters after command.
You can either escape the / in this case or use different delimiters.
Personally i would use awk for this
awk -vLine="$Line" '!index($0,Line)' file
Which searches for an exact string and has none of the drawbacks of the sed command.
You might have success with grep instead of sed
read -p "Enter a regex to remove lines: " filter
grep -v "$filter" "$file"
Storing in-place is a little more work:
tmp=$(mktemp)
grep -v "$filter" "$file" > "$tmp" && mv "$tmp" "$file"
or, with sponge (apt install moreutils)
grep -v "$filter" "$file" | sponge "$file"
Note: try to get out of the habit of using ALLCAPSVARS: one day you'll accidentally use PATH=... and then wonder why your script is broken.
I found this, it allows for a range deletion with variables:
#!/bin/bash
lastline=$(whatever you need to do to find the last line)` //or any variation
lines="1,$lastline"
sed -i "$lines"'d' yourfile
keeps it all one util.
Please try this :
sed -i "${DELETELINE}d" $FILE
I have some strings in a textfile that look like this:
[img:3gso40ßf]
I want to replace them to look like normal BBCode:
[img]
How can I do that with sed? I tried this one but it doesn't do anything:
sed -i 's/^[img:.*]/[img]/g' file.txt
Escape those square brackets
Square brackets are metacharacters: they have a special meaning in POSIX regular expressions. If you mean [ and ] literally, you need to escape those characters in your regexp:
$ sed -i .bak 's/\[img:.*\]/\[img\]/g' file.txt
Use [^]]* instead of .*
Because * is greedy, .* will capture more than what you want; see Jidder's comment. To fix this, use [^]]*, which captures a sequence of characters up to (but excluding) the first ] encountered.
$ sed -i .bak 's/\[img:.[^]]\]/\[img\]/g' file.txt
Are you using an incorrect sed -i syntax?
(Thanks to j.a. for his comment.)
Depending on the flavour of sed that you're using, you may be allowed to use sed -i without specifying any <extension> argument, as in
$ sed -i 's/foo/bar/' file.txt
However, in other versions of sed, such as the one that ships with Mac OS X, sed -i expects a mandatory <extension> argument, as in
$ sed -i .bak 's/foo/bar/' file.txt
If you omit that extension argument (.bak, here), you'll get a syntax error. You should check out your sed's man page to figure out whether that argument is optional or mandatory.
Match a specific number of characters
Is there a way to tell sed that there are always 8 random characters after the colon?
Yes, there is. If the number of characters between the colon and the closing square bracket is always the same (8, here), you can make your command more specific:
$ sed -i .bak 's/\[img:[^]]\{8\}\]/\[img\]/g' file.txt
Example
# create some content in file.txt
$ printf "[img:3gso40ßf]\nfoo [img:4t5457th]\n" > file.txt
# inspect the file
$ cat file.txt
[img:3gso40ßf]
foo [img:4t5457th]
# carry out the substitutions
$ sed -i .bak 's/\[img:[^]]\{8\}\]/\[img\]/g' file.txt
# inspect the file again and make sure everything went smoothly
$ cat file.txt
[img]
foo [img]
# if you're happy, delete the backup that sed created
$ rm file.txt.bak
I need to remove the character : from a file. Ex: I have numbers in the following format:
b3:07:4d
I want them to be like:
b3074d
I am using the following command:
grep ':' source.txt | sed -e 's/://' > des.txt
I am new to Linux. The file is quite big & I want to make sure I'm using the write command.
You can do without the grep:
sed -e 's/://g' source.txt > des.txt
The -i option edits the file in place.
sed -i 's/://' source.txt
the first part isn't right as it'll completely omit lines which don't contain :
below is untested but should be right. The g at end of the regex is for global, means it should get them all.
sed -e 's/://g' source.txt > out.txt
updated to better syntax from Jon Lin's answer but you still want the /g I would think