I have an enormous file, about 10 MB, and it has about 175,000 lines. I tried truncated it like this:
sed '500,175000d' <file-name.data>
I reopen the file, and all of the lines are still there! I tested this with other files and it works. For some reason the .data extension doesn't work? How do I delete these lines?
You need to either redirect the output to a new file like
sed '500,175000d' file-name.data >newFile
or use the edit in place option which rewrites the input file
sed -i '500,175000d' file-name.data
as pointed out by Wintermute
Edit:
A faster sed would be just
sed -i '500q' file-name.data # prints 1-500 and quits after line 500
Related
I have a large text file that I would like to divide into segments and use sed to delete certain patterns in place. I would like to do this in a single command line using a pipe. For example:
sed -n 1,10p <text file> | sed -i 's/<pattern to remove>//'
The code above attempts to take the first 10 lines of the text file and remove the patterns from the 10 lines in place. The resulting text file should have the first 10 lines modified. The code above doesn't work because the second command after the pipe requires a input file. Please help!
Something like
sed -i '1,10s/pattern//' foo.txt
though for in place editing of a file I prefer ed or perl instead of relying on a non standard sed extension like -i.
This seems to do what you're asking ....
sed -ni '1,10s/pattern//p' file
I'm trying to use sed but can't get it to work. My system is Ubuntu 16.04.2 with GNU sed 4.2.2
I have a folder with numerous text files that I want to edit. From a terminal in that folder I have tried the following (separately - one command at a time).
To remove blank lines from all the files: sed -i '/^$/d' *txt
In case the lines include white spaces: sed -i '/^\s*$/d' *txt
To remove line 4: sed -i '4d' *.txt
In each case, there is no error message in the terminal but the changes do not happen. I've tried the same commands but for an individual file rather than all, so with the filename instead of *.txt, but still no changes achieved.
The only sign of sed being active at all is if I don't do the -i but ask for a new file: sed '4d' fred.txt > fred2.text
The new file fred2 was created - but with line 4 still there!
What am I missing? How can I get sed to actually carry out these commands?
Thanks for any help.
After looking at the files in more detail, and doing some experimenting, I discovered what the problem was: The text files I was trying to edit with sed had CR line-end characters. When I changed this to LF the sed commands worked fine.
Thank you to those who made suggestions.
I have a file contained \n hidden behind each line:
input:
s3741206\n
s2561284\n
s4411364\n
s2516482\n
s2071534\n
s2074633\n
s7856856\n
s11957134\n
s682333\n
s9378200\n
s1862626\n
I want to remove \n behind
desired output:
s3741206
s2561284
s4411364
s2516482
s2071534
s2074633
s7856856
s11957134
s682333
s9378200
s1862626
however, I try this:
tr -d '\n' < file1 > file2
but it goes like below without space and new line
s3741206s2561284s4411364s2516482s2071534s2074633s7856856s11957134s682333s9378200s1862626
I also try sed $'s/\n//g' -i file1 and it doesn't work in mac os.
Thank you.
This is a possible solution using sed:
sed 's/\\n/ /g'
with awk
awk '{sub(/\\n/,"")} 1' < file1 > file2
What you are describing so far in your question+comments doesn't make sense. How can you have a multi-line file with a hidden newline character at the end of each line? What you show as your input file:
s3741206\n
s2561284\n
s4411364\n
etc.
where each "\n" above according to your comment is a single newline character "\n" is impossible. If those "\n"s were newline characters then your file would simply look like:
s3741206
s2561284
s4411364
etc.
There's really only 2 possibilities I can think of:
You are wrongly interpreting what you are seeing in your input file
and/or using the wrong terminology and you actually DO have \r\n
at the end of every line. Run cat -v file to see the \rs as
^Ms and run dos2unix or similar (e.g. sed 's/\r$//' file) to
remove the \rs - you do not want to remove the \ns or you will
no longer have a POSIX text file and so POSIX tools will exhibit
undefined behavior when run on it. If that doesn't work for you then
copy/paste the output of cat -v file into your question so we can
see for sure what is in your file.
Or:
It's also entirely possible that your file is a perfectly fine POSIX
text file as-is and you are incorrectly assuming you will have a
problem for some reason so also include in your question a
description of the actual problem you are having, include an example
of the command you are executing on that input file and the output
you are getting and the output you expected to get.
You could use bash-native string substitution
$ cat /tmp/newline
s3741206\n
s2561284\n
s4411364\n
s2516482\n
s2071534\n
s2074633\n
s7856856\n
s11957134\n
s682333\n
s9378200\n
s1862626\n
$ for LINE in $(cat /tmp/newline); do echo "${LINE%\\n}"; done
s3741206
s2561284
s4411364
s2516482
s2071534
s2074633
s7856856
s11957134
s682333
s9378200
s1862626
I'm using a diff command and it's printing out to a file. The file keeps getting an extra line in the end that I don't need to appear. How can I prevent it from being there?
The command is as follows:
diff -b <(grep -B 2 -A 1 'bedrock.local' /Applications/MAMP/conf/apache/httpd.conf) /Applications/MAMP/conf/apache/httpd.conf > test.txt
The file being used is here (thought I don't think it matters): http://yaharga.com/httpd.txt
Perhaps at least I'd like to know how to check the last line of the file and delete it only if it's blank.
To delete empty last line you can use sed, it will delete it only if it's blank:
sed '${/^\s*$/d;}' file
Ok i made research with your file on my MacOS.
I created file new.conf by touch new.conf and then copied data from your file to it.
btw i checked file and didn't have extra empty line at the bottom of it.
I wrote script script.sh with following:
diff -b <(grep -B 2 -A 1 'bedrock.local' new.conf) new.conf > test.txt
sed -i.bak '1d;s/^>//' test.txt
It diffed what was needed and deleted first useless row and all > saving it to a new file test.txt
I checked again and no extra empty line was presented.
Additionaly i would suggest you to try and delete the extra line you have like this: sed -i.bak '$d' test.txt
And check a number of lines before and after sed = test.txt
Probably your text editor somehow added this extra line to your file. Try something else - nano for example or vi
I am looking for command or a shortcut to comment out multiple lines at once in Linux/Unix. For example, in a file we have 200 lines and I want to comment first 100 lines only. I know we can use # before every line to make it as comment. But is there a way to do all 100 lines at once and save time?
Sure, using sed:
sed -i '1,100s/^/# /' file
-i makes sed modify file in-place; 1,100 is the range of lines 1–100 in the file; “s” is for “substitute”; ^ is the beginning of the line; # is its replacement.